JP2008005204A - Video section determining device, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine a video section being an event from video contents only through video processing without using a sensor etc. <P>SOLUTION: Cut points of a plurality of cuts constituting the video contents and camera works of the respective cuts are stored. According to kinds of events, patterns of camera works of the respective events are stored. Based upon a telop, a cut being a starting point of an event is specified, and a pattern of camera works of a plurality of successive cuts including the cut is compared with a stored pattern of a prescribed camera work. According to the comparison result, a cut including a convergence point of the event is determined. Further, the convergence point of the event is corrected according to the state of the event and the number of kinds of camera works of the respective cuts. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for automatically assigning metadata indicating a video section to video content such as a television program, particularly a sports program.

For important actions (hereinafter referred to as “events”) in video content, metadata indicating the start point and convergence point of the video section may be attached. Specifically, the metadata is defined by the standard STD-B38 in ARIB (Association of Radio Industries Businesses) and the TV-Anytime forum.
When this metadata is used, for example, when the video content is a sports broadcast program such as baseball, if the metadata is given to a predetermined event, the viewer of the video content can, for example, only score the scene. You can watch it as a digest.

Also, for TV broadcaster program producers, if metadata is added to video content, the effort to extract video including events from the video content can be greatly reduced. Useful.
However, when such metadata indicating the video section is manually added, the person who performs the assignment must watch the video content and search for the video section to which the metadata should be added, which is very time-consuming. Requires a lot of effort.

Therefore, conventionally, various techniques for automatically adding metadata to video content instead of manually are used.
For example, there is an HDD / DVD recorder that has a function to automatically generate a digest video by detecting the peak of the audio level and using it as the start point of the video section of the event, and after a certain time has elapsed from that start point. .

Patent Document 1 below discloses a technique for attaching a sensor to each of a plurality of cameras and generating an index of video content using information on the shooting range of each camera obtained from the sensors. Specifically, when the pattern of the shooting range of the camera matches a pattern prepared in advance, an index is generated according to the matched pattern.
JP 2002-223375 A

However, as described above, when the convergence point is after the elapse of a certain time from the start point of the video section, since the convergence point of the video section is uniformly set after the elapse of the certain time, the video section before the event is completed. A convergence point may be determined. Conversely, redundant video may be included in the video section even though the event is completed.
For example, in the case of video content such as baseball, considering the demands of viewers and the like, the digest video such as a scoring scene due to a hit is originally intended to be a scene where the runner returns to the home base and the score is added. Nevertheless, the point in time before the runner's return to home is determined as the convergence point of the video section, and if only the digest video is viewed, it may be unclear whether or not the runner has returned.

Further, the technique described in Patent Document 1 is based on the premise that a parameter regarding the shooting range of the camera can be acquired from the sensor. For this reason, it takes time to install the sensor, and an index cannot be generated only by video processing.
Therefore, an object of the present invention is to provide a video section determination device, a video section determination method, and a video section determination program that can determine a video section with high accuracy only by video processing of video content.

  In order to solve the above-described problems, the present invention provides a video section determination device that determines a video section of a part of a video from video content including a plurality of cuts, and for each cut, the movement of a camera in the cut is determined. When the camera work storage means for storing the camera work shown and the judgment means for judging whether or not a series of camera work patterns of a plurality of consecutive cuts match a predetermined camera work pattern, And determining means for determining a predetermined time point determined according to the pattern as a boundary of the video section when the determination is made.

In video content, camera work in a video section that constitutes an event may have characteristics for each event type.
For example, when the video content is baseball, when the batter hits a home run, the camerawork draws a parabola while panning in order for the camera to follow the shot. On the other hand, when the batter strikes out, the camera work is almost fixed.

Focusing on such features, the type of event can be determined according to the content of the camera work.
In addition, the fact that each event type has a feature in the camera work of the event also means that the time when the event converges can be specified to some extent from the camera work.
For example, in the case of a home run, if the video section is to the point where the hit ball stands in, the end point of the camera work cut that draws a parabola may be set as the convergence point of the video section.

The video section determination apparatus of the present invention having the above-described configuration determines the type of event in the plurality of cuts by determining whether the camera work of the plurality of cuts of the video content matches a predetermined pattern. Therefore, the boundary of the video section can be determined with high accuracy according to the determination result.
Here, the video section determination device further includes cut point storage means for storing the start time and the end time of each cut in the video content, and the determination means is a target to be determined by the determination means. A predetermined cut determined according to the pattern may be selected from the cuts, and the determination may be performed by determining the predetermined time based on the start time and / or end time of the selected cut.

  Normally, when an event converges in a certain cut of video content, the cut is switched and another video is displayed. For example, in the case of baseball, when a batter hits a home run, after the hit ball stands in, the cut is switched to be a cut of the batter running on the base. That is, the fact that the cut is switched means that other situations or scenes are displayed.

Since the video section device of the present invention having the above-described configuration determines the boundary of the video section based on the start time and end time of the cut, the video section boundary is set so that redundant video is not included in the video section. It can be determined with high accuracy.
Further, the video section determination device further includes an acquisition unit that acquires start point information indicating a start point of the video section, and the determination unit includes a start point that includes a start point indicated by the acquired start point information. A start point cut specifying unit for specifying a cut from the cut point storage unit, and performing the determination with the plurality of consecutive cuts including the start point cut, and the determining unit is configured to start the selected cut or / In addition, a convergence point of the video section may be determined based on the end time, and the start point and the convergence point may be determined as the boundary.

Thereby, when the start point of the video section is known, the convergence point of the video section can be determined with high accuracy.
Further, the video section determination device further includes a telop storage unit that stores telop information in which information indicated by the telop displayed in the video of the video content is associated with the display time of the telop, The acquisition means may acquire a display time of a predetermined telop determined according to information indicated by the telop as the start point information.

In some cases, the status changed by the occurrence of an event is displayed by a telop. For example, when the video content is baseball, the score obtained by each team is displayed as a score telop. If the score is added, the content displayed by the telop is updated to indicate that the score has been added.
Therefore, it can be considered that an event has occurred with the update of the contents of the telop as a clue. Here, when the conventional technology for automatically detecting the contents of the telop from the video content without using the manual operation, the video section can be determined only by processing the video content.

By the way, depending on the situation at the time of the event occurrence, there is a case where it is desired to have different convergence points depending on the situation even for the same event.
For example, in the case of baseball, if the hit is in a situation where there is no runner on the base, the viewer can know whether the batter's hit is out or hit when it is caught by an outfielder, and more information is needed is not. That is, the point at which the hit ball is caught by the outfielder may be set as the event convergence point.

  On the other hand, if a runner is on the top of the scoring area, in addition to the information on whether the batter is out or hit, the viewer can score the team according to the number of runners and the progress of the runner. Information that is highly likely to have a significant impact on the outcome of the match, such as whether or not to be added, is also considered necessary. For example, it is desired that the event is converged until the hit ball is caught by the outfielder and received by the infielder or until the runner returns to the main base and the score is added.

Therefore, the video section determination device further includes event status storage means for storing event status information indicating the status of the event in the video content, and the determination means is indicated by the pattern and the event status information. The predetermined time point may be determined based on the situation and determined as the boundary.
Thereby, the boundary of a video section can be determined according to the situation at the time of event occurrence, and the accuracy of determination of the video section can be further improved.

  In addition, the video section determination device further includes telop storage means for storing telop information in which information indicated by the telop displayed in the video of the video content is associated with the display time of the telop, Event status calculation means for generating the event status information based on telop information, and storing the generated event status information in the event status storage means, and the determination means is calculated by the event status calculation means. The determination may be made using event status information stored in the event status storage means.

As a result, the status of the event can be acquired using the update of the contents of the telop as a clue.
By the way, since an event is an important action in video content, the number of types of camera work in the cut increases as the camera tries to take a moving video while the event occurs. There is. For example, when a batter hits the ball, the camera follows the hit ball, pans right, zooms in to capture the fielder that catches the hit ball, and the fielder who catches the hit ball returns to the infield Pan left to chase the ball. At this time, the number of types of camera work in the cut is three types of right pan, zoom IN, and left pan.

Then, after the event has converged, the cut changes, the motion of the video is reduced, and the number of types of camera work is also reduced. For example, the camera hits a hit with a fixed camera work with almost no camera movement. At this time, the number of types of camera work is one fixed type.
In this way, the number of camerawork types increases between cuts while an event occurs, and the number of camerawork types tends to decrease compared to the previous cut when the cuts change and the event converges There is.

Therefore, the camera work storage means further stores the number of camera parameters indicating the number of types of the camera work in each cut for each of the cuts, and the determination means includes the pattern and each of the determinations. The predetermined time point may be determined based on the increase / decrease in the number of camera parameters of the cut and determined as the boundary.
Thereby, the boundary of a video section can be determined according to the number of types of camera work, and the accuracy of determination of a video section can be further improved.

  In addition, processing for determining a video section of a part of video from video content constituted by a plurality of cuts is performed, and video section determination includes a storage unit that stores camera work indicating camera movement in the cut for each cut. A method for determining a video section used in an apparatus, wherein a series of camerawork patterns of a plurality of consecutive cuts are determined to match a determination step of determining whether or not a predetermined camerawork pattern matches. The video section determination method may include a determination step of determining a predetermined time determined according to the pattern as a boundary of the video section.

  In addition, a video section of a part of video from the video content is stored in a video section determination device including storage means for storing, for each cut, camera work indicating camera movement in the cut of video content configured by a plurality of cuts. It is a control program for performing the determining process, and it is determined that it matches the determination step for determining whether or not a series of camerawork patterns of a plurality of consecutive cuts match a predetermined camerawork pattern. The control program may include a determination step of determining a predetermined time point determined according to the pattern as a boundary of the video section.

<Embodiment>
Hereinafter, an embodiment of a video section determination device according to the present invention will be described with reference to the drawings.
<Overview>
In the following embodiment, a case where metadata is added to video content of a baseball program will be described as an example. In the following embodiment, an important action in a video is referred to as an “event”. Hereinafter, a process in which the video section determination device determines the start point and the convergence point of the video section constituting the event will be mainly described.

<Configuration>
1.1 Configuration FIG. 1 is a functional block diagram of a video section determination apparatus 1000 according to the present invention.
As shown in the figure, the video section determination device 1000 includes a video storage unit 101, a cut point detection unit 102, a cut point storage unit 103, a camera work detection unit 104, a camera work storage unit 105, and a camera work. Counting unit 106, telop detection unit 107, telop information storage unit 108, event occurrence detection unit 109, event occurrence information acquisition unit 110, event situation calculation unit 111, event situation storage unit 112, and event convergence point A specifying unit 113 and a metadata storage unit 114 are provided.

1.1.1 Video storage unit 101
The video storage unit 101 is a storage device such as a large capacity HDD (Hard Disk Drive) or a semiconductor memory. The video storage unit 101 stores, as digital data, video content compressed and encoded by a method such as MPEG-2 (Moving Picture Experts Group phase 2), MPEG-4, or H.264.

1.1.2 Cut point detection unit 102
The cut point detection unit 102 reads out the video content stored in the video storage unit 101, and automatically determines the point (cut point) at which the scene is switched by turning on / off the camera, switching the camera, or editing the video in the video. Detect. The detected cut point is stored in the cut point storage unit 103. Further, the detected cut point is output to the camera work detection unit 104. The detection of the cut point is performed by comparing adjacent frames, for example, by measuring the amount of change in the color change of the frame and determining whether the cut point is based on the difference in the amount of change. Although it is performed by a conventional technique such as searching for an image having significantly different image characteristics, the present invention is not limited to this.

1.1.3 Cut point storage unit 103
Based on the detection result of the cut point detection unit 102, the cut point storage unit 103 stores the cut identification number and the time of the start point and the end point of the cut change in association with each other as cut point information.
1.1.4 Camerawork detection unit 104
The camera work detection unit 104 reads the video content stored in the video storage unit 101 and detects the camera work in the video section divided by the cut points output by the cut point detection unit 102. The camera work detection unit 104 also detects the camera swing speed in the video section delimited by the cut points. The detected camera work and camera swing speed are stored in the camera work storage unit 105. The detection of camera work and camera swing speed is performed by a conventional technique such as detecting a change in a moving vector of a video, but is not limited thereto.

1.1.5 Camerawork storage unit 105
Based on the detection result of the camera work detection unit 104, the camera work storage unit 105 associates the camera work of each cut with the camera swing speed and stores it as camera work information.
1.1.6 Camerawork counting unit 106
The camera work counting unit 106 reads out the camera work of each cut from the camera work storage unit 105 and counts the number of types of camera work in each cut. The counted number of types of camera work is stored as the number of camera parameters, and is output to the event convergence point specifying unit 113. The process for counting the number of camera parameters will be described later.

1.1.7 Telop detection unit 107
The telop detection unit 107 reads the video content stored in the video storage unit 101 and reads the telop in the video content. The content of the read telop is recognized, and the recognized content and the telop display time are associated with each other and stored in the telop information storage unit 108 as telop information. As in this embodiment, when targeting a baseball program, the telop recognized by the telop detection unit 107 is, for example, a runner telop indicating a runner on the heel, or an SBO indicating a count number such as a strike, a ball, or an out There are a telop, a score telop indicating the score of each team, a ball speed telop indicating the speed of the ball when the pitcher pitches the batter. Each time the contents of each telop are displayed, the display time is stored. Note that telop detection uses a conventional technique such as determining the presence or absence of a strong edge between frames.

1.1.8 Telop information storage unit 108
The telop information storage unit 108 stores telop information based on the detection result of the telop detection unit 107. The telop information will be described later.
1.1.9 Event occurrence detection unit 109
The event occurrence detection unit 109 detects the occurrence of an event based on the telop information stored in the telop information storage unit 108. The occurrence time of the detected event is stored as event occurrence information in association with the event identification number. When the occurrence of an event is detected, the event occurrence information acquisition unit 110 is notified of the occurrence of the event.

1.1.10 Event occurrence information acquisition unit 110
When the event occurrence information acquisition unit 110 receives an event occurrence notification from the event occurrence detection unit 109, the event occurrence information acquisition unit 110 acquires event occurrence information from the event occurrence detection unit 109. The acquired event occurrence information is output to the event convergence point specifying unit 113 and the event status calculating unit 111.

1.1.11. Event status calculation unit 111
The event status calculation unit 111 includes telop information stored in the telop information storage unit 108, event occurrence information output from the event occurrence information acquisition unit 110, and camera work information stored in the camera work storage unit 105. Based on the above, event status information indicating the status of the event is calculated. The calculated event status information is stored in the event status storage unit 112. The event status is, for example, whether or not there is a runner on the heel at the time of the event, the number of batters that have advanced due to the occurrence of the event, whether the hit result of the batter is a hit or a general hit, etc. Information about the content. The event status information calculation process will be described later.

1.1.12. Event status storage unit 112
The event status storage unit 112 stores event status information calculated by the event status calculation unit 111.
1.1.13 Event convergence point identification unit 113
The event convergence point specifying unit 113 includes event occurrence information output from the event occurrence information acquisition unit 110, cut point information read from the cut point storage unit 103, camera work information read from the camera work storage unit 105, and camera work count. Based on the number of camera parameters output from the unit 106, event status information read from the event status storage unit 112, and the like, the convergence point of the video section that becomes the event is specified. The convergence point of the specified video section and the start point of the video section indicated in the event occurrence information are stored in the metadata storage unit 114. The process for specifying the convergence point will be described later.

1.1.14 Metadata storage unit 114
The metadata storage unit 114 stores the start point and the convergence point, which are the boundaries of the video section, in association with the event identification number as an event convergence point table.
<Data>
1.2 Data Next, each data used in the video section determination apparatus 1000 will be described.

1.2.1 Cut Point Information 200
FIG. 2 is a diagram showing the cut point information 200.
The cut point information 200 stored in the cut point storage unit 103 stores a cut identification number and video switching start time and end time in association with each other.
As shown in the figure, one record of the cut point information 200 (cut point information 200a, 200b,...) Includes a cut number 21, a start point 22, and an end point 23.

The cut number 21 is a number for identifying each cut included in the cut point information 200.
The start point 22 indicates the cut start time indicated by the cut number 21.
The end point 23 indicates the cut end time indicated by the cut number 21.
In this embodiment, the time is expressed as hours, minutes, seconds and frames. For example, 01: 12: 19: 00 represents 1 hour 12 minutes 19 seconds 00 frames.

1.2.2 Camerawork information 300
FIG. 3 is a diagram showing camera work information 300 and types of camera work.
FIG. 4A shows camera work information 300.
The camera work information 300 stored in the camera work storage unit 105 stores the camera work and the camera swing speed in each cut identified by the cut number 21 of the cut point information 200.

As shown in FIG. 6A, one record (camera work information 300a, 300b,...) Of the camera work information 300 includes a cut number 31, a camera work 32, and a camera swing speed 33. .
The cut number 31 is a number for identifying each cut. The value indicated by the cut number 31 corresponds to the value of the cut number 21 in the cut point information 200. That is, the cut with the cut number 21 value “100” and the cut with the cut number 31 value “100” indicate the same cut.

The camera work 32 indicates the camera work included in each cut. The camera work will be described in detail with reference to FIG.
When multiple types of camera work are executed simultaneously, for example, when right pan and zoom IN are executed simultaneously, in the description of this embodiment, each camera is represented as “right pan + zoom IN”. The work type is represented by adding “+”. Also, if a camera cut includes multiple camera works, for example, if the camera work is fixed and then pans left, add “→” to each camera work type, as in “Fix → Left Pan”. It shall be expressed as

  In addition, the cut with the cut number 31 “101” in FIG. 5A includes “right pan + zoom in” camera work and “left pan down” camera work. Then, when a plurality of camera works are included in a certain cut, it is assumed that each camera work is sequentially executed in the cut. For example, in the example in which the cut number 31 in FIG. 11A is “101”, this means that the camera work “left pan down” is executed after the camera work “right pan + zoom IN” is executed. It shall be.

  The camera swing speed 33 indicates the camera swing speed in each cut. The camera swing speed is stored, for example, as “fast” when the camera swing speed is faster than the threshold, and “slow” when the camera swing speed is slower than the threshold, based on a predetermined threshold. As in this embodiment, in the case of a baseball program, for example, the camera swing speed is set to “fast” when the camera follows the shot at the time of hit or the like, and the batter walks after the strike Then, the threshold value is determined so as to store the camera swing speed as “slow” when the camera captures the appearance when returning to the bench.

Next, the types of camera work will be described.
FIG. 4B is a table showing the types of camera work.
For example, the camera work is “fixed” means that the shooting direction of the camera is constant and there is no pan, tilt, or zoom.
Also, “almost fixed” camera work means that the shooting direction of the camera is substantially constant. Here, the fact that the shooting direction of the camera is almost constant means that the motion of the video is relatively small. Whether or not it is substantially constant may be determined based on, for example, whether or not the camera swing speed detected by the motion of the video is smaller than a predetermined threshold.

Also, the camera work “parabola” means that tilting up or tilting down or both is performed while panning the shooting direction of the camera. For example, in the case of a baseball program, the camera work serving as a parabola is used when the camera shoots a hit ball such as a home run or an outfield fly.
Other camera work such as panning, zooming, and tilting is as shown in FIG.

1.2.3 Ticker information 400
FIG. 4 is a diagram showing the telop information 400.
The telop information 400 stored in the telop information storage unit 108 includes the uphill runner telop 41, the SBO telop 42, the score telop 43, and the ball speed telop 44.
The heel runner telop 41 indicates the hour, minute, second and frame in which the telop indicating the runner on the heel is displayed along with the display contents. In the present embodiment, the display content is “1” for each heel when the runner is displayed on each heel and “0” when the terop is displayed when there is no runner. Each time a telop is displayed, its display time and display contents are stored. In the example of the figure, it is shown that at 01: 13: 35: 00, a telop indicating that there is no runner on 1st and 3rd bases and that there is a runner on 2nd base is displayed. Normally, a telop indicating whether or not there are runners on the 1st, 2nd, and 3rd bases is displayed at the same time. In other words, for example, it is not normally performed to display only the telop indicating whether or not there is a runner in the first base without displaying the telop indicating whether or not there is a runner in the second or third base. Therefore, as shown in the figure, the telop indicating the runners of the 1st, 2nd, and 3rd bases is updated at the same time (01: 13: 35: 00).

The SBO telop 42 indicates when the telop indicating the number of strikes, balls, and outs is displayed together with the display contents. “S” indicates a strike, “B” indicates a ball, and “O” indicates an out count. For example, as for the ball count, “0” is stored at 01: 11: 52: 00, and then “1” is stored at 01: 12: 49: 00.
The score telop 43 indicates when the telop indicating the score is displayed together with the display content. The display content is a score. In the example shown in the figure, a score telop indicating one point is displayed at 01: 13: 33: 00.

The ball speed telop 44 indicates when the display of the ball speed telop is updated together with the number of times the ball speed telop indicating the ball speed of the ball thrown by the pitcher is displayed.
1.2.4 Event occurrence information 500
FIG. 5 is a diagram showing the event occurrence information 500.
The event occurrence information 500 stored in the event occurrence detection unit 109 stores the event occurrence time together with a number for identifying the event.

As shown in the figure, one record of event occurrence information 500 (event occurrence information 500a, 500b,...) Includes an event ID 51 and an event occurrence time 52.
The event ID 51 is a number for identifying each event.
The event occurrence time 52 indicates at what hour / minute / second / frame.
1.2.5 Event status information 600
FIG. 6 is a diagram showing the event status information 600.

The event status information 600 stored in the event status storage unit 112 stores the status of each event. One record of the event status information 600 (event status information 600a, 600b,...) Includes an event ID 61, a pre-occurrence runner 62, a batter advance number 63, and an event type 64.
The event ID 61 is a number for identifying each event. The value indicated by event ID 61 corresponds to the value of event ID 51 of event occurrence information 500.

The pre-occurrence heel runner 62 indicates whether or not there is a runner on the heel at the event occurrence time. When there is a runner on the heel, the runner is “present”, and when there is no runner, the runner is “none”.
The batter advance number 63 indicates how much the batter has advanced due to the occurrence of the event. “No advance”, “1”, “2”, “3” and “main” are stored according to the number of advances made by the batter. For example, when the batter advance number 63 is “2”, it indicates that the batter has advanced to 2 km.

The event type 64 indicates the type of event. There are three types of events: “hit”, “general hit”, and “other”. “Hit” indicates that the batter has released a hit. “Most hit” indicates that the batter has retreated. “Other” indicates that the event type is other than “hit” or “general hit”.
1.2.6 700 camera parameters
FIG. 7 is a diagram showing the camera parameter number 700.

The camera parameter number 700 stored by the camera work counting unit 106 stores the number of types of camera work for each camera work. One record of the camera parameter number 700 (camera parameter number 700a, 700b,...) Includes a cut number 71, a camera work 72, and a camera parameter number 73.
The cut number 71 is a number for identifying each cut. The value indicated by the cut number 71 corresponds to the value of the cut number 21 in the cut point information 200.

The camera work 72 indicates the camera work included in each cut. Since details are the same as the camera work 32 of the camera work information 300, the description is omitted.
The camera parameter number 73 indicates the number of types of camera work. For example, when the camera work is “right pan + parabola → zoom in”, the camera work number 73 is “3” because there are three camera works: right pan, parabola, and zoom in. If the camera work is “fixed”, the camera work number 73 is “1” because the camera work is one fixed.

1.2.7 Event Convergence Point Table 800
FIG. 8 shows an event convergence point table 800.
The event convergence point table 800 stored in the metadata storage unit 114 stores the time of the convergence point of the video section for each event.
One record (event convergence point table 800a, 800b,...) Of the event convergence point table 800 includes an event ID 81 and an event convergence point 82.

The event ID 81 is a number for identifying each event. The value indicated by event ID 81 corresponds to the value of event ID 51 of event occurrence information 500.
The event convergence point 82 indicates the time at which each event converges.
<Operation>
1.3 Operation Next, the operation of the video section determination device will be described.

1.3.1 Operation of Video Section Determination Device FIG. 9 is a flowchart showing the operation of the video section determination device 1000.
As shown in the figure, the cut point detection unit 102 of the video section determination device 1000 reads video content from the video storage unit 101 and detects a cut point. The detected cut point is stored in the cut point storage unit 103 as cut point information 200 (step S901). Further, the detected cut point is output to the camera work detection unit 104.

The camera work detection unit 104 reads video content from the video storage unit 101 and detects the camera work of each cut. The detected camera work is stored in the camera work storage unit 105 as camera work information 300 (step S902).
Further, the telop detection unit 107 reads video content from the video storage unit 101, reads the telop in the video content, and stores it as telop information 400 in the telop information storage unit 108 (step S903).

  The event occurrence detection unit 109 performs processing for detecting the occurrence of an event based on the telop information 400 stored in the telop information storage unit 108, but until the event occurrence detection unit 109 detects the event (step 904: NO), sequential processing (step S901, step S902, step S903) such as cut point detection, camera work detection, telop detection, and the like are performed. The event occurrence detection process by the event occurrence detection unit 109 will be described in detail in “1.3.2 Event occurrence detection process” described later.

When the event occurrence detection unit 109 detects the occurrence of an event, the event occurrence detection unit 109 stores the detection result as event occurrence information 500, and notifies the event occurrence information acquisition unit 110 of the occurrence of the event (step S905).
When receiving the notification from the event occurrence detection unit 109, the event occurrence information acquisition unit 110 acquires the event occurrence information 500 stored in the event occurrence detection unit 109. The acquired event occurrence information 500 is output to the event convergence point identification unit 113 and the event status calculation unit 111.

  The event status calculation unit 111 calculates the event status based on the telop information 400 stored in the telop information storage unit 108 and the event occurrence information 500 output from the event occurrence information acquisition unit 110. The calculation result is stored in the event status storage unit 112 as event status information 600 (step S906). The calculation process by the event situation calculation unit 111 will be described in detail in “1.3.3 Event situation calculation process” described later.

  When the calculation process by the event situation calculation unit 111 is performed, the event convergence point specifying unit 113 determines the convergence point of the event indicated in the event occurrence information 500 received from the event occurrence information acquisition unit 110 (step S907). When the event convergence point is determined, the determined convergence point is stored in the metadata storage unit 114 in association with the event ID. The event convergence point determination process by the event situation calculation unit 111 will be described in detail in “1.3.4 Event convergence point determination process” described later.

When the event convergence point specifying unit 113 determines the event convergence point, the video section determination device 1000 repeatedly performs a series of processes from step S901 onward.
1.3.2 Event Occurrence Detection Processing Next, the event occurrence detection processing by the event occurrence detection unit 109 in step S904 will be described in detail.

FIG. 10 is a flowchart illustrating a process in which the event occurrence detection unit 109 detects the occurrence of an event.
In this embodiment, when the runner on the heel changes, when the outcount increases, or when a score is added, it is estimated that an event has occurred and the event occurrence time is determined.

  The event occurrence detection unit 109 will be described in detail below. The event occurrence detection unit 109 includes an item indicating the outcount of the uphill runner telop 41 and the SBO telop 42 in the telop information 400 stored in the telop information storage unit 108. It is sequentially determined whether or not the display content has been updated (step S1001, step S1002, and step S1003). When it is determined that there is no update (step S1001: NO, step S1002: NO, step S1003: NO), the event occurrence detection unit 109 determines that no event occurrence has been detected in step S904 (step S904: NO).

  On the other hand, when the uphill runner telop 41 is updated (step S1001: YES), the item indicating the outcount of the SBO telop 42 is updated (step S1002: YES), or the score telop 43 is updated (step S1003: YES), the event occurrence detection unit 109 selects the ball speed telop displayed most recently at the time when the updated telop is displayed among the ball speed telops 44 displayed before the time when the updated telop is displayed. Set the display time to the event occurrence time. At this time, an ID for identifying the event is also given and stored as event occurrence information 500 (step S1004). In step S904, it is determined that an event has been detected (step S904: YES).

  In the example of the telop information 400 shown in FIG. 4, since the score telop 43 is updated at 01: 13: 33: 00 (step S1003: YES), the time when the updated telop is displayed 01:13 : Of the ball speed telops 44 displayed before 33:00, the most recent ball speed telop display time 01: 12: 49: 00 at the time when the update telop was displayed is set as the event occurrence time. Set.

1.3.3 Event Status Calculation Processing Next, the event status calculation processing by the event status calculation unit 111 in step S906 will be described in detail.
The event status calculation unit 111 calculates the event status based on the telop information 400, the event occurrence information 500, and the camera work information 300, and updates the event status information 600.

FIG. 11 is a flowchart illustrating a process in which the event status calculation unit 111 calculates an event status.
The event status calculation unit 111 receives the event occurrence information 500 output from the event occurrence information acquisition unit 110 and acquires the event occurrence time. It is determined from the uphill runner telop 41 of the telop information 400 stored in the telop information storage unit 108 whether or not there is a runner on the uphill at the event occurrence time (step S1101).

Specifically, the event status calculation unit 111 detects a runner telop 41 displayed before the event occurrence time, the one closest to the event occurrence time. It is determined whether or not the detected telop indicates that a runner is present in at least one of the first, second, and third bases.
If the detected telop indicates that there is a runner on the heel, the event status calculation unit 111 determines that there is a runner on the heel at the event occurrence time (step S1101: YES). ). On the other hand, when the detected telop indicates that there is no runner on the heel, the event status calculation unit 111 determines that there is no runner on the heel at the event occurrence time (step S1101: NO).

  When the event situation calculation unit 111 determines that there is a runner on the heel at the event occurrence time (step S1101: YES), the event situation calculation unit 111 associates the event with the event ID 61 that identifies the event, and the pre-occurrence heel runner 61 of the event situation information 600 Is set to “Yes”. If it is determined that there is no runner on the heel at the event occurrence time (step S1101: NO), the pre-occurrence heel runner 61 is set to “none”.

The event situation calculation unit 111 refers to the SBO telop 42 of the telop information 400 after the setting of the pre-occurrence saddle runner 61, and determines whether or not the outcount has changed (step S1103).
Specifically, a telop indicating the out count of the SBO telop 42 between the display time of the ball speed telop that is the event occurrence time indicated by the event ID 61 of the ball speed telop 44 and the display time of the next subsequent ball speed telop. It is determined whether there is a change in the display content of.

If it is determined that there is a change (step S1103: YES), the batter advance number 63 is set to “none” in association with the event ID 61. Further, the event type 64 is set to “general hit” (step S1104). Thereafter, the process of step S1105 is performed.
If it is determined in step S1103 that there is no change (step S1103: NO), the process of step S1105 is performed.

After the process of step S1103, the event status calculation unit 111 determines whether or not the runner on the saddle has changed (step S1105).
Specifically, the uphill runner telop 41 of the telop information 400 is displayed between the time when the ball speed telop 44 becomes the event occurrence time indicated by the event ID 61 and the time when the next ball speed telop is displayed. It is determined whether or not the display content of the telop indicating the runner on the heel has changed.

If it is determined in step S1105 that there is no change in the runner on the heel (step S1105: NO), the process is terminated. In this case, when the batter advance number 63 and the event type 64 are not set, the event status calculation unit 111 sets the batter advance number 63 to “no advance” and the event type 64 to “others”. To "".
If it is determined in step S1105 that the runner on the heel has changed (step S1105: YES), the event status calculation unit 111 stores the cut points stored in the cut point storage unit 103 based on the event occurrence time. From the information 200, the cut number of the cut including the event occurrence time is acquired. The camera swing speed of the cut indicated by the acquired cut number or a predetermined cut subsequent to the cut is acquired from the camera swing speed 33 of the camera work information 300 stored in the camera work storage unit 105. It is determined whether or not the acquired camera swing speed is “fast” (step S1106).

Specifically, for example, a predetermined cut is set as the next cut following the cut including the event occurrence time, and it is determined whether the camera swing speed of the next cut is “fast”.
If the acquired camera swing speed is “fast” (step S1106: YES), the event type 64 is set to “hit” in association with the event ID. In addition, the batter advance number 63 is set as the event ID based on the updated content of the uphill runner telop 41 from the display time of the ball speed telop shown in the event occurrence time to the display time of the next subsequent ball speed telop. The settings are made in association with each other, and the process ends (step S1107).

Specifically, the batter advance number 63 is set according to which saddle the runner telop 41 indicates that there is a runner. For example, referring to the example of the telop information 400 in FIG. 4, the upper runner telop 41 indicates that there is a runner on the second base. Therefore, in this case, the batter advance number 63 is set to “2”.
If there are multiple runners indicating runners, the batter advance number 63 is set so that “1” is given the highest priority and “2” is given the highest priority according to the display contents. And For example, if the runner-up runner telop 41 indicates that there are runners on the first and second bases, the batter advance number 63 is set to “1 base” with priority on the base one. Similarly, when the runner-up runner telop 41 indicates that there are runners on the first and third bases, the batter advance number 63 is set to “1 base”. Also, if the runner-up telop 41 indicates that there are runners on the second and third bases, the batter advance number 63 is set to “2 bases” giving priority to the two bases.

If the acquired camera swing speed is not “fast” in step S1106 (step S1106: NO), the event type 64 is set to “other” in association with the event ID. In the same manner as described above, the batter advance number 63 is set in association with the event ID, and the process ends (step S1108).
1.3.4 Event Convergence Point Determination Process As described above, the video section determination device 1000 performs the event occurrence detection process and the event situation calculation process to update the event occurrence information 500 and the event situation information 600. .

The event convergence point specifying unit 113 acquires the event ID and the event occurrence time from the event occurrence information 500, and identifies a cut including the event occurrence time based on the cut point information 200. The identified cut is defined as the first cut. Cuts subsequent to the first cut are referred to as second cut, third cut, fourth cut,.
Further, the definition of each event up to the convergence point is defined as shown in FIG.

FIG. 12 is a diagram illustrating the definition of each event up to the convergence point.
For example, when comparing the event names “four balls”, “dead ball” and “three strikes”, the definition up to the convergence point of each event is “four balls” as shown in the figure. Until the pitch becomes a ball. In the case of “dead ball”, it is from when the pitcher starts pitching until the pitch hits the batter. Further, in the case of “three strikes”, it is from the pitcher starts pitching until the batter swings (or sees off).

At this time, the first cut of each event of “four balls”, “dead ball” and “three strikes” is that the pitcher starts throwing and throws the ball, the throw becomes a ball, the throw hits the batter, I'm shooting up to (or seeing off) and the camerawork is fixed.
The next second cut is often a cut taken with the batter walking up to the bench and taking a picture of the pitcher of the mound with the batter fixed in strike. For this reason, the camera swing speed in the cut is often slower than the camera swing speed in the cut that shot the hit ball.

In this way, camera work has a characteristic for each event. Its characteristics are shown in FIG.
FIG. 13 is a diagram illustrating an example of camera work characteristics of each event.
The first cut of each event is fixed. In this cut, an image of the pitcher and batter is displayed from a camera installed at the center of the stadium. On the other hand, the camera work and the camera swing speed after the second cut are characteristic for each event.

For example, the second cut of each event of “Three Strikes”, “Four Balls”, and “Dead Balls” is often taken from the camera near the 1st bench, and the shots of the batters after the third strike, the fourth ball, and the dead ball are taken. Is slow. Camera work is “almost fixed”, “left pan”, “right pan + zoom in”, “fixed → left pan”, and “right pan”.
On the other hand, each event of “hit”, “home run”, and “normal hit (fly)” includes a cut in which the camera follows the hit ball after the second cut, so the camera swing speed is high, and the camera work is “right pan” + Parabola → Zoom IN "or" Right Pan → Zoom IN ", etc.

Thus, focusing on the camera work characteristics of each event, the convergence point of each event is determined. Based on the definition of each event in FIG. 12, the convergence point in the case of “three strikes”, “four balls”, and “dead ball” is the end point of the first cut. The convergence point in the case of “hit”, “home run”, “normal hit (fly)” is assumed to be after the second cut.
The reason for determining the convergence point in this manner is that, in the case of “three strikes”, only the first cut image is sufficient to satisfy the definition of the convergence point. That is, after the second cut, there are many cases in which the batter walks back to the bench and the like, and so on. In view of the definition of the convergence point, the second and subsequent cuts are cuts that do not include the convergence point. Because.

On the other hand, as shown in FIG. 12, in the case of “hit” or the like, the definition of the convergence point should be a video section including the whereabouts of the hit ball. The end point of subsequent cuts is defined as the event convergence point.
For example, in the case of a home run, after the second cut, an image in which the hit ball stands in is included. In the case of a hit ball that draws a parabola, whether the hit ball is a hit or whether it is caught by a fielder. In order to include in the video section, the end point of the cut after the second cut is set as the convergence point of the event.

  In addition, for example, a hit that a hit ball flies to the outside field with a parabola and a hit (fly) that is similarly drawn with a fielder and draws a parabola are very similar in camera work. In this way, the camera work for each cut is approximate, but the contents of the event may be different. In this case, since the convergence point is defined for each event content, the time point that becomes the convergence point may be different depending on the event content.

  For example, whether or not the hit ball when the batter hits is captured by the fielder is taken in the second cut, and in the case of a hit, the batter is scrambling after the second cut. Assuming that the shot was taken in another cut, the viewer of the video content may want to know in more detail the end of the event, such as how far the batter has advanced. Therefore, so that the number of batters can be confirmed, the fielder catches the ball and sends it to the infielder, etc. near the base, and the second cut following the second cut, such as the cut that the infielder received the ball. The end point of the 3rd cut or the 4th cut is the convergence point of the event.

On the other hand, in the case of an ordinary hit (fly), the batter is out by ball catching by the fielder and the event converges, so the end point of the second cut is desired to be the event convergence point.
Further, there is a case where it is desired to determine a cut as a convergence point according to the event occurrence state.
For example, in the case of a hit, comparing the case where there is no runner on the base and the case where there is no runner on the base, if there is no runner on the base, the scene where the fielder catches the hit ball is the video section of the event do it. On the other hand, if there is a runner on the top, I want to include how much the top runner has advanced in the video section that will be the event. That is, there is a desire to make the event convergence point the time point that follows on the time axis from the convergence point when the runner is on the heel.

Therefore, in the present embodiment, it is assumed that the convergence point is determined according to the contents of the event and the event occurrence status in addition to the characteristics of the camera work. The video section determination apparatus 1000 uses the event status information 600 for the video section determination processing in order to accurately identify the contents of the event such as “hit” and “normal hit (fly)”.
FIG. 14 shows a process for determining the convergence point of each event by paying attention to the characteristics of the camera work for each event as described above.

FIG. 14 is a flowchart showing a process in which the event convergence point specifying unit 113 determines the event convergence point.
In the figure, first, according to the contents of the camera work, a group of “three strikes”, “four balls”, “dead balls” and a group of “hits”, “home run”, “normal hit (fly)” are identified. Then, the event status information 600 is used to identify “hit” or “fly” and determine a convergence point.

  Explaining the outline of the processing, the event convergence point specifying unit 113 acquires the first cut of the event identified by the event ID and the camera work of each cut after the first cut from the camera work information 300, and acquires the acquired camera work information 300. It is determined whether or not the cut camerawork pattern is the same as the predetermined camerawork pattern. According to the determination result, a cut for setting the convergence point of the event is determined, and the end point of the cut is set as the convergence point of the event.

Hereinafter, specific processing will be described with reference to FIG. 14. The event convergence point specifying unit 113 determines that the second cut is “almost fixed”, “left pan”, “right pan + zoom IN”, “fixed → left pan”. It is determined whether it is either (step S1401).
If it is one of the above cuts (step S1401: YES), it is determined whether the camera swing speed of the second cut is “slow” (step S1402).

If it is determined in step S1402 that the camera swing speed is “slow” (step S1402: YES), the end point of the first cut indicated in the cut point information 200 is determined as the event convergence point (step S1403). .
If it is determined in step S1401 that none of the above-described cuts (step S1401: NO), or if it is determined in step S1402 that the camera swing speed is not “slow” (step S1402: NO), the second It is determined whether the cut is “right pan” (step S1404).

If it is determined in step S1404 that the “right pan” is selected (step S1404: YES), it is determined whether the camera swing speed of the second cut is “slow” (step S1405).
If it is determined in step S1405 that the camera swing speed is “slow” (step S1405: YES), the end point of the first cut indicated in the cut point information 200 is determined as the event convergence point (step S1403). .

Through the processes from step S1401 to step S1405, the group of “three strikes”, “four balls”, and “dead balls” can be identified.
Hereinafter, a process of determining a convergence point in accordance with the contents of an event for the groups of “hit”, “home run”, and “normal hit (fly)” will be described following the above process.
Returning to the description of the process of the event convergence point identification unit 113, the event convergence point identification unit 113 determines that the camera swing speed is not “slow” in step S1405 (step S1405: NO), event status information. Referring to 600 pre-occurrence saddle runner 61 and event type 64, it is determined whether the event is a general hit without a saddle runner. That is, it is determined whether the pre-occurrence runner 61 is “none” and the event type 64 is “general hit” (step S1406).

In step S1406, when it is determined that the hitting without a runner-up runner (step S1406: YES), the end point of the third cut indicated in the cut point information 200 is determined as the event convergence point (step S1407). .
If it is determined in step S1406 that there is no extra hit without a runner-up runner (step S1406: NO), the event convergence point specifying unit 113 sets the pre-occurrence runner 61 and the event type 64 of the event status information 600. Referring to the event, it is determined whether the event is a hit with a runner-up. That is, it is determined whether the pre-occurrence runner 61 is “Yes” and the event type 64 is “Hit” (step S1408).

In step S1408, if it is determined that the hit is a runner-up hit (step S1408: YES), the end point of the fourth cut indicated in the cut point information 200 is determined as the event convergence point (step S1409). .
In addition, when it is determined in step S1404 that the second cut is not “right pan” (step S1404: NO), or in step S1408, it is determined that the hit is not a lift runner (step S1408: NO). The event convergence point specifying unit 113 determines whether the second cut is “right pan + parabola → zoom in” (step S1410).

  If it is determined in step S1410 that the second cut is “right pan + parabola → zoom in” (step S1410: YES), the event convergence point specifying unit 113 sets the batter advance number 63 in the event status information 600 and With reference to the event type 64, it is determined whether the event is a hit of 2 strokes or more. That is, it is determined whether the batter advance number 63 is “2”, “3” or “main” and the event type 64 is “hit” (step S1411).

If it is determined in step S1411 that the hit is 2 hits or more (step S1411: YES), the end point of the fourth cut indicated in the cut point information 200 is determined as the event convergence point (step S1409).
If it is determined in step S1410 that the second cut is not “right pan + parabola → zoom in” (step S1410: NO), or if it is determined in step S1411 that the second cut is not a hit of two or more strokes ( In step S1411, NO, the event convergence point specifying unit 113 determines whether the second cut is “right pan → zoom in” (step S1412).

If it is determined in step S1412 that the second cut is “right pan → zoom in” (step S1412: YES), the event convergence point specifying unit 113 determines that the event condition information 600 before the occurrence runner 61 and the event Referring to the type 64, it is determined whether the event is a hit with a lift runner (step S1413).
In step S1413, when it is determined that the hit is a runner-up hit (step S1413: YES), the end point of the fourth cut indicated in the cut point information 200 is determined as the event convergence point (step S1409). .

  If it is determined in step S1412 that the second cut is not “right pan → zoom in” (step S1412: NO), or if it is determined in step S1413 that it is not a hit with a runner-up runner (step S1412) (S1413: NO), the event convergence point specifying unit 113 determines whether or not “parabolic” is included in the camera work of the cut after the second cut (step S1414).

In step S1414, when it is determined that “parabolic” is included in the camera work of the cut after the second cut (step S1414: YES), the event convergence point specifying unit 113 displays the cut point information 200 and the camera work information 300. Referring to, the end point of the last cut including “parabola” is determined as the event convergence point (step S1415).
On the other hand, when it is determined in step S1414 that the “parabola” is not included (step S1414: NO), the event convergence point specifying unit 113 determines the end point of the second cut indicated in the cut point information 200 as the event. The convergence point is determined (step S1416).

The video section determination apparatus 1000 according to the present invention determines the convergence point according to the contents of the event and the event occurrence state by the processing in steps S1406 to S1416 described above.
For example, in the case of a hit with a runner-up runner or a hit of two or more strokes (step S1408: YES, step S1411: YES, step S1413: YES), whether the score has been added and how far the batter has progressed Is included in the video section, the cut that becomes the convergence point of the video section is a cut after the second cut.

  In addition, in step S1414, when the “parabolic” is included in the camera work of the cut after the second cut (step S1414: YES), it is often a hit (fly) or a hit where the batter advances to one base. Therefore, in order to follow the definition up to the convergence point of the event, the end of the last cut that draws a parabola, which has a high possibility of being cut after chasing a hit ball, is set as the convergence point of the video section.

Regarding the processing of the event convergence point specifying unit 113, the determination points of the convergence point are illustrated in FIG.
FIG. 15 illustrates a process for determining a convergence point of an event based on camera work characteristics.
In the figure, a convergence point 1501 illustrates the difference in convergence point for each camera work. The start point of the video section, that is, the occurrence time of each event indicated in the event occurrence information 500 is indicated as “start point”.

The camera work 1502 particularly indicates the type of camera work in the second cut.
In the camera work 1502, the second cut camera work is divided into three groups. That is, the group (A) (step S1401: YES or step S1404: YES) in which the second cut is “almost fixed”, “left pan”, “right pan + zoom IN”, “fixed → left pan”, “right pan”, Group (B) whose second cut is “right pan + parabola → zoom in” (step S1410: YES) and group (C) whose second cut is “right pan → zoom in” (step S1412: YES) These are divided into three groups. Note that the camera swing speed of each camera work belonging to the group (A) is “slow” (step S1401: YES and step S1402: YES, or step S1404: YES and step S1405: YES). As shown in FIG. 13, the camerawork pattern of the group (A) is a camerawork pattern mainly in the case of “three strikes”, “four balls”, and “dead balls”. The camera work patterns of the group (B) and the group (C) are mainly camera work patterns in the case of “hit”, “home run”, and “general hit”.

Such a convergence point of each group is indicated by a convergence point 1501.
As indicated by the convergence point 1501, in the case of the camera work group of group (A), the event convergence point specifying unit 113 determines the end point of the first cut as the event convergence point (step S1403).
If the event content is a hit of 2 hits or more in group (B) (step S1411: YES), the end point of the fourth cut is determined as the event convergence point (step S1409). On the other hand, if the hit is not two hits or more in the group (B) (step S1411: NO), for example, the end point of the second cut is determined as the event convergence point according to the above-described processing (step S1416).

  If the event content is a hit with a runner in group (C) (step S1413: YES), the end point of the fourth cut is determined as the event convergence point (step S1409). On the other hand, if it is not a hit with a runner in group (C) (step S1413: NO), for example, the end point of the second cut is determined as the event convergence point according to the above-described process (step S1416).

1.3.5 Modified Example of Event Convergence Point Determination Process In the above “1.3.4 Event convergence point determination process”, the event convergence point is determined from the camera work of each cut after the event occurs. In addition to camera work, the convergence point is determined according to the event occurrence status. By determining the convergence point according to the event occurrence state, each event is accurately identified and the video section is determined.

In addition to this, in order to appropriately determine the convergence point of each event, the event convergence point may be determined using the camera parameter number 700 stored in the camera work counting unit 106.
In general, when an event occurs in video content, the number of camerawork types in a cut increases if the video is moving, and then the number of camerawork types decreases when the event converges. Relatively many.

  For example, in the case of a baseball program, when the batter hits the ball, it becomes camera work chasing the hit ball, and the number of types of camera work increases. As a result, the motion of the video is reduced and the number of types of camera work is reduced. In other words, a cut in which the number of types of camera work is smaller than the immediately preceding cut is relatively often a video that shows a situation after the event has converged and has a small video motion.

Therefore, by determining the event convergence point in accordance with the increase or decrease in the number of types of camera work for each cut, it is possible to determine a video section that becomes an event more appropriately.
In this embodiment, in addition to the increase / decrease in the number of types of camera work, the event convergence point is further determined according to the content of the event.
Specifically, a cut in which the number of types of camera work in the event has decreased from the previous cut is determined as a cut after the event has converged, and if the event content is “general hit”, the event has converged. The end point of the cut immediately before the cut in which the number of types of camera work has been reduced is set as the event convergence point so that the situation after this is not included in the video section.

On the other hand, if the content of the event is “hit”, etc., the number of types of camera work has been reduced to include in the video section until the event has converged, such as how far the runner has advanced. The end point of the subsequent cut is set as the event convergence point.
FIG. 16 is a flowchart illustrating processing in which the event convergence point specifying unit 113 determines an event convergence point using the number of camera parameters.

In the figure, the processing of step S1401, step S1402, step S1403, step S1404, and step S1405 is the same as that shown with reference numerals in FIG.
As illustrated in FIG. 16, the event convergence point specifying unit 113 determines in step S1404 that the second cut is not “right pan” (step S1404: NO), or in step S1405, the camera swing speed is “ If it is determined that it is not “slow” (step S1405: NO), it is determined whether or not the number of camera parameters has decreased after the second cut (step S1606).

If it is determined in step S1606 that the number of camera parameters has decreased (step S1606: YES), the event convergence point identifying unit 113 refers to the batter advance number 63 and the event type 64 in the event status information 600, and It is determined whether or not the event is a hit of 2 strokes or more (step S1607).
If it is determined in step S1607 that the hit is 2 hits or more (step S1607: YES), the end point of the most recent cut following the cut with the reduced number of camera parameters is determined as the event convergence point (step S1607). S1608).

If it is determined in step S1607 that the hit is not more than two strokes (step S1607: NO), the end point of the cut immediately before the cut with the reduced number of camera parameters is determined as the event convergence point (step S1607). S1609).
If it is determined in step S1606 that the number of camera parameters does not decrease after the second cut (step S1606: NO), the event convergence point specifying unit 113 sets the end point of the second cut as the event convergence point. Determination is made (step S1610).

Among the processes described above, step S1606 starts the execution of the process for determining the event convergence point in accordance with the increase or decrease in the number of types of camera work for each cut.
In step S1607, step S1608, and step S1609, in addition to the increase / decrease in the number of types of camera work, the convergence point of the event is determined according to the content of the event. In this embodiment, when the content of the event is a hit of 2 or more hits, the cut that becomes the convergence point is controlled to be the subsequent cut, so that the situation after the event convergence is also included in the video section. A cut that becomes a convergence point can be determined so as to be included.

  The above process will be described using the example of the camera parameter number 700 in FIG. In the example of FIG. 7, the process of step S <b> 1607 in which the cut with the value “133” of the cut number 71 is the first cut and the event content indicated by the video content in the subsequent series of cuts is, for example, “general hit”. In step S1607, it is determined that the hit is not more than two strokes (step S1607: NO), that is, the end point of the cut immediately before the cut in which the number of camera parameters is decreased is determined as the event convergence point (step S1609).

At this time, when the second cut whose cut number 71 is “134” and the third cut whose cut number 71 is “135” are compared, the type of camera work indicated by the camera parameter number 73 is The number decreases from the second cut to the third cut. That is, the number of camera parameters decreases after the second cut (step S1606: YES).
Furthermore, since the content of the event shown in the cut is not a hit of 2 strokes or more (step S1607: NO), the cut immediately before the cut in which the number of camera parameters is decreased, that is, the cut immediately before the third cut. The end point of a certain second cut is determined as the event convergence point (step S1609).

  Explaining with other examples as well, in the example of FIG. 7, the cut with the value “151” of the cut number 71 is the first cut, and the content of the event indicated by the video content in the subsequent series of cuts is, for example, 3 strokes. In the process of step S1607, such as a hit, what is determined as a hit of two or more hits (step S1607: YES), that is, the end point of the most recent cut following the cut with the reduced number of camera parameters is the event. It is assumed that the convergence point is determined (step S1608).

At this time, when the second cut with the cut number 71 value “152” and the third cut with the cut number 71 value “153” are compared, the type of camera work indicated by the camera parameter number 73 is The number decreases from the second cut to the third cut. That is, the number of camera parameters decreases after the second cut (step S1606: YES).
Furthermore, since the content shown in the cut is a hit of 2 strokes or more (step S1607: YES), the most recent cut following the cut with the reduced number of camera parameters, that is, the most recent cut following the third cut. The end point of the fourth cut, which is a cut, is determined as the event convergence point (step S1608).
<Supplement>
1.4 Supplement As described above, the video section determination device according to the present invention has been described based on the embodiment. However, the video section can be modified as follows, and the present invention is applicable to the video section shown in the above embodiment. Of course, it is not limited to the determination device.
(1) In the above example, the processing in the case where the content of the video content is a baseball program has been described. However, the content of the video content is not limited to baseball, and may be other content. The video section can be determined based on the feature of the work. For example, the present invention can be applied to a golf program.

When the video content is a golf program, the features of camera work are, for example, as shown in FIG.
As shown in the figure, the contents of the event include “tee shot (first hit)”, “approach shot (after the second hit)”, “pat (success)” “pat (failure)”, “bunker shot”, etc. is there.

The camera work for each event is a cut with little movement in the image, such as “almost fixed” or “fixed” at the beginning. After the player hits the ball, the “parabola” or “ The cut that follows the player's hit ball continues with "Zoom IN".
If the put is successful, the event's convergence point is set so that the cut where the ball is cupped in is the event's convergence point, and if the put is unsuccessful, the reaction of the player's put failure is included in the event video section. It will be defined.

  In addition, in the same figure, the camera work enclosed in parentheses indicates a cut that is not included in the video section that becomes the event. For example, in the case of “tee shot (first hit)”, the third cut “parabolic → right pan + zoom in” is the event video interval, and the fourth cut “almost fixed” is the event video interval. Not included. That is, in this case, the event convergence point specifying unit 113 determines the end point of the third cut as the event convergence point.

Based on the camerawork characteristics of each event, the event convergence point specifying unit 113 becomes the event convergence point by performing the processing described in the above-mentioned “1.3.4 Event convergence point determination processing”. The cut is specified, and the end point of the cut is determined as the event convergence point.
In addition, it can be applied to American football, rugby, etc. by extracting camerawork features. For example, in American football, the camerawork is “fixed” or “almost fixed” until the player on the line throws the ball to QB (quarterback). When QB catches the ball, the camerawork pans, When the movement of the player is small and the movement of the camera is small, the event convergence point specifying unit 113 performs a cut such as “fixed → pan”, “fixed → right pan → left pan”, etc. One cut. Cuts before and after the cut with the reduced number of camera parameters, or cuts with the reduced number of camera parameters, cuts where the camera work is “almost fixed” after the second cut, cuts where the camera swing speed is “slow”, etc. Let the end point of be the convergence point. Thereby, one play can be determined as a video section.
(2) In the above-described embodiment, the video section determination device 1000 has been described as the event occurrence detection unit 109 updating the event occurrence information 500 by detecting the occurrence of an event based on the telop information 400. However, the occurrence of the event is not limited to the case based on the telop information 400. In addition, for example, it may be input from the outside without being based on video content.
(3) In the above-described embodiment, the event occurrence time that is the start point of the video segment is first given, and the video segment is determined by determining the convergence point of the event. However, the present invention is not limited to this. The convergence point may be determined in advance, and the start point of the event may be determined based on each camera work.

In addition, not only when either the start point or the convergence point is fixed, the camera work pattern of a series of continuous cuts is compared with a predetermined pattern, and a predetermined time point is determined according to the pattern for comparison. It may be a starting point and a convergence point.
For example, when the camera work of a series of cuts is the same as a predetermined pattern, the start point of the series of cuts is the time when the event occurs, and the end point of the series of cuts is the convergence point of the event It is good.

In the above-described embodiment, the processing for detecting the occurrence of an event while performing detection of a cut point, detection of camera work, detection of a telop, and the like has been described (steps S901, S902, S903, However, the present invention is not limited to this, and the occurrence of an event may be detected after the process of detecting all cut points and the like from the video content is completed. Moreover, although the detection process of these cut points etc. was demonstrated as what a video area determination apparatus performs by the cut point detection part 102 grade | etc., It does not necessarily need to be performed by a video area determination apparatus. For example, the cut point detection or the like is performed in advance by another device or the like, and the cut point information 200 or the like is acquired from the other device and stored in the cut point storage unit 103 or the like. The convergence point specifying unit 113 can determine the video section. That is, the functional block that detects from the video content, such as the cut point detection unit 102, is not an essential component of the video section determination device.
(4) Specifically, each of the above devices is a computer system including a microprocessor, a ROM, a RAM, a hard disk unit, a display unit, a keyboard, a mouse, and the like. A computer program is stored in the RAM or the hard disk unit. Each device achieves its function by the microprocessor operating according to the computer program. Here, the computer program is configured by combining a plurality of instruction codes indicating instructions for the computer in order to achieve a predetermined function.
(5) A part or all of the constituent elements constituting each of the above-described devices may be constituted by one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, ROM, RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.
(6) A part or all of the constituent elements constituting each of the above devices may be configured as an IC card that can be attached to and detached from each device or a single module. The IC card or the module is a computer system including a microprocessor, a ROM, a RAM, and the like. The IC card or the module may include the super multifunctional LSI described above. The IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.
(7) The present invention may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.

  The present invention also provides a computer-readable recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray Disc). ), Recorded in a semiconductor memory or the like. Further, the present invention may be the computer program or the digital signal recorded on these recording media.

Further, the present invention may transmit the computer program or the digital signal via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.
The present invention may be a computer system including a microprocessor and a memory, wherein the memory stores the computer program, and the microprocessor operates according to the computer program.

In addition, the program or the digital signal is recorded on the recording medium and transferred, or the program or the digital signal is transferred via the network or the like, and is executed by another independent computer system. It is good.
(8) The above embodiment and the above modifications may be combined.

  According to the present invention, it is possible to appropriately identify the convergence point of an event and determine a video section that becomes an event. By automatically and efficiently determining a video section that is an important event, metadata that can realize digest viewing and highlight viewing can be automatically assigned. Therefore, the present invention can be applied to a metadata providing device, a digest playback device, and the like.

The functional block diagram of the image area determination apparatus 1000 of this invention. The figure which shows the cut point information 200. FIG. The figure which shows the camera work information 300 and the kind of camera work. The figure which shows the telop information 400. The figure which shows the event generation information 500. FIG. The figure which shows the event status information 600. FIG. The figure which shows the camera parameter number 700. FIG. The figure which shows the event convergence point table. The flowchart which shows operation | movement of the video area determination apparatus 1000. The flowchart which shows the process which the event generation | occurrence | production detection part 109 detects generation | occurrence | production of an event. The flowchart which shows the process which the event condition calculation part 111 calculates an event condition. The figure which shows the definition to the convergence point of each event. The figure which showed an example of the characteristic of the camera work of each event. The flowchart which shows the process in which the event convergence point specific | specification part 113 determines the convergence point of an event. The figure which showed the process which determines the convergence point of an event based on the characteristic of camera work. The flowchart which shows the process in which the event convergence point specific | specification part 113 determines the convergence point of an event using the number of camera parameters. The figure which showed the characteristic of the camera work in case a video content is a golf program.

Explanation of symbols

101 Video storage unit 102 Cut point detection unit 103 Cut point storage unit 104 Camera work detection unit 105 Camera work storage unit 106 Camera work counting unit 107 Telop detection unit 108 Telop information storage unit 109 Event generation detection unit 110 Event generation information acquisition unit 111 Event situation calculation unit 112 Event situation storage unit 113 Event convergence point identification unit 114 Metadata storage unit

Claims (9)

A video section determination device that determines a video section of a part of a video from video content composed of a plurality of cuts,
Camera work storage means for storing camera work indicating the movement of the camera in the cut for each cut;
A judging means for judging whether or not a series of camerawork patterns of a plurality of continuous cuts match a predetermined camerawork pattern;
A video section determination apparatus comprising: a determination unit that determines a predetermined time point determined according to the pattern as a boundary of a video section when it is determined that they match. The video section determination device further includes:
In the video content, comprising cut point storage means for storing the start time and end time of each cut,
The determination unit selects a predetermined cut determined according to the pattern from each cut that is a target of determination by the determination unit, and the predetermined time point based on a start time and / or end time of the selected cut. The video section determination device according to claim 1, wherein the determination is performed by defining The video section determination device further includes:
An acquisition means for acquiring start point information indicating a start point of the video section;
The determination unit includes a start point cut specifying unit that specifies a start point cut including the start point indicated in the acquired start point information from the cut point storage unit,
Making the determination with the plurality of consecutive cuts including the starting point cut;
The determination means determines a convergence point of the video section based on a start time and / or an end time of the selected cut, and determines the start point and the convergence point as the boundary. 2. The video section determining device according to 2. The video section determination device further includes:
Telop storage means for storing telop information in which information indicated by the telop displayed in the video of the video content is associated with the display time of the telop,
The video section determination device according to claim 3, wherein the acquisition unit acquires a display time of a predetermined telop determined according to information indicated by the telop as the start point information. The video section determination device further includes:
Event status storage means for storing event status information indicating the status of events in video content,
2. The video section determining apparatus according to claim 1, wherein the determining unit determines the predetermined time point and determines the boundary as the boundary based on the situation indicated by the pattern and the event situation information. The video section determination device further includes:
Telop storage means for storing telop information in which information indicated by the telop displayed in the video of the video content is associated with the display time of the telop;
An event situation calculation means for generating the event situation information based on the telop information and storing it in the event situation storage means,
6. The video section determination device according to claim 5, wherein the determination unit performs the determination using event status information calculated by the event status calculation unit and stored in the event status storage unit. The camera work storage means further includes:
Stores the number of camera parameters indicating the number of types of the camera work for each cut,
The video section determination according to claim 1, wherein the determination unit determines the predetermined time point and determines the boundary as the boundary based on the pattern and an increase or decrease in the number of camera parameters of each cut for the determination. apparatus. In a video section determination device comprising a storage means for storing a camera work indicating the movement of a camera in a cut for each cut, by performing processing for determining a video section of a part of the video from video content constituted by a plurality of cuts A video segment determination method used,
A determination step of determining whether or not a series of camerawork patterns of a plurality of consecutive cuts match a predetermined camerawork pattern;
A determination step of determining a predetermined time point determined according to the pattern as a boundary of the image interval when it is determined that they match. A video section of a part of video is determined from the video content in a video section determination device including a storage unit that stores, for each cut, camera work indicating camera movement in the cut of video content configured by a plurality of cuts. A control program for performing processing,
A determination step of determining whether or not a series of camerawork patterns of a plurality of consecutive cuts match a predetermined camerawork pattern;
And a determining step of determining a predetermined time point determined according to the pattern as a boundary of the video section when it is determined that the two match.

Images