JP2007124368A - Segment metadata creation device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a segment metadata creation device for making it easy to create segment metadata. <P>SOLUTION: A video/voice input part 101 inputs an animation with a time code which is a creation object of the segment metadata. The creation schedule input part 105 inputs segment metadata creation schedule information including video content information to comprise the segment metadata. The creation schedule information is displayed together with the animation. A start point/end point input part 106 being an endpoint input means inputs at least either a start point or an end point of the segment metadata as a segment endpoint. The time code of the segment endpoint is caused to correspond to the video content of the segment metadata creation schedule so as to create the segment metadata. Verification data identifying the segment metadata is displayed so as to support a search operation of the segment endpoint. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a segment metadata creating apparatus and a segment metadata creating method for adding segment metadata to a moving image.

  With the development of various broadcasting technologies against the background of the development of information and communication technologies, the start of server-type broadcasting is scheduled. In the server-type broadcasting, a service for storing and utilizing broadcast video and information and video and information acquired from communication in a receiver is performed. As one of the services, digest viewing that uses segment metadata, which is information about the section of the video to be broadcast, to watch only a part of the stored video, or to extract and watch only the highlights of the program Etc. are expected to be performed.

  Here, the metadata is not data itself but data describing information about the data. The segment metadata is section metadata when a moving image is divided into a plurality of sections. The segment metadata includes information on the content of the image in the target section (also referred to as semantic metadata, for example, a segment title), and information on the start time and end time of the target section (also referred to as section metadata).

Conventionally, in the technology for creating segment metadata, different metadata contents are assigned to each key of a key input device. The creator operates each key while referring to the moving image when an image of metadata content corresponding to the key is displayed. By this key input, metadata with time code is created (see, for example, Patent Document 1).
JP 2005-222329 A

  However, the conventional segment metadata creation apparatus described above has the following problems.

  In the prior art, metadata to be created by the creator is stored, and metadata is created depending on the storage. The creator must store the metadata to be created so that there is no omission of creation of the metadata, and therefore there is a problem that the creation work is not easy. Furthermore, in the prior art, the creator needs to determine the association between the content of the metadata assigned to the key and the moving image. For this reason, if the creator does not understand and memorize the faces and features of the person appearing in the moving image, key input cannot be performed, and there is a problem that the creation work is not easy.

  In the prior art, the number of keys to which the contents of metadata are allocated increases in proportion to the increase in the types of metadata. When the number of types of metadata increases, there is a problem that the task of searching for a desired key becomes difficult, and the workload and time of the task increase accordingly.

  The present invention has been made to solve the conventional problems, and an object of the present invention is to provide a segment metadata creating apparatus and a segment metadata creating method that make it easy to create segment metadata.

  The segment metadata creation device of the present invention is a device for creating segment metadata representing information of a section of a moving image, and includes segment metadata creation schedule information including video content information that should constitute segment metadata to be created. A creation schedule input means for inputting; a display means for displaying the video and the segment metadata creation schedule information; and at least one of a start point and an end point of segment metadata corresponding to the segment metadata creation schedule information as a segment End point input means for inputting as an end point, and segment metadata generation means for generating created segment metadata by associating the time code of the input segment end point with the video content information of the segment metadata creation schedule information And the video and the segment metadata A storage means for storing data created schedule information and said created segments meta data.

  With this configuration, video content information that should constitute segment metadata to be created is input as segment metadata creation schedule information, and the creation schedule information is displayed. Even if the creator does not memorize all the contents of the moving image, it is only necessary to input the segment end point by looking at the creation schedule information, and therefore it becomes easy to create the segment metadata.

  In the segment metadata creation device of the present invention, the segment metadata creation schedule information includes confirmation data that is information identifying the segment video to which the segment metadata is attached together with the video content information, The confirmation data is composed of one or more of a still image, a moving image, a sound, and a character string, and is output to support the work of searching for the segment end point from the moving image.

  With this configuration, confirmation data is displayed together with video content information. The confirmation data is information for identifying the segment video to which the segment metadata is attached, and characterizes the target segment video. Since the confirmation data is output, the creator can easily search for the segment end points (start point and end point) while viewing the confirmation data. For example, the face and characteristics of a person appearing in the segment video are displayed as confirmation data. As a result, the creator can create segment metadata even if he / she does not understand all the faces and features of the person appearing in the video. In this way, segment metadata can be easily created.

  In the segment metadata creating apparatus of the present invention, the creation schedule input means inputs segment metadata creation schedule information of a plurality of types of segment metadata, and the end point input means stores the plurality of types of segment metadata. Each of the segment end points is input, the segment metadata generation unit generates each of the plurality of types of created segment metadata, and the display unit includes the plurality of types of segment metadata generation schedule information and the generation Segmented segment metadata by type.

  With this configuration, a plurality of types of segment metadata can be displayed for each data type, and a plurality of types of segment metadata can be created.

  In the segment metadata creating apparatus of the present invention, the display means further displays the created segment metadata generated by the segment metadata generating means in time series.

  With this configuration, it is possible to easily select the segment metadata to be confirmed after the segment metadata is created.

  Further, in the segment metadata creating apparatus of the present invention, the creation schedule input means further inputs scheduled appearance order information of a plurality of segment videos scheduled to be provided with a plurality of segment metadata, and the segment metadata The generating means updates the display of the display means with the segment metadata creation schedule information of the segment metadata to be created next based on the appearance schedule order information.

  With this configuration, it is possible to display the creation schedule information of the segment metadata to be created next. Typically, when one segment metadata is generated, the display means is controlled by the segment metadata generation means so as to display the creation schedule information of the segment metadata to be created next. The creator inputs the segment end points of the segment metadata to be created that are sequentially displayed. Therefore, it becomes easy to create segment metadata.

  Further, in the segment metadata creating apparatus of the present invention, the creation schedule input means further inputs information on the number of times of appearance of a plurality of segment videos scheduled to be provided with a plurality of segment metadata, and the segment metadata The generation means updates the display of the display means with the segment metadata creation schedule information of the segment metadata to be created next based on the expected appearance frequency information.

  With this configuration, it is possible to display the creation schedule information of the segment metadata to be created next. Typically, when one segment metadata is generated, the display means is controlled by the segment metadata generation means so as to display the creation schedule information of the segment metadata to be created next. The creator inputs the segment end points of the segment metadata to be created that are sequentially displayed. Therefore, it becomes easy to create segment metadata.

  Further, the segment metadata creating apparatus of the present invention includes a cut point detecting unit that detects a time code when a video feature amount changes from a moving image by a threshold value or more as a cut point time code, and the segment metadata generating unit Detects the segment end point by determining whether the cut point is the segment end point based on the segment metadata creation schedule information and the moving image, and detects the time code of the detected segment end point The segment metadata is automatically generated in association with the video content information.

  With this configuration, an automatic segment metadata generation function can be added. Conventionally, when trying to generate segment start and end points with frame-level accuracy, it is necessary to refer to the frames one by one to find the start and end frames of the segment. It was hanging. In the present invention, since the cut point is detected from the feature amount by image processing, it is possible to reduce the time required to create the segment metadata with frame level accuracy.

  The segment metadata creating apparatus of the present invention further includes a selection accepting unit that accepts selection of the created segment metadata displayed on the display unit, and the created segment metadata that has been accepted by the selection accepting unit. The display means displays the segment video corresponding to the data.

  With this configuration, the start point and end point of the created metadata and the content of the metadata can be quickly confirmed even in the case of a service that updates the metadata one after another in a live broadcast program or the like.

  Another aspect of the present invention is a segment metadata creation method for creating segment metadata representing information on a section of a moving image, and this method includes a segment including video content information to constitute segment metadata to be created A creation schedule input step for inputting metadata creation schedule information, a display step for displaying the video and the segment metadata creation schedule information, and a start point and an end of segment metadata corresponding to the segment metadata creation schedule information By creating at least one of the points as a segment end point and associating the time code of the input segment end point with the video content information of the segment metadata creation schedule information, the generated segment metadata is generated. To generate segment metadata And a storage step of storing the video and the segment metadata created schedule information and the created segment metadata. This method also provides the above-described advantages of the present invention. In addition, various additional configurations described in the above-described aspect of the segment metadata creation apparatus can be applied to this mode, and the advantages of these additional configurations can be similarly obtained.

  Another aspect of the present invention is a segment metadata creation program that causes a computer to execute a process of creating segment metadata that represents information on a section of a moving image, and this program should constitute segment metadata to be created A creation schedule input step for inputting segment metadata creation schedule information including video content information, a display step for displaying the video and the segment metadata creation schedule information, and a segment meta data corresponding to the segment metadata creation schedule information Created by associating the input time code of the segment end point with the end point input step of inputting at least one of the data start point and end point as the segment end point, and the video content information of the segment metadata creation schedule information Used segment metadata And segment metadata generating step of forming, to execute a storage step of storing the video and the segment metadata created schedule information and the created segment metadata to the computer. This configuration also provides the above-described advantages of the present invention. In addition, various additional configurations described in the above-described aspect of the segment metadata creation apparatus can be applied to this mode, and the advantages of these additional configurations can be similarly obtained.

  The present invention provides a segment metadata creation device and a segment metadata having an effect of facilitating the creation work of segment metadata by providing a configuration that makes it possible to use a segment metadata creation schedule for the creation work of segment metadata. A data creation method can be provided.

  Hereinafter, a segment metadata creation device according to an embodiment of the present invention will be described with reference to the drawings.

  A segment metadata creation apparatus according to the present embodiment is shown in FIG. This embodiment assumes that the creator of segment metadata creates segment metadata for a moving image in real time while viewing the moving image in real time.

  In FIG. 1, a segment metadata creating apparatus 100 includes a video / audio input unit 101 that is a means for inputting a moving image with a time code, and a video / audio that performs display control of the input moving image and display control of the stored moving image. The audio control unit 102, the video feature amount calculation unit 103 that detects a time code in which the video feature amount changes by a threshold value or more from the input video, and the segment metadata creation schedule information that is the segment metadata creation schedule are input. A creation schedule input unit 105 as a means, a start point / end point input unit 106 as a means for inputting a start point and an end point as segment end points of segment metadata, a video and segment metadata creation schedule information, and a created Display unit 107 for displaying segment metadata, video, segment metadata creation schedule information, and created A storage unit 111 for storing segment metadata and create schedule display condition, and the segment metadata creation control unit 104 is a means for generating segment metadata performs control for the segment metadata creation.

  The segment metadata creation apparatus 100 of FIG. 1 is configured by a computer having an arithmetic processing function including an image processing function, an input / output function, and a storage function, the display unit 107 is realized by a display, and various input units are keyboards. Realized with mouse and other input devices. Further, the start point / end point input unit 106 of FIG. 1 functions as an end point input unit of the present invention, and the segment metadata creation control unit 104 functions as a segment metadata generation unit.

  Here, the segment metadata and its creation schedule information in the present embodiment will be described. As already described, metadata is not data itself but data that describes information about the data. The segment metadata is section metadata when a moving image is divided into a plurality of sections. The segment metadata is composed of video content information and segment end point information. The video content information is information on the video content of the target section, and is, for example, a segment title or a character (also referred to as semantic metadata). The segment end point is information on the start point and end point of the segment video in the target section (also referred to as section metadata).

  The segment metadata creation schedule information of the present embodiment includes the video content information of the segment metadata scheduled to be created. Such creation schedule information is input in advance and displayed. Then, the creator inputs the segment end point of the target segment metadata with reference to the display of the creation schedule information. In this way, the segment end point is input from the input device, and is associated with the video content information of the creation schedule information, and segment metadata is generated.

  In the present embodiment, the segment metadata creation schedule information includes confirmation data in addition to the video content information. The confirmation data is information for identifying the segment video to which the segment metadata is attached, and is composed of any one or more of a still image, a moving image, audio, and a character string. The confirmation data is output together with the video content information of the segment metadata scheduled to be created, and supports the work of searching for the segment end point from the moving image.

  As a specific example, in this embodiment, the moving image is baseball, and segment metadata is added to the baseball video. For example, the video of each batter at each batter is a segment video and is subject to segment metadata. The segment metadata creation schedule information is, for example, the name of the batter. On the other hand, the confirmation data is, for example, a batter's face photo and voice. The names and face photos of these batters are displayed. Segment end points are searched and input with reference to the face photo and voice. The worker can easily proceed with the work even if the worker is not familiar with all the batters in advance. When a segment end point is input from the input device, the segment end point is associated with the video content information (batter name), and segment metadata is generated.

  In this embodiment, as described above, it is assumed that the creator creates segment metadata for a moving image in real time while viewing the moving image in real time. Taking baseball as an example, the name and face of the batter are displayed along with the baseball video. While viewing these displays, the worker inputs the segment end point each time the batter changes. For example, if the batter of one batter A is over and the batter of the next batter B starts, the creator inputs the start point and the end point of the batter A during the batter B batter. This process is repeated, and segment metadata is sequentially created in real time.

  In the present embodiment, as will be described later, a plurality of types of segment metadata are generated. That is, a plurality of types of segment metadata creation schedule information are input, and segment metadata corresponding to the creation schedule information is generated. For example, in the example of baseball, the first type is an image of each batter of each batter described above. The second type is an image of each inning (one-time front, first back, second front...). Still other types of segment metadata may be generated. Creation schedule information is input and processed for each type of segment metadata.

  Further, in this embodiment, as described later, when one segment metadata is generated, the scheduled creation information of the segment metadata to be generated next is displayed. This process is performed based on information indicating the segment metadata creation order. Specifically, batting order data is input, stored, and used for the segment metadata for each batting order. As for the segment metadata of each inning, data of the number of times (a total of 18 on the front and back of 1 to 9 times) is input, stored and used. The former corresponds to the appearance schedule order information of the present invention, and the latter corresponds to the expected appearance frequency information of the present invention.

  The outline of the segment metadata creation schedule information and the processing thereof according to the present embodiment has been described above. Next, returning to FIG. 1, the configuration of each part of the segment metadata creation apparatus 100 will be further described.

  In FIG. 1, a display unit 107 displays a moving image display unit 108 that displays a moving image, a timeline display unit 109 that displays time code in which the created segment metadata and the feature amount of the video change more than a threshold value, A group display unit 110 that displays segment metadata creation schedule information and created segment metadata for each type of segment metadata.

  In addition, the storage unit 111 includes a video / audio storage unit 112 that stores moving images, a creation schedule storage unit 113 that stores input segment metadata creation schedule information, and a created segment that stores created segment metadata. It includes a metadata storage unit 114 and a creation schedule display condition setting storage unit 115 that stores display conditions for segment metadata creation schedules. Display conditions may also be input from the creation schedule input unit 105.

  The group display unit 110 functions as a display unit of the present invention, and displays segment metadata creation schedule information and created segment metadata for each type of segment metadata. An example of a screen displayed on the display by the group display unit 110 is shown in FIG. FIG. 2 assumes the case where segment metadata is created for a moving image in which a person appears. Using the above-mentioned baseball example, one of a plurality of types (groups) is a batting unit video and the other is an inning video. In this case, FIG. 2 corresponds to a screen on which the information related to the former at-bat is displayed. The display of other groups is performed in the same manner, and a plurality of group displays are preferably arranged side by side on the screen. The screen may be divided and various images such as a moving image and a plurality of group displays may be displayed side by side.

  In FIG. 2, one group display includes a list 201 of segment metadata creation schedule information titles, a list 202 of titles of created segment metadata, a creation schedule and created segment metadata contents 203, and a creation schedule. And confirmation data 204 for searching the target video of the created segment metadata from the moving image. The creation schedule list and content information correspond to segment metadata creation schedule information.

  The confirmation data 204 is the face and voice of the target person of the segment metadata in the example of FIG. The face and voice of the target person are input as part of the segment metadata creation schedule information, stored in the creation schedule storage unit 113, and output. When the voice confirmation button is clicked by the operator, the voice of the target person is acquired from the creation schedule storage unit 113 and output from a speaker (not shown).

  The confirmation data 204 is used by the creator to search for the start point and end point of the segment metadata scheduled to be created. In this embodiment, the creator operates an input device such as a button when the start point and the end point are found by moving the moving image forward or backward using an input device such as a jog dial. At this time, the input device functions as a segment end point input means, and segment end points such as a start point and an end point are input. In this process, the confirmation data 204 serves as a reference for the creator searching for segment metadata and supports the segment metadata creation work. Even if the creator's face and voice are not stored in advance, the creator can search for the relevant scene by referring to the confirmation data 204 and search for the corresponding scene, and can create segment metadata. . Here, the case of creating segment metadata has been described, but the confirmation data 204 may be used in the same manner when searching for created segment metadata.

  The confirmation data 204 is an example, and the present invention is not limited to the above example. The confirmation data 204 may be a still image or a moving image. When creating segment metadata of sports video such as baseball, the confirmation data 204 may be a back number. Further, the confirmation data 204 may be a character string indicating the characteristics of a person. Furthermore, in the present embodiment, it is needless to say that segment metadata may be assigned to an object other than a person. The confirmation data 204 may be, for example, a building shape or color.

  As shown in the above description, the confirmation data 204 is information for identifying the target segment video, and may be composed of any one or more of a still image, a moving image, audio, and a character string. The confirmation data 204 may be appropriately set so as to facilitate the search operation according to the video that is the target of the segment metadata.

  Next, the timeline display unit 110 will be described. The timeline display unit 110 displays the created segment metadata in time series for each type of segment metadata. FIG. 3 shows an example of a screen displayed by the timeline display unit 110. Here, FIG. 3 assumes the case of creating segment metadata for each batter in professional baseball. One timeline display includes created segment metadata for each group, segment metadata 301 in the middle of creation, time code when the video feature amount changes by more than a threshold value, an image 302 of a change point, and a playback position of a moving image It is comprised with the information 303 which shows.

  Here, the video feature amount calculation unit 103 calculates and detects a change in the video feature amount that is equal to or greater than the threshold value. The feature amount of the video is, for example, a luminance value such as a luminance average value. The video feature amount calculation unit 103 determines whether or not the amount of change in luminance value between adjacent frames arranged in the front and rear is greater than or equal to a predetermined threshold value. By this process, the point at which the scene changes is detected. This point is called a cut point. The threshold value of the change amount of the feature amount is set between a large change amount that generally appears at the cut point and a small change amount that is generally found in a portion other than the cut point.

  FIG. 4 shows processing of the video feature quantity calculation unit 103. As shown in FIG. 4A, the video feature amount calculation unit 103 calculates the time code of the frame at the cut point where the video feature amount has changed by a threshold value or more. As shown, the cut point appears when the camera is switched. Then, based on the calculation result of the video feature amount change point, the section data is obtained and stored in the storage unit. As shown in FIG. 4B, the section data includes a time code (time code start point) of a frame in which the video feature amount has changed by a threshold value or more and a time code (time) of a frame in which the video feature value has changed by a threshold value or more. Code end point). The time code end point of one section is the same as the time code start point of the next section.

  Note that the relationship between the section between the cut points and the target section of the segment metadata will be explained. Typically, a plurality of sections shown in FIG. It becomes a section. For example, it is assumed that one baseball is a piece of segment metadata. Each time the camera is switched in the middle of a bat, the video feature amount exceeds the threshold, and the section (a pair of time code start point and time code end point) in FIG. 4 is detected. Thus, a plurality of sections in FIG. 4B are detected for one segment data.

  As the section data, the time code of the representative frame of the video in the section data is also calculated and recorded. The time code of the representative frame is also calculated by the video feature amount calculation unit 103. The representative frame is a frame representing the video between the time code start point and the time code end point. For example, a frame containing a telop is detected by image recognition and designated as a representative frame.

  The section data may also be displayed on the display unit. In the screen of FIG. 3, a time code and an image when the video feature amount changes to a threshold value or more are displayed. This screen is also generated based on the information obtained in FIG.

  Next, the segment metadata creation control unit 104 will be described. The segment metadata creation control unit 104 controls various configurations of the entire apparatus for creating segment metadata. Further, the segment metadata creation control unit 104 functions as a segment metadata generation unit that generates segment metadata, and generates the segment metadata by generating the time code of the start point and the end point input from the start / end point input unit 106. Segment metadata is generated by associating it with the video content information of the schedule information. The time code is attached to the moving image and may be obtained from the moving image. The time code corresponding to the video displayed at the input time of the start point and the end point is acquired from the moving image data and recorded.

  Further, the segment metadata creation control unit 104 determines the creation schedule information of the segment metadata to be created next from the segment metadata creation schedule information stored in the creation schedule storage unit 113, and obtains the created schedule information Is displayed on the display unit 107. This process is performed when one segment metadata is generated. This process is performed using the display condition setting information stored in the creation schedule display condition setting storage unit 115. The display condition setting information is input from the creation schedule input unit 105 and stored in the creation schedule display condition setting storage unit 115.

  The above display process will be described by taking as an example the case of creating segment metadata for a baseball batter. As shown in FIG. 5, the creation schedule storage unit 113 stores batter and batting order data as segment metadata creation schedule information. The name of the batter corresponds to the video content information of the segment metadata. The batting order is an example of the appearance order information of the present invention. In the illustrated example, a batter table and a batting order table are linked. In response to the input of the batting order for each batter, the illustrated associating process is performed, and the batting order for each batter is specified by referring to these tables. As shown in FIG. 6, the creation schedule display condition setting storage unit 115 stores “display order”, “display condition”, and “display content” as display conditions. In the example shown in the figure, the display order is “ascending order of batting order”, the display condition is “repeat”, and the display content is designated as “batter name”.

  The segment metadata creation control unit 104 displays the creation schedule information of the segment metadata to be created next in accordance with the display condition setting information of FIG. Therefore, the segment metadata creation control unit 104 updates the display of the batter name according to the batting order according to the conditions “display order = ascending order of batting order” and “display content = batter name”. In addition, since “display condition = repeated”, when the final hit order is reached and the segment metadata of the batter in the final hit order is generated, the name of the head batter is displayed. Of course, when the batting order is changed, the batter names can be displayed in ascending order of the changed batting order by changing the association between the batter and the batting order.

  In the above example, the batting order information corresponds to the appearance order information of the present invention, and is handled as information indicating the segment metadata creation order. The creation order information is not limited to the expected appearance order, and may be set according to the type of segment metadata to be created. For example, when creating segment metadata for each inning, the number of times (a total of 18 on the front and back of 1 to 9 times) may be used, and this information corresponds to the expected appearance frequency information of the present invention.

  The operation of the segment metadata creation apparatus configured as described above will be described below. FIG. 7 shows an operation when segment metadata is created using segment metadata creation schedule information.

  First, the creation schedule input unit 105 fetches segment metadata creation schedule information (S100), and the segment metadata creation control unit 104 stores the fetched segment metadata creation schedule information in the creation schedule storage unit 113 and also stores the segment metadata. The data creation schedule information is grouped for each segment metadata type, and the segment metadata creation schedule information is displayed for each group (S101).

  The segment metadata creator searches the start and end points of the segment metadata to be created from the video using the confirmation data displayed during the creation schedule displayed for each group. Perform end point input operations. The start point and end point are segment end points. In the search operation, a jog dial or the like is used to advance or return the image. First, one of the start point and the end point is searched and input, and then the other is searched and input. When the above operation is performed, the segment metadata creating apparatus 100 operates as follows.

  As shown in FIG. 7, the start / end point input unit 106 inputs the start point or the end point as the segment end point in response to the operation of the creator (S102). The segment metadata creation unit 104 selects the segment metadata creation schedule information displayed by the group display unit 110 when inputting the segment end point, and selects the video content information included in the selected creation schedule and the input segment end point. Correlate (S103).

  Then, the segment metadata creation control unit 104 determines whether both a start point and an end point are given (S104). When only either the start point or the end point is given (S104, No), the segment metadata creation control unit 104 registers the segment metadata of only the start point or the end point in the storage unit (S105), It returns to step S102 and waits for further input.

  If both the start point and end point are assigned in step S104, the segment metadata creation control unit 104 displays the segment metadata provided with them as created segment metadata on the display unit (S106), The created segment metadata is registered in the storage unit (S107).

  In the above process, for example, when a start point is first input in step S102, step S104 is No, and the process returns to step S102. When the end point is input in step S102 of the next loop, step S104 is Yes and segment metadata is displayed and registered.

  When the created segment metadata is registered in step S107, the segment metadata creation control unit 104 determines whether or not to finish creating the segment metadata (S108). If the determination in step S108 is No, the segment metadata creation control unit 104 creates the segment metadata creation schedule information to be created next from the segment metadata creation schedule information other than the start and end points. Is extracted and displayed on the display unit 107 (S109), and the process returns to step S102.

  The above process is repeated until the creation of segment metadata is completed. For example, when the creator's creation termination instruction is input from the input device, the termination determination in step S108 is Yes. In this case, the creation of the segment metadata is terminated.

  The segment metadata creation process in FIG. 7 has been described above. The above creation process is performed as follows, taking a baseball broadcast as an example. Here, a group of segment metadata for each at-bat is taken up as one of a plurality of groups. In step S100 of FIG. 7, the batter name, its confirmation data, and batting order are input as segment metadata creation schedule information, and the creation schedule information is displayed in step S101. First, the name and confirmation data of the first batter are displayed. The confirmation data is, for example, a face photograph.

  As the moving image, the image of the batter of the first batter is displayed, and then the image of the batter of the second batter is displayed. The creator searches for and inputs the start and end points of the video of the top batter when the batter finishes. Confirmation data is referred to at this time. For example, first, a start point is input, and then an end point is input. The start point may be input in the middle of the first batter's batting. When the start point and end point are input, the determination in step S104 is Yes, the segment metadata already created is displayed and registered in steps S106 and S107, and the segment metadata creation schedule information of the second batter is displayed in step S109. Is displayed.

  Such processing is repeated to sequentially create segment metadata, and segment metadata is created in real time in parallel with the moving image display.

  As described above, in the present embodiment, the video content information of the segment metadata is input in advance as the creation schedule information, and the segment end point input by looking at the video is included in the video content information. It is associated. In addition to such pre-input video content information, video content information may be added later. The input of the additional video content information may be performed together with the segment end point input operation. Additional video content information may be written in the screen of FIG.

  As other functions of the segment metadata creating apparatus 100, the following moving image display function is provided. In this function, the segment metadata creation control unit 104 displays the created segment data on the display unit 107. The segment metadata creation control unit 104 functions as a selection accepting unit, and accepts a display selection operation for the created segment metadata via an input device (not shown). When the segment metadata creation control unit 104 receives a display selection operation, the segment metadata creation control unit 104 causes the display unit 107 to display a moving image of the corresponding segment metadata. In this control, the segment metadata creation control unit 104 supplies the video / audio control unit 102 with the time code of the start point and end point of the selected segment metadata. The video / audio control unit 102 reads the video from the start point to the end point from the video / audio storage unit 112 with reference to the time code attached to the video, and displays the video on the display unit 107.

  Next, an application example of this embodiment will be described. Here, a segment metadata automatic generation function will be described as an additional function. This function is mainly realized by the video feature amount calculation unit 103 and the segment metadata creation control unit 104.

  As described above, the video feature amount calculation unit 103 detects the time code when the video feature amount changes from the moving image by a threshold value or more as the cut point time code. At this time, the video feature amount calculation unit 103 functions as a cut point detection unit. The video feature amount is, for example, a change amount of the luminance average value of the frame, and a determination threshold is set between the luminance change amount that appears when the camera is switched and the luminance change amount in other video portions.

  The cut point is detected every time the screen is switched. Therefore, the cut point may or may not match the segment end point (start point or end point) of the segment metadata (that is, the camera image is switched in the middle of the segment metadata target section). In the case of a spot). Therefore, the segment metadata creation control unit 104 extracts and detects segment end points from the cut points as follows.

  The segment metadata creation control unit 104 has an image processing function, captures a moving image, determines whether each cut point is a segment end point based on the segment metadata creation schedule information and the moving image, and thereby Detect segment endpoints. In the image processing, it is determined which cut point to which cut point is the target segment of the segment metadata. In the present embodiment, characters in the video are recognized by image processing, and segment end points are detected from the characters.

  For example, it is assumed that segment metadata of baseball batting video is generated. As the image processing, the batter name is recognized from the ticker video of the batter name. As an example, assume that segment metadata of batter X is automatically generated. Assume that the batter name (telop) of the batter X is first recognized at time TX. In this case, it is determined that the most recent cut point before the time TX is the start point of the segment metadata. Also, assume that the batter name of the batter Y next to the batter X is first recognized at time TY. In this case, it is determined that the cut point immediately before the time TY is the end point of the batter X's segment metadata.

  The segment metadata creation control unit 104 associates the segment end point determined as described above with the video content information, thereby automatically generating segment metadata.

  The automatically generated segment metadata is displayed on the display unit 107. The creator looks at the display and determines whether the automatically generated segment metadata is correct. If the segment metadata is not correct, the correct segment end point is input via the start / end point input unit 106. If the segment metadata is correct, an approval signal is input via the start point / end point input unit 106. The input of the approval signal also corresponds to the input of the segment end point.

  The segment metadata creation device 100 and the segment metadata creation method using the segment metadata creation device 100 according to the embodiment of the present invention have been described above. According to the present embodiment, video content information that should constitute segment metadata to be created is input as segment metadata creation schedule information, and the creation schedule information is displayed. Even if the creator does not memorize all the contents of the moving image, it is only necessary to input the segment end point by looking at the creation schedule information, and therefore it becomes easy to create the segment metadata.

  Further, according to the present embodiment, confirmation data is displayed together with video content information. The confirmation data is information for identifying the segment video to which the segment metadata is attached, and characterizes the target segment video. Since the confirmation data is output, the creator can easily search for the segment end points (start point and end point) while viewing the confirmation data. In the above example, the face and features of the person appearing in the segment video are displayed as confirmation data, so that even if the person's face and features appearing in the video are not fully understood, the creator Can create. In this way, segment metadata can be easily created.

  Further, according to the present embodiment, segment metadata creation schedule information of a plurality of types of segment metadata is input, and a plurality of types of segment metadata creation schedule information and created segment metadata are displayed by type. Therefore, since various types of segment metadata can be displayed for each type of data, it is possible to create various types of segment metadata.

  Further, according to the present embodiment, the created segment metadata is displayed in time series. Therefore, it becomes possible to easily select segment metadata to be confirmed after the segment metadata is created.

  Further, according to the present embodiment, the creation schedule input means inputs the appearance order information of the plurality of segment videos scheduled to be assigned with the plurality of segment metadata, and the segment metadata generation means outputs the expected appearance order. Based on the information, the display of the display means is updated with the segment metadata creation schedule information of the segment metadata to be created next. The appearance schedule order information is, for example, the hitting order described above. With this configuration, it is possible to display the creation schedule information of the segment metadata to be created next. The creator inputs the segment end points of the segment metadata to be created that are sequentially displayed. Therefore, it becomes easy to create segment metadata.

  In addition, according to the present embodiment, the creation schedule input unit inputs the number of times of appearance of the plurality of segment videos scheduled to be provided with the plurality of segment metadata, and the segment metadata generation unit Based on the information, the display of the display means is updated with the segment metadata creation schedule information of the segment metadata to be created next. The expected number of appearances is, for example, the number of baseball innings. This configuration also facilitates the creation of segment metadata.

  Further, according to the present embodiment, the feature amount calculation unit 103 functions as a cut point detection unit, and detects a cut point time code when the video feature amount changes by more than a threshold value from the moving image. Then, segment end points are detected from the cut points, and segment metadata is automatically generated. This further facilitates the generation of segment metadata. In this regard, conventionally, when trying to generate the segment start point and end point with frame-level accuracy, it is necessary to refer to the frames one by one to find the start and end frames of the segment. It took time. In the present embodiment, since the cut point is detected from the feature amount with the frame level accuracy by the image processing, the time required for creating the segment metadata with the frame level accuracy can be shortened.

  Further, according to the present embodiment, the selection of the created segment metadata displayed on the display means is accepted, and the segment video corresponding to the created segment metadata accepted for selection is displayed on the display means. . Therefore, even in the case of a service that updates metadata one after another in a live broadcast program or the like, the start point and end point of the created metadata and the content of the metadata can be quickly confirmed.

  The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above-described embodiments, and it goes without saying that those skilled in the art can modify the above-described embodiments within the scope of the present invention.

  As described above, the segment metadata creation device and the segment metadata creation method according to the present invention have the effect of facilitating the creation of segment metadata, and create segment metadata for video that is being broadcast at a broadcasting station. It is useful as a device to perform.

The block diagram of the segment metadata production apparatus in embodiment of this invention The figure which shows an example of the display of the group display part of the segment metadata production apparatus in embodiment of this invention The figure which shows an example of the display of the timeline display part of the segment metadata production apparatus in embodiment of this invention The figure which shows an example of the calculation data of the video feature-value calculation part of the segment metadata production apparatus in embodiment of this invention, (a) The figure which shows a video feature-value change point, The figure which shows (b) section data The figure which shows an example of the data memorize | stored in the production schedule memory | storage part of the segment metadata production apparatus in embodiment of this invention The figure which shows an example of the data memorize | stored in the production schedule display condition setting memory | storage part of the segment metadata production apparatus in embodiment of this invention Flow chart of segment metadata creation operation of segment metadata creation device in the embodiment of the present invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Segment metadata production apparatus 101 Image | video / audio input part 102 Image | video / audio control part 103 Image | video feature-value calculation part 104 Segment metadata creation control part 105 Creation schedule input part 106 Start point / end point input part 107 Display part 108 Animation display Unit 109 Timeline display unit 110 Group display unit 111 Storage unit 112 Video / audio storage unit 113 Creation schedule storage unit 114 Created segment metadata storage unit 115 Creation schedule display condition setting storage unit

Claims (17)

A segment metadata creation device that creates segment metadata representing information of a video section,
Creation schedule input means for inputting segment metadata creation schedule information including video content information that should constitute segment metadata to be created;
Display means for displaying the video and the segment metadata creation schedule information;
Endpoint input means for inputting at least one of a start point and an end point of segment metadata corresponding to the segment metadata creation schedule information as a segment endpoint;
Segment metadata generation means for generating created segment metadata by associating the input time code of the segment end point with the video content information of the segment metadata creation schedule information,
Storage means for storing the video, the segment metadata creation schedule information, and the created segment metadata;
A segment metadata creating apparatus characterized by comprising:   The segment metadata creation schedule information includes confirmation data that is information for identifying a segment video to which the segment metadata is attached together with the video content information, and the confirmation data includes a still image, a moving image, audio, and text 2. The segment metadata creation apparatus according to claim 1, wherein the segment metadata creation apparatus is composed of any one or more of columns and is output to support a work of searching for the segment end point from the moving image.   The creation schedule input means inputs segment metadata creation schedule information of a plurality of types of segment metadata, and the endpoint input means inputs the segment endpoint of each of the plurality of types of segment metadata, and the segment metadata The data generation unit generates each of the plurality of types of created segment metadata, and the display unit displays the plurality of types of segment metadata creation schedule information and the created segment metadata by type. The segment metadata creation apparatus according to claim 1 or 2.   4. The segment metadata creation device according to claim 1, wherein the display means further displays the created segment metadata generated by the segment metadata generation means in time series. . The creation schedule input means further inputs appearance schedule order information of a plurality of segment videos scheduled to be provided with a plurality of segment metadata,
The segment metadata generation means updates the display of the display means with the segment metadata creation schedule information of the segment metadata to be created next based on the appearance schedule order information. The segment metadata creation apparatus according to any one of 1 to 4. The creation schedule input means further inputs information on the number of times of appearance of a plurality of segment videos scheduled to be provided with a plurality of segment metadata,
The segment metadata generation means updates the display of the display means with the segment metadata creation schedule information of the segment metadata to be created next based on the expected appearance frequency information. The segment metadata creation apparatus according to any one of 1 to 4. A cut point detecting means for detecting a time code when a video feature amount changes from the moving image by a threshold value or more as a cut point time code;
The segment metadata generation means detects the segment endpoint by determining whether the cut point is the segment endpoint based on the segment metadata creation schedule information and the moving image, and detects the detected segment 7. The segment metadata creating apparatus according to claim 1, wherein the segment metadata is automatically generated by associating a time code of an end point with the video content information.   The display unit includes a selection receiving unit that receives selection of the created segment metadata displayed on the display unit, and the display unit displays a segment video corresponding to the created segment metadata that has been selected by the selection receiving unit. The segment metadata creation device according to any one of claims 1 to 7, wherein A segment metadata creation method for creating segment metadata representing information about a video section,
A creation schedule input step for inputting segment metadata creation schedule information including video content information that should constitute the segment metadata to be created;
A display step for displaying the video and the segment metadata creation schedule information;
An endpoint input step of inputting at least one of the start point and the end point of the segment metadata corresponding to the segment metadata creation schedule information as a segment endpoint;
A segment metadata generation step for generating created segment metadata by associating the input time code of the segment end point with the video content information of the segment metadata creation schedule information;
A storage step of storing the video, the segment metadata creation schedule information, and the created segment metadata;
A segment metadata creation method characterized by comprising:   The segment metadata creation schedule information includes confirmation data that is information for identifying a segment video to which the segment metadata is attached together with the video content information, and the confirmation data includes a still image, a moving image, audio, and text The segment metadata creation method according to claim 9, wherein the segment metadata creation method is configured to include any one or more of columns, and is output in order to support an operation of searching for the segment end point from the moving image.   The creation schedule input step inputs segment metadata creation schedule information of a plurality of types of segment metadata, and the endpoint input step inputs the segment endpoint of each of the plurality of types of segment metadata, The data generation step generates each of the plurality of types of created segment metadata, and the display step displays the plurality of types of segment metadata creation schedule information and the created segment metadata by type. The segment metadata creation method according to claim 8 or 9.   The segment metadata creation according to any one of claims 9 to 11, wherein the display step further displays the created segment metadata generated by the segment metadata generation means in time series. Method. The creation schedule input step further inputs schedule appearance order information of a plurality of segment videos scheduled to be provided with a plurality of segment metadata,
13. The segment metadata generation step performs control to display the segment metadata creation schedule information of the segment metadata to be created next based on the expected appearance order information. The segment metadata creation method according to any one of the above. The creation schedule input step further inputs information on the number of times of appearance of a plurality of segment videos scheduled to be provided with a plurality of segment metadata,
13. The segment metadata generation step performs control to display the segment metadata creation schedule information of the segment metadata to be created next based on the expected appearance frequency information. The segment metadata creation method according to any one of the above. A cut point detecting step of detecting a time code when the feature amount of the video changes from the moving image by a threshold value or more as a cut point time code;
The segment endpoint is detected by determining whether the cut point is the segment endpoint based on the segment metadata creation schedule information and the moving image, and the time code of the detected segment endpoint is the video content The segment metadata creation method according to claim 9, wherein the segment metadata is automatically generated in association with information.   16. The method according to claim 9, further comprising a selection receiving step for receiving selection of the displayed created segment metadata, wherein a segment video corresponding to the created segment metadata for which selection has been received is displayed. The method for creating segment metadata as described above. A segment metadata creation program for causing a computer to execute a process for creating segment metadata representing information of a video section,
A creation schedule input step for inputting segment metadata creation schedule information including video content information that should constitute the segment metadata to be created;
A display step for displaying the video and the segment metadata creation schedule information;
An endpoint input step of inputting at least one of the start point and the end point of the segment metadata corresponding to the segment metadata creation schedule information as a segment endpoint;
A segment metadata generation step for generating created segment metadata by associating the input time code of the segment end point with the video content information of the segment metadata creation schedule information;
A storage step of storing the video, the segment metadata creation schedule information, and the created segment metadata;
A segment metadata creation program which causes the computer to execute

Images