JP3781715B2 - Metadata production device and search device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a metadata production device and a search device in content production.
[0002]
[Prior art]
In recent years, in the production of video / audio contents, metadata related to these contents has been assigned.
[0003]
However, the above-mentioned metadata is added manually by checking the information to be metadata while playing the produced video / audio content based on the produced video / audio content scenario or narration manuscript. The method of producing by computer input is common, and requires considerable labor.
[0004]
In addition, there is a system for tagging using voice recognition during camera shooting, but it was only used at the same time as shooting. (See Patent Document 1)
[0005]
[Patent Document 1]
Japanese Patent Application Laid-Open No. 09-130736
[Problems to be solved by the invention]
The present invention is intended to solve the problems related to the above-mentioned conventional problems, and by confirming information to be metadata by reproducing the produced video / audio content, the audio input can be performed. An object of the present invention is to provide a system for producing and searching for the metadata by inputting it to a computer or the like.
[0007]
[Means for Solving the Problems]
The present invention to solve the above problems, production scenarios was produced in accordance with the content, or the content audio signal of the extracted content management keyword from the content is input, the data of the previous SL audio signal And a speech recognition means for recognizing a keyword from the speech signal data converted into data by the input means, and a general attribute keyword related to the keyword is appended to the keyword output from the speech recognition means , using the time code supplied from the previous SL content, that a file processing means for storing the metadata file information indicating a time position relationship between the image signal included in the said keyword content by appended It is.
As a result, it is possible to automatically produce metadata with time code by automatically inputting metadata using speech recognition, which has been inputted and produced using a conventional keyboard.
In particular, when metadata is given at intervals of several seconds, it is difficult by key input. However, according to this configuration, metadata can be given efficiently even at intervals of several seconds.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
The invention according to claims 1-3 of the present invention, production scenarios was produced in accordance with the content, or the audio signal of content management keyword extracted from the content of the content is input, the previous SL audio signal An input means for converting to data, a voice recognition means for recognizing a keyword from the voice signal data converted to data by the input means, and a general attribute keyword related to the keyword added to the keyword output from the voice recognition means to together, before SL using the timecode supplied from the content, and a file processing means to note the information indicating the time position relationship between the image signal included in said keyword the content stored in the metadata file This is a metadata production apparatus characterized by the above.
[0009]
The invention according to 6 claim 4 of the present invention, production scenarios was produced in accordance with the content, or the content audio signal of the extracted content management keyword from the content is entered, the voice An input means for converting a signal into data, a voice recognition means for recognizing a keyword from voice signal data converted into data by the input means, and a keyword output from the voice recognition means, and a general attribute keyword related to the keyword with by appending, by using the time code supplied from the previous SL content, and file processing means for storing the metadata file information indicating a time position relationship between the image signal included in the said keyword content by appended the image contained in the content corresponding to the keyword or the general attribute keyword Comprises a search means for searching for signals, as well as a searchable the keyword search when a metadata search apparatus, wherein the is also searchable general attribute keyword.
[0010]
According to the seventh aspect of the present invention, a scenario produced according to the produced content or an audio signal of a content management keyword extracted from the content is input, and the audio signal is converted into data. A plurality of input means and genre-specific dictionaries are prepared, a genre dictionary adapted to the content is selected, and voice recognition means for recognizing a keyword from voice signal data converted into data by the input means, and the voice recognition means File processing means for storing the output keyword in the metadata file together with a time code indicating the time position with the image signal included in the content, and a control file for managing the relationship between the recording position of the content file and the metadata file Content information file processing means for generating the content file and the content file And selecting a keyword from a common dictionary used in the voice recognition unit suitable for the field of the content to be searched, a recording unit that records the metadata file and the control file together, and the selected keyword is Search means for specifying the recorded metadata file, searching for the content to be searched from the contents recorded in the recording means, and searching for the recording position of the scene to be searched from the control file. and a metadata search apparatus characterized by identifying a recording position of the recorded content file to the recording means from the metadata.
[0011]
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a metadata production apparatus according to Embodiment 1 of the present invention.
[0012]
In FIG. 1, 1 is a content database (DB), 2 is input means, 3 is speech recognition means, 4 is a dictionary database (DB), 5 is file processing means, and 11 is a video monitor.
The content DB 1 is, for example, a VTR (or a video / audio signal reproducing means composed of a hard disk, or a video / audio signal reproducing means using a memory means such as a semiconductor memory as a recording medium, or an optical recording type or magnetic recording type rotation. Video / audio signal playback means composed of type discs, and video / audio playback means for primary recording and playback of transmitted / broadcast video / audio signals, etc. Content recording means provided with means for reproducing while generating.
A video signal with a time code reproduced from the content DB 1 is output to the video monitor 11 and displayed on the video monitor 11.
Next, the narrator 12 inputs a voice signal of narration using a microphone in accordance with the content displayed on the video monitor 11. At this time, the narrator confirms the content displayed on the video monitor 11 or the time code, and utters the content management keyword extracted based on the scenario, the narration manuscript, or the contents of the content. Is used to input a voice signal as a narration.
Therefore, the recognition rate in the speech recognition means 3 in the subsequent stage can be improved by using a keyword limited in advance from a scenario or the like for the input speech signal as described above.
Next, the input means 2 converts the audio signal input from the microphone into data using a clock synchronized with the vertical synchronization signal output from the content DB 1.
Next, the voice signal data converted into data by the input means 2 is input to the voice recognition means 3. At the same time, a dictionary necessary for speech recognition is supplied from the dictionary DB 4.
Here, the voice recognition dictionary to be used is set in the dictionary DB 4 from the terminal 102.
For example, if the dictionary DB 4 is configured for each field as shown in FIG. 2, the field to be used is set from the terminal 102 (for example, a keyboard terminal capable of key input).
In the case of a cooking program, the dictionary DB 4 is set from the terminal 102 such as cooking-Japanese cuisine-cooking method-stir-fried vegetables.
By setting the dictionary DB 4 as described above, the words used and the words that should be recognized are restricted, and the recognition rate of the speech recognition means 3 is improved.
Further, returning to FIG. 1, a keyword extracted from a scenario, a scenario document, or content is input from the terminal 102.
For example, in the case of a cooking program, the recipe shown in FIG.
Therefore, since the word entered in the recipe is likely to be input as a speech signal, the dictionary DB 4 clearly indicates the recognition priority of the recipe word input from the terminal 102 and performs speech recognition with priority. To.
For example, when “柿” and “shellfish oyster” are in the dictionary, and the recipe word input from the terminal 102 is only “shellfish oyster”, priority 1 is given to “shellfish oyster”. .
When the voice recognition means 3 recognizes the voice “Kaki”, it recognizes it as “shellfish oyster” in which the priority order 1 of the word set in the dictionary DB 4 is specified.
Therefore, in the dictionary DB 4, words are limited in the field input from the terminal 102, and further, the scenario is input from the terminal 102 and the priority of the word is specified, thereby improving the recognition rate in the speech recognition means 3. be able to.
Returning to FIG. 1, the voice recognition unit 3 recognizes the voice signal data input from the input unit 2 according to the dictionary supplied from the dictionary DB 4 and generates metadata.
Next, the metadata output from the voice recognition unit 3 is input to the file processing unit 5.
Here, as described above, the input unit 2 converts the audio signal into data in synchronization with the vertical synchronization signal reproduced from the content DB 1.
Accordingly, the file processing means 5 uses the synchronization information from the input means 2 and the time code value supplied from the content DB 1 to the metadata output from the voice recognition means 3 every second from the start of file. The file is processed in a format in which the reference time (TM_ENT (seconds)) of the data, the number of frame offsets from the reference time (TM_OFFSET), and the time code are added.
For example, in the case of the above-described cooking program, a TEXT format metadata file as shown in FIG.
Next, the recording unit 7 records the metadata file output from the file processing unit 5 and the content output from the content DB 1.
Here, the recording means 7 is composed of an HDD, a memory, an optical disk, etc., and also records the content output from the content DB 1 in a file format.
[0013]
(Embodiment 2)
Next, a second embodiment will be described.
In the second embodiment, as shown in FIG. 5, content information file processing means 6 is added to the first embodiment. The content information file processing means 6 generates a control file indicating the recording position relationship of the content recorded in the recording means 7 and records it in the recording means 7.
That is, in the content information file processing means 6, based on the content output from the content DB 1 and the recording position information of the content output from the recording means 7, the time axis information possessed by the content, the recording Information indicating the address relationship of the content recorded in the means 7 is generated, converted into data, and output as a control file.
For example, as shown in FIG. 6, TM_ENT #j indicating the time axis reference of the content is pointed at an equal time axis interval with respect to the recording media address indicating the recording position of the content. For example, TM_ENT #j is pointed to the recording media address every second (30 frames for NTSC signal).
By mapping as described above, the recording address of the recording means 7 can be uniquely obtained from TM_ENT #j even if the content is distributed and recorded every second. Further, as described above with reference to FIG. 4, the metadata file includes a reference time (TM_ENT (seconds)) from the start of the file, a frame offset number from the reference time (TM_OFFSET), a time code, , Metadata and are recorded in TEXT format.
Therefore, if the metadata 1 is designated in the metadata file, the time code, the reference time, and the frame offset value can be known, so the recording position in the recording means 7 can be immediately known from the control file shown in FIG. It will be.
Here, the example in which the equal time axis interval of the TM_ENTj is pointed every other second has been described, but it can be described according to the GOP unit used in MPEG2 compression or the like.
Furthermore, in NTSC, which is a television vision signal, the vertical sync signal is 60 / 1.001 Hz, so the time code matched to the drop frame mode to match the absolute time and the non-sync matched to the vertical sync signal (60 / 1.001 Hz). Use two types of drop time codes. In this case, the non-drop time code can be represented by TM_ENT #j, and TC_ENT #j can be represented by a drop frame compatible time code.
In addition, the control file can be converted into data using an existing language such as SMIL2, and if the function of SMIL2 is used, the related content and the file name of the metadata file are combined. Can be converted into data and stored in the control file.
Further, in FIG. 6, the recording address of the recording means is directly displayed, but instead of the recording address, the data capacity from the head of the content file to the time code is displayed, and the data capacity and the recording address of the file system are displayed. From this, the recording address of the time code in the recording means may be calculated and detected.
In the present embodiment, the TM_ENTj and time code correspondence table is described as being stored in the metadata file. However, the same effect can be obtained by storing the TM_ENTj and time code correspondence table in the control file. .
(Embodiment 3)
Next, Embodiment 3 will be described.
[0014]
In the third embodiment, as shown in FIG. 7, search means 8 is added to the second embodiment. The search means 8 selects and sets the keyword of the scene to be searched from the same dictionary DB 4 used for speech recognition and metadata detection.
Next, the search means 8 searches the metadata items of the metadata file and displays a list of title names and content scene positions (time codes) that match the keywords.
[0015]
Next, when one specific scene is set from the list display, the reference time (TM_ENT (second)) of the metadata file and the recording media address in the control file are calculated from the number of frame offsets (TM_OFFSET). Is automatically detected and set in the recording means 7, and the content scene recorded at the recording media address from the recording means 7 is reproduced on the monitor 11. By configuring as described above, it is possible to provide an apparatus that can detect a scene to be viewed immediately after detecting metadata.
If a thumbnail file linked to the content is prepared, a representative thumbnail image of the content can be reproduced and displayed when displaying the list of content names that match the keyword.
(Embodiment 4)
Next, another embodiment will be described.
In the above-described first to third embodiments, the system for giving metadata to pre-recorded content has been described. However, the present invention is a system for giving metadata at the time of shooting, such as a camera, in particular, the content content is limited in advance. It can be extended to a system for taking a landscape or providing a shooting position as metadata.
The configuration of this system is shown as Embodiment 4 in FIG.
At the same time that the image is captured by the camera 51 and the video content is recorded in the content DB 54, the location where the camera is capturing is detected by the GPS 52. The position information converted into an audio signal is also recorded in another audio channel. In this case, the camera 51, the GPS 52, the voice synthesis 53, and the content DB 54 may be integrally configured as the camera 50 with a recording unit.
Next, the content DB 54 inputs the position information of the sound signal recorded in the sound channel to the sound recognition means 56.
Here, the area name, landmark, etc. in the dictionary DB 55 are selected and restricted from the terminal 105 by a key input board or the like and output to the voice recognition means 56.
The voice recognition means 56 detects the area name and landmark using the recognized history value and the data in the dictionary DB 55 and outputs it to the file processing means 57.
Next, the file processing unit 57 generates a metadata file by converting the time code output from the content DB 54 and the region name and landmark output from the voice recognition unit 56 into metadata.
Next, the recording unit 58 records the metadata file output from the file processing unit 57 and the content data output from the content DB 54.
With this configuration, it is possible to automatically add area name and landmark metadata for each photographed scene.
[0016]
In each of the above embodiments, in general, there is a possibility that some misrecognition may occur in the speech recognition. When erroneous recognition occurs, the produced metadata and tags can be corrected by using information processing means such as computer means.
[0017]
In addition, the speech recognition means according to the present invention adopts a word recognition method that recognizes speech in units of words, and particularly improves the speech recognition rate by limiting the number of words for speech input and the number of words in the recognition dictionary to be used. be able to.
[0018]
Further, in the present invention, the configuration has been described in which the keyword recognized by the voice recognition means is filed in the metadata file together with the time code. However, in addition to the keyword recognized by the voice recognition means, a related keyword is added to form a file. Also good.
[0019]
For example, when the Yodo River is recognized by voice, general attribute keywords such as terrain and river are also added to the file. By doing so, keywords such as added terrain and river can be used at the time of searching, so that the searchability can be improved.
[0020]
【The invention's effect】
As described above, the present invention inputs a keyword extracted in advance from a scenario or the like of the produced content as an audio signal when creating or tagging metadata related to the content, and based on the scenario. Since the dictionary field is set and the priorities of keywords are set, metadata can be generated from the speech recognition means efficiently and accurately.
In particular, when metadata is given at intervals of several seconds, it is difficult by key input. However, if voice input and voice recognition like this configuration are used, metadata can be efficiently stored even at intervals of several seconds. Can be granted.
Further, by using the control file indicating the recording position of the content and the metadata, and the metadata file indicating the time code, etc., a necessary scene is uniquely searched from the meta keyword, and the recording means Can be played.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a metadata production apparatus according to Embodiment 1 of the present invention. FIG. 2 is a configuration diagram showing an example of a dictionary DB according to the present invention. FIG. 3 is an example of a scenario according to the present invention. FIG. 4 is a data diagram in TEXT format showing an example of a metadata file according to the present invention. FIG. 5 is a block diagram showing a configuration of a metadata search apparatus according to Embodiment 2 of the present invention. FIG. 7 is a block diagram showing a configuration of a metadata search apparatus according to the third embodiment of the present invention. FIG. 8 is a metadata production according to the fourth embodiment of the present invention. Block diagram showing device configuration 【Explanation of symbols】
1 Content DB
2 Input means 3 Voice recognition means 4 Dictionary DB4
5 File processing means 6 Content information file processing means 7 Recording means 11 Video monitor 50 Camera with recording device 51 Camera 52 GPS
53 Speech synthesis means 54 Content DB
55 Dictionary DB
56 voice recognition means 57 file processing means 58 recording means 101 voice input terminal 102 dictionary field selection input terminal 105 dictionary place name selection input terminal

Claims (7)

A metadata production device for producing metadata related to content,
Production scenario was produced in accordance with the content, or the audio signal of the content keyword for content management extracted from the content is input, an input unit for data of the previous SL audio signal,
Voice recognition means for recognizing a keyword from voice signal data converted into data by the input means;
Wherein the keywords output from the voice recognition unit, as well as note the general attributes keyword associated with the keyword, before SL using the timecode supplied from the content, the keyword with the image signal and the time included in the content A metadata production apparatus comprising: file processing means for adding information indicating a positional relationship and storing the information in a metadata file. The voice recognition means includes a plurality of genre dictionary metadata production device according to claim 1, wherein the selecting a dictionary genre adapted to the content. Wherein said speech recognition means includes a plurality of genre dictionary, select a dictionary of a genre that conform to the content, further characterized in that the said scenarios, or keywords extracted from the content of the content to preferentially recognize Item 3. The metadata production device according to item 1 or 2. A metadata search device for searching metadata related to content,
Production scenario was produced in accordance with the content, or the audio signal of the extracted content management keyword from the content of the content is input, an input unit for data of the audio signal,
Voice recognition means for recognizing a keyword from voice signal data converted into data by the input means;
Wherein the keywords output from the voice recognition unit, as well as note the general attributes keyword associated with the keyword, before SL using the timecode supplied from the content, the keyword with the image signal and the time included in the content File processing means for appending information indicating the positional relationship and storing it in the metadata file;
Search means for searching for the image signal included in the content corresponding to the keyword or the general attribute keyword ,
A metadata search apparatus , which can be searched by the keyword at the time of search and can also be searched by the general attribute keyword . A content information file processing means for generating a control file for managing the relationship between the recording position of the content file and the metadata file;
Before SL control file, characterized in that the Meikisuru table recording position of the file of the content of the recording means to match the recording time of the contents, and to search the recording position of the file of the content from the time code The metadata search device according to claim 4. The voice recognition means includes a plurality of genre dictionary, select a dictionary of a genre that conform to the content, further characterized in that the said scenarios, or keywords extracted from the content of the content to be preferentially recognized The metadata search device according to claim 4 or 5. A search device for metadata related to content,
An input means for inputting a scenario produced according to the produced content or an audio signal of a content management keyword extracted from the content, and converting the audio signal into data;
A plurality of genre dictionaries are prepared, a genre dictionary suitable for the content is selected, and voice recognition means for recognizing a keyword from voice signal data converted into data by the input means;
File processing means for storing a keyword output from the voice recognition means together with a time code indicating a time position with an image signal included in the content in a metadata file;
Content information file processing means for generating a control file for managing the relationship between the recording position of the content file and the metadata file;
Recording means for recording the content file, the metadata file, and the control file together;
Select a keyword from the common dictionary used by the voice recognition unit, specify the metadata file in which the selected keyword is recorded, and search for the content to be searched from the content recorded in the recording unit And a search means for searching for a recording position of a scene to be searched from the control file,
A metadata search apparatus, wherein content management metadata is automatically generated, and a recording position of a content file recorded in the recording means is specified from the metadata.

Images