JP2004153764A - Meta-data production apparatus and search apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for producing and searching meta-data by reproducing produced video/audio contents to confirm information to be defined as the meta-data and inputting the information to a computer or the like with a voice input to produce the meta-data. <P>SOLUTION: The production apparatus for the meta-data associated with contents is provided with: an input means for inputting an audio signal of a contents management keyword extracted from a scenario produced in accordance with produced contents or from the contents to make the audio signal into data; an audio recognizing means for recognizing the keyword from audio signal data which are made by the input means; and a file processing means for storing the keyword outputted form the audio recognizing means in a meta-data file together with a time code indicating a temporal position with an image signal contained in the contents. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a metadata production device and a retrieval device in content production.
[0002]
[Prior art]
2. Description of the Related Art In recent years, in the production of video / audio contents, metadata related to these contents has been added.
[0003]
However, the provision of the above metadata is based on the scenario of the produced video / audio content or the narration manuscript, and while reproducing the produced video / audio content, confirming the information to be metadata, and manually The method of producing by computer input was common and required a considerable amount of labor.
[0004]
Further, there is a system for tagging using voice recognition at the time of photographing a camera, but it is only used simultaneously with photographing. (See Patent Document 1)
[0005]
[Patent Document 1]
JP 09-130736 A
[Problems to be solved by the invention]
An object of the present invention is to solve the problems related to the conventional problems described above, and confirms information to be metadata by reproducing produced video and audio contents, and performs audio input. It is an object of the present invention to provide a system for creating and searching for the metadata by inputting it to a computer or the like.
[0007]
[Means for Solving the Problems]
In order to solve the above problems, the present invention is to input a scenario created according to created content or an audio signal of a content management keyword extracted from the content of the content and convert the audio signal into data. A voice recognition means for recognizing a keyword from input means and voice signal data digitized by the input means; and a time code indicating a time position of a keyword output from the voice recognition means and an image signal included in the content. And a file processing means for storing the metadata in a metadata file.
As a result, it is possible to automatically input metadata produced by the keyboard using the speech recognition and produce metadata with a time code.
In particular, when metadata is added at intervals of a few seconds, it is difficult to input metadata by key input. However, according to this configuration, metadata can be added efficiently even at intervals of a few seconds.
In addition, when inputting voice, the keyword is limited and input based on the pre-scenario or the content, so that the recognition rate of voice recognition can be improved.
[0008]
BEST MODE FOR CARRYING OUT THE INVENTION
The invention according to claims 1 to 4 of the present invention provides an input unit for inputting an audio signal of a content management keyword extracted from a scenario created according to created content, and converting the audio signal into data. A voice recognition unit for recognizing a keyword from the voice signal data digitized by the input unit, and a keyword output from the voice recognition unit in a metadata file together with a time code indicating a time position of an image signal included in the content. A metadata production device comprising a file processing means for storing.
[0009]
The invention according to claims 5 to 7 of the present invention inputs a scenario created according to the created content or an audio signal of a content management keyword extracted from the content of the content and converts the audio signal into data. A voice recognition means for recognizing a keyword from input means and voice signal data digitized by the input means; and a time code indicating a time position of a keyword output from the voice recognition means and an image signal included in the content. File processing means for storing a metadata file together with the content file; content information file processing means for generating a control file for managing the relationship between the content file and the metadata file; the content file; the metadata file; Recording means for recording files and A metadata search apparatus characterized.
[0010]
The invention according to claim 8 of the present invention is an input means for inputting an audio signal of a content management keyword extracted from a scenario created according to the created content or the content of the content, and converting the audio signal into data. A plurality of dictionaries for each genre are prepared, a dictionary of a genre suitable for the content is selected, and voice recognition means for recognizing a keyword from voice signal data digitized by the input means is output from the voice recognition means. File processing means for storing the keyword in the metadata file together with the time code indicating the time position of the image signal included in the content, and a control file for managing the relationship between the recording position of the content file and the metadata file. Content information file processing means for causing Recording means for recording the metadata file and the control file together; and selecting a keyword from a common dictionary used by the voice recognition means, which is suitable for the field of the content to be searched, and the selected keyword is recorded. Searching means for specifying the metadata file, searching for the content to be searched from among the contents recorded in the recording means, and searching for the recording position of the scene to be searched from the control file, A metadata search device characterized by specifying a recording position of a content file recorded in the recording means from the metadata.
[0011]
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a metadata production device according to Embodiment 1 of the present invention.
[0012]
In FIG. 1, 1 is a content database (DB), 2 is an input unit, 3 is a voice recognition unit, 4 is a dictionary database (DB), 5 is a file processing unit, and 11 is a video monitor.
The content DB1 is, for example, a VTR (or a video / audio signal reproducing unit constituted by a hard disk, a video / audio signal reproducing unit using a memory unit such as a semiconductor memory as a recording medium, or a rotation such as an optical recording type or a magnetic recording type). And audio signal playback means composed of a disc, and a time code adapted to the content, such as a video and audio playback means for temporarily recording and playing back transmitted or broadcasted video and audio signals. Content recording means provided with a means for reproducing while generating an image).
The video signal with the time code reproduced from the content DB 1 is output to the video monitor 11 and is projected on the video monitor 11.
Next, the narrator 12 inputs a voice signal of a narration using a microphone in accordance with the content displayed on the video monitor 11. At this time, the narrator checks the content or time code displayed on the video monitor 11, utters a scenario, or a narration manuscript, or a content management keyword extracted based on the contents of the content, etc. A voice signal is input as a narration using.
Therefore, as described above, by using a keyword limited from a scenario or the like in advance for the input speech signal, the recognition rate in the speech recognition unit 3 at the subsequent stage can be improved.
Next, the input means 2 converts the audio signal input from the micro horn into data using a clock synchronized with the vertical synchronization signal output from the content DB1.
Next, the voice signal data converted into data by the input unit 2 is input to the voice recognition unit 3. At the same time, a dictionary necessary for speech recognition is supplied from the dictionary DB4.
Here, the speech recognition dictionary to be used is set in the dictionary DB4 from the terminal 102.
For example, assuming that the dictionary DB4 is configured for each field as shown in FIG. 2, the field to be used is set from the terminal 102 (for example, a keyboard terminal that allows key input).
In the case of a cooking program, a dictionary DB4 is set from the terminal 102 for cooking-Japanese cuisine-cooking method-fried vegetables.
By setting the dictionary DB 4 as described above, words to be used and words to be subjected to voice recognition are restricted, and the recognition rate of the voice recognition unit 3 is improved.
Further, returning to FIG. 1, a scenario, a scenario manuscript, or a keyword extracted from the contents of the content is input from the terminal 102.
For example, in the case of a cooking program, the recipe shown in FIG.
Therefore, the words written in the recipe are likely to be input as voice signals, so that the dictionary DB4 specifies the recognition priority of the recipe words input from the terminal 102 and preferentially performs voice recognition. To
For example, if “persimmon” and “shell oyster” are in the dictionary, and if the recipe word input from the terminal 102 is only “shell oyster”, priority is given to “shell oyster”. .
When the voice recognition unit 3 recognizes the voice of “Kaki”, the voice recognition unit 3 recognizes the word “shell oyster” in which the priority order 1 of the word set in the dictionary DB 4 is specified.
Therefore, in the dictionary DB 4, words are limited in the field input from the terminal 102, and the scenario is input from the terminal 102 to specify the priority of the words, thereby improving the recognition rate of the voice recognition unit 3. be able to.
Returning to FIG. 1, the voice recognition unit 3 recognizes voice signal data input from the input unit 2 according to the dictionary supplied from the dictionary DB 4, and generates metadata.
Next, the metadata output from the voice recognition unit 3 is input to the file processing unit 5.
Here, as described above, the input means 2 converts the audio signal into data in synchronization with the vertical synchronization signal reproduced from the content DB1.
Therefore, the file processing unit 5 uses the synchronization information from the input unit 2 and the time code value supplied from the content DB 1 to store the metadata output from the voice recognition unit 3 every second from the start of the file. (TM_ENT (seconds)), the number of frame offsets from the reference time (TM_OFFSET), and the file is processed in a format in which a time code is added.
For example, in the case of the above-mentioned cooking program, a TEXT format metadata file as shown in FIG.
Next, the recording unit 7 records the metadata file output from the file processing unit 5 and the content output from the content DB 1.
Here, the recording unit 7 includes an HDD, a memory, an optical disk, and the like, and also records the content output from the content DB 1 in a file format.
[0013]
(Embodiment 2)
Next, a second embodiment will be described.
In the second embodiment, as shown in FIG. 5, a content information file processing means 6 is added to the first embodiment. The content information file processing means 6 generates a control file indicating a recording position relationship of the content recorded in the recording means 7 and records the control file in the recording means 7.
That is, the content information file processing unit 6 records the time axis information held by the content based on the content output from the content DB 1 and the recording position information of the content output from the recording unit 7. Information indicating the address relationship of the content recorded in the means 7 is generated, converted into data, and output as a control file.
For example, as shown in FIG. 6, TM_ENT #j indicating a time axis reference of the content is pointed at equal time axis intervals to a recording media address indicating a recording position of the content. For example, TM_ENT #j points to the recording media address every 1 second (30 frames in the case of the NTSC signal).
By performing the mapping as described above, the recording address of the recording unit 7 can be uniquely obtained from TM_ENT #j even if the content is distributedly recorded in units of one second. Further, as described above with reference to FIG. 4, the metadata file includes a reference time (TM_ENT (second)) every second from the start of the file, the number of frame offsets from the reference time (TM_OFFSET), a time code, , And metadata are recorded in the TEXT format.
Therefore, if the metadata 1 is specified in the metadata file, the time code, the reference time, and the frame offset value are known, so that the recording position in the recording means 7 can be immediately known from the control file shown in FIG. Will be.
Here, an example has been described in which the isochronous axis interval of the TM_ENTj is, for example, a point every one second, but it can also be described in accordance with a GOP unit used in MPEG2 compression or the like.
Further, since the vertical synchronization signal is 60 / 1.001 Hz in NTSC of the television vision signal, the time code adjusted to the drop frame mode and the vertical synchronization signal (60 / 1.001 Hz) are adjusted to match the absolute time. Use two types of combined non-drop timecode. In this case, the non-drop time code can be represented by TM_ENT #j, and the TC_ENT #j can be represented by a time code corresponding to a drop frame.
Further, the control file can be converted into data using an existing language such as SMIL2. Further, if the function of SMIL2 is used, the related contents and the file name of the metadata file are also combined. And store it in a control file.
Further, FIG. 6 shows a configuration in which the recording address of the recording means is directly displayed. However, instead of the recording address, the data capacity from the beginning of the content file to the time code is displayed, and the data capacity and the recording address of the file system are displayed. Then, the recording address of the time code in the recording means may be calculated and detected.
Further, in the present embodiment, the description has been given of the format in which the correspondence table between TM_ENTj and the time code is stored in the metadata file. However, the same effect can be obtained by storing the correspondence table between TM_ENTj and the time code in the control file. .
(Embodiment 3)
Next, a third embodiment will be described.
[0014]
In the third embodiment, a search unit 8 is added to the second embodiment as shown in FIG. The search means 8 selects and sets a keyword of the scene to be searched from the same dictionary DB4 used for detecting the metadata by voice recognition.
Next, the search means 8 searches the metadata items of the metadata file and displays a list of title names and content scene positions (time codes) that match the keyword.
[0015]
Next, when one specific scene is set from the list, if the reference time (TM_ENT (second)) of the metadata file and the number of frame offsets (TM_OFFSET), the recording media address in the control file is determined. Is automatically detected and set in the recording means 7, and the content scene recorded at the recording media address from the recording means 7 is reproduced on the monitor 11. With the configuration described above, it is possible to provide an apparatus that can detect a scene to be viewed immediately upon detecting metadata.
If a thumbnail file linked to the content is prepared, a representative thumbnail image of the content can be reproduced and displayed when displaying a list of content names that match the keyword described above.
(Embodiment 4)
Next, another embodiment will be described.
In the first to third embodiments, a system for adding metadata to pre-recorded content has been described. However, the present invention relates to a system for adding metadata at the time of shooting, such as a camera, and particularly, the content is limited in advance. It can be extended to a system that takes a landscape or a shooting position as metadata.
FIG. 6 shows the configuration of this system as a fourth embodiment.
At the same time that the camera 51 captures an image and the video content is recorded in the content DB 54, the location where the camera is capturing is detected by the GPS 52, and the position information (longitude and latitude values) output from the GPS 52 is output by the The position information converted into an audio signal is also recorded on another audio channel. In this case, the camera with recording means 50, the camera 51, the GPS 52, the voice synthesis 53, and the content DB 54 may be integrally configured.
Next, the content DB 54 inputs the position information of the audio signal recorded in the audio channel to the audio recognition means 56.
Here, the area names, landmarks and the like of the dictionary DB 55 are selected and restricted from the terminal 105 by a key input board or the like, and output to the voice recognition means 56.
The voice recognition means 56 detects a region name and a landmark using the recognized background numerical value and the data of the dictionary DB 55, and outputs it to the file processing means 57.
Next, the file processing unit 57 TEXT-converts the time code output from the content DB 54 and the area name and landmark output from the voice recognition unit 56 as metadata to generate a metadata file.
Next, the recording unit 58 records the metadata file output from the file processing unit 57 and the content data output from the content DB 54.
With this configuration, it is possible to automatically add the metadata of the area name and the landmark to each captured scene.
[0016]
In each of the above embodiments, generally, there is a possibility that some erroneous recognition will occur in the speech recognition. When erroneous recognition occurs, the produced metadata and tags can be corrected using information processing means such as computer means.
[0017]
In addition, the speech recognition unit according to the present invention employs a word recognition method that performs speech recognition in units of words, and particularly improves the speech recognition rate by limiting the number of words in speech input and the number of words in a recognition dictionary to be used. be able to.
[0018]
Further, in the present invention, the configuration in which the keyword recognized by the voice recognition unit is filed together with the time code in the metadata file is described. However, in addition to the keyword recognized by the voice recognition unit, the related keyword is added and filed. Is also good.
[0019]
For example, when the Yodogawa is recognized by voice, a file with general attribute keywords such as topography and river is also added. By doing so, at the time of searching, keywords such as added terrain and river can be used, so that searchability can be improved.
[0020]
【The invention's effect】
As described above, according to the present invention, in creating or tagging metadata related to content, a keyword extracted in advance from a scenario or the like of the created content is input as an audio signal, and based on the scenario. Since the dictionary fields are set and keywords are prioritized, metadata can be efficiently and accurately generated from the voice recognition means.
In particular, it is difficult to input metadata at intervals of several seconds, but it is difficult to input metadata by using key input. Can be given.
In addition, by using the control file and the metadata indicating the recording position of the content, and the metadata file indicating the time code and the like, a uniquely required scene is searched for from the meta keyword, and Can be played.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a metadata production device according to Embodiment 1 of the present invention. FIG. 2 is a configuration diagram illustrating an example of a dictionary DB according to the present invention. FIG. 3 is an example of a scenario according to the present invention. FIG. 4 is a TEXT format data diagram showing an example of a metadata file according to the present invention. FIG. 5 is a block diagram showing a configuration of a metadata search device according to Embodiment 2 of the present invention. FIG. 7 is a configuration diagram showing an example of an information file of the present invention. FIG. 7 is a block diagram showing a configuration of a metadata search device according to a third embodiment of the present invention. FIG. 8 is a metadata production according to a fourth embodiment of the present invention. Block diagram showing the configuration of the device.
1 Content DB
2 input means 3 voice recognition means 4 dictionary DB4
5 File processing means 6 Content information file processing means 7 Recording means 11 Video monitor 50 Camera with recording device 51 Camera 52 GPS
53 Voice synthesis means 54 Content DB
55 Dictionary DB
56 voice recognition means 57 file processing means 58 recording means 101 voice input terminal 102 dictionary field selection input terminal 105 dictionary place name selection input terminal

Claims (8)

An apparatus for producing metadata related to content,
Input means for inputting a voice signal of a content management keyword extracted from a scenario or the content of the content created according to the generated content, and converting the voice signal into data;
Voice recognition means for recognizing a keyword from voice signal data digitized by the input means;
A metadata processing apparatus comprising: a file processing unit that stores a keyword output from the voice recognition unit in a metadata file together with a time code indicating a time position with respect to an image signal included in content. 2. The metadata producing apparatus according to claim 1, wherein the voice recognition unit prepares a plurality of dictionaries for each genre and selects a dictionary of a genre suitable for the content. The voice recognition means prepares a plurality of dictionaries for each genre, selects a dictionary of a genre suitable for the content, and further preferentially recognizes a keyword extracted from the scenario or the content of the content. The metadata production device according to claim 1. An apparatus for producing metadata related to content,
Input means for inputting a voice signal of a narration to the produced content, and converting the voice signal into data.
A plurality of dictionaries for each genre are prepared, a dictionary of a genre suitable for the content is selected, and a scenario of the content or a keyword extracted from the content of the content is set to be preferentially recognized. Voice recognition means for recognizing a keyword from the voice signal data,
A metadata processing apparatus comprising: a file processing unit that stores a keyword output from the voice recognition unit in a metadata file together with a time code indicating a time position with respect to an image signal included in content. A search device for metadata related to content,
Input means for inputting a voice signal of a content management keyword extracted from a scenario or the content of the content created according to the generated content, and converting the voice signal into data;
Voice recognition means for recognizing a keyword from voice signal data digitized by the input means;
File processing means for storing the keyword output from the voice recognition means in a metadata file together with a time code indicating a time position with respect to an image signal included in the content,
Content information file processing means for generating a control file for managing the relationship between the recording position of the content file and the metadata file,
A metadata search device comprising: a recording unit that records the content file, the metadata file, and the control file. The control file output from the content information file processing unit is a table that specifies the content recording position of the recording unit according to the recording time of the content, and the recording position of the content can be searched from the time code. The metadata search device according to claim 5, wherein: The voice recognition means prepares a plurality of genre-specific dictionaries, selects a dictionary of a genre suitable for the content, and further preferentially recognizes the scenario or a keyword extracted from the content of the content. 7. The metadata search device according to claim 5, wherein A search device for metadata related to content,
Input means for inputting a voice signal of a content management keyword extracted from a scenario or the content of the content created according to the generated content, and converting the voice signal into data;
A plurality of dictionaries for each genre are prepared, a dictionary of a genre suitable for the content is selected, and voice recognition means for recognizing a keyword from the voice signal data digitized by the input means,
File processing means for storing a keyword output from the voice recognition means in a metadata file together with a time code indicating a time position with respect to an image signal included in the content;
Content information file processing means for generating a control file that manages the relationship between the recording position of the content file and the metadata file,
Recording means for recording the content file, the metadata file, and the control file together,
A keyword is selected from the common dictionary used by the voice recognition unit, the metadata file in which the selected keyword is recorded is specified, and a content to be searched is searched for from contents recorded in the recording unit. And a search unit for searching a recording position of a scene to be searched from the control file,
A metadata production apparatus and a retrieval apparatus, wherein content management metadata is automatically generated, and a recording position of a content file recorded in the recording means is specified from the metadata.

Images