JP4866396B2 - Tag information adding device, tag information adding method, and computer program - Google Patents

Description

The present invention relates to a tag information adding device, a tag information adding method, and a computer program for adding tag information for searching a user's desired image frame to a plurality of image frames.
The present invention particularly relates to tag information for searching for a desired image frame of a user for a plurality of image frames constituting a moving image file shot by a video camera or a still image file shot by a still camera or the like. The present invention relates to a tag information adding device, a tag information adding method, and a computer program.

As a conventional technique, Patent Document 1 below describes a method in which an operator performs an input operation for adding a tag to a frame in order to highlight a desired frame of a moving image being reproduced. Further, as another conventional technique, Patent Document 2 below describes a method in which an operator performs an input operation for adding a key image to a desired partial area in an image frame.
Japanese Patent Laying-Open No. 2005-181599 (FIG. 13) Japanese Patent Laying-Open No. 2007-19768 (FIG. 10)

  However, the above-described conventional technique has a problem that it takes an enormous amount of time for the input operation work because the operator needs to perform an input operation for adding a tag.

  In view of the above problems of the prior art, the present invention automatically generates tag information for searching for a desired image frame for a plurality of image frames constituting a moving image file or a still image file. It is an object to provide a tag information adding device, a tag information adding method, and a computer program that can be added.

In order to achieve the above object, the tag information adding apparatus of the present invention automatically adds tag information for searching for a user's desired image frame to a plurality of image frames imaged and generated by the imaging means. A tag information adding device that generates and adds to a current position detecting means for detecting a current position of the imaging means during imaging by the imaging means, and a current position detected by the current position detecting means, Using map data having a geographical name, peripheral keyword extracting means for extracting a geographical name near the current position as a peripheral keyword, voice recognition means for recognizing a user's voice during imaging by the imaging means, When the voice is not recognized by the voice recognition means, the current position is detected from the peripheral keywords extracted by the peripheral keyword extraction means. Wherein the first tag information adding means for adding a tag information to the image frame time, if the voice is recognized by the voice recognition unit, from the recognized speech to extract noun, extracted with the noun Comparing with the peripheral keyword extracted by the peripheral keyword extracting means, the matched noun is added as tag information to the image frame at the time when the speech is recognized, and if it does not match, the noun or the peripheral keyword a second tag information adding means for adding a tag information to the image frame at the time when the voice is recognized, with.

In order to achieve the above object, the tag information adding method of the present invention automatically applies tag information for searching a user's desired image frame to a plurality of image frames captured and generated by the imaging means. generated by a tag information adding process for adding, the current position and the current position detection step of detecting, the detected current position detecting step current position and the map of the imaging means in the imaging by the imaging means based on the data, and a peripheral keyword extracting a geographical name of the current location near the neighborhood keyword, a speech recognition step for recognizing the voice of the user during imaging by the imaging means, the voice by the speech recognition step If not recognized, the field at which the peripheral keyword extracted by the peripheral keyword extraction step, the current position is detected A first tag information adding step of adding a tag information to the frame, when the voice is recognized by the speech recognition step extracts the noun from the recognized speech, the peripheral keyword extraction and extracted the noun Comparing with the peripheral keywords extracted in the step, the matched noun is added as tag information to the image frame at the time when the voice is recognized, and if it does not match, the noun or the peripheral keyword is added to the voice And a second tag information adding step for adding as tag information to the image frame at the time when is recognized .

In order to achieve the above object, the computer program according to the present invention enables the computer to automatically generate tag information for searching for a user's desired image frame for a plurality of image frames captured and generated by the imaging means. A computer program for generating and adding to a current position detecting step for detecting a current position of the imaging means during imaging by the imaging means, and a current position and map data detected in the current position detecting step Based on the above, a peripheral keyword extraction step of extracting a geographical name near the current position as a peripheral keyword, a voice recognition step of recognizing a user's voice during imaging by the imaging means, and voice recognition by the voice recognition step if not, the peripheral keyword extracted by the peripheral keyword extracting step, before A first tag information adding step of adding a tag information to the image frames of the current when the position is detected, when the voice is recognized by the speech recognition step extracts the noun from the recognized speech, When the extracted noun is compared with the peripheral keyword extracted in the peripheral keyword extraction step, the matched noun is added as tag information to the image frame at the time when the speech is recognized. A second tag information adding step of adding the noun or the peripheral keyword as tag information to the image frame at the time when the sound is recognized .

  With this configuration, tag information for searching for a user's desired image frame can be automatically generated and added to a plurality of image frames constituting a moving image file or a still image file.

  According to the present invention, tag information for searching for a desired image frame of a user is automatically added to a plurality of image frames constituting a moving image file or a still image file generated by being imaged by an imaging means. Can be generated and added.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a tag information adding apparatus according to the present invention, and FIG. 2 is a flowchart for explaining the operation of the moving picture processing apparatus of FIG.

  In FIG. 1, an outside camera 11 that is an image capturing unit captures a surrounding moving image of a vehicle (not shown) such as a front, and an in-vehicle camera 12 that is another image capturing unit captures an in-vehicle moving image such as a driver inside the vehicle. Are arranged as follows. The in-vehicle microphone 13 records the voice of the driver in the vehicle, and the GPS (Global Positioning System) unit 14 acquires the current position (latitude, longitude) of the vehicle. The voice recognition device 15 recognizes the voice recorded by the in-vehicle microphone 13. The moving image processing device 16 is configured to obtain audio data obtained by the in-vehicle microphone 13 and the voice recognition device 15 for an arbitrary frame among a plurality of frames constituting each moving image captured by the in-vehicle camera 11 and the in-vehicle camera 12; A search tag or the like is automatically generated and added based on the current position acquired by the GPS unit 14 and recorded in the moving image recording device 17 as tagged moving image data. Here, the moving image processing device 16 stores map data in advance, and acquires a geographical name (name of place, name of mountain, river, bridge, building, etc.) in the vicinity of the current position as a peripheral key code from the map data. be able to. The number of peripheral key codes to be acquired is preferably a plurality, for example, about 20.

  The operation of the moving image processing device 16 will be described with reference to FIG. Here, the structure of the moving image information of the moving image data includes a moving image data name (for example, a number indicating the shooting order), a keyword serving as a tag for searching for a user's desired image frame, and a start frame to which the same tag is assigned Number, end frame number, position data (latitude, longitude) and the like are included. Here, the number of frames to which the same tag is assigned is uniquely 100 frames (= 1 block) so that it can be easily seen at the time of search display.

  First, it is checked whether there is moving image data from the outside camera 11 and the inside camera 12 (step S1). If the moving image data exists in step S1, the process proceeds to step S2, and the moving image data name is described in the moving image information. Next, it is checked whether or not position data from the GPS unit 14 exists (step S3). If there is no position data, the process proceeds to step S11, and the moving image information is recorded in the moving image recording device 17. If position data exists in step S3, the process proceeds to step S4, and the position data is described in the moving image information. Next, one or more geographical names near the current position are extracted as peripheral keywords based on the position data and map data (step S5).

  Next, it is checked whether or not there is voice data from the voice recognition device 15 (step S6). If there is no audio data, the process proceeds to step S12, where the moving image frame number at the time when the position data is acquired is set as the start frame number, and the frame after 100 frames is set as the end frame number, and one or more extracted from the position data in step S5 Are described in the tag of the moving image information, and then the process proceeds to step S11 to record the moving image information in the moving image recording device 17.

  If there is speech data in step S6, the process proceeds to step S7, where a noun is extracted from the speech data, and then the extracted noun is matched with one or more peripheral keywords extracted from the location data in step S5 ( Step S8). If not matched, one or more peripheral keywords extracted from the position data branching to step S9 are described in the video information tag. On the other hand, if matched, the process proceeds to step S10 and the matched noun is converted to the video information. Describe in the tag. In steps S9 and S10, the moving image frame number at the time when the audio data is acquired is set as the starting frame number, and the frame after 100 frames is set as the ending frame number. Then, the process proceeds to step S11 and moving image information is recorded in the moving image recording device 17.

  FIG. 3 shows an example of moving image information given to moving image data shot in the vicinity of Mt. Fuji. In FIG. 3, latitude data (latitude) and longitude data (Longitude) and a keyword = Gotemba are assigned to one block from the start frame number = 1 to the end frame number = 100 of the video data of Mt. Fuji. An example is shown in which latitude data (latitude) and longitude data (Longitude) and keyword = Mt. Fuji are assigned to one block from frame number = 90 to end frame number = 189.

  FIG. 4 shows an example of moving image information given to moving image data of Mt. Fuji for which no audio data exists. In FIG. 4, latitude and longitude data and peripheral keywords extracted from position data in step S5 in FIG. 3 = Gotemba, Mt. Fuji, and Ashigara are assigned to one block from start frame number = 1 to end frame number = 100. An example is shown.

  FIG. 5 shows an example of the moving image information given to the moving image data of Mt. Fuji in which the audio data “Today's Mt. Fuji, beautiful” exists from the middle of the moving image data (frame number = 90). In FIG. 5, latitude and longitude data and peripheral keywords extracted from the position data in step S5 = Gotemba are assigned to one block from start frame number = 1 to end frame number = 100, and start frame number = An example in which latitude and longitude data and the noun “Mt. Fuji” extracted by speech recognition are assigned to one block from 90 to end frame number = 189.

<Application example of the present invention>
FIG. 6 shows a system to which the tag information adding apparatus 10 according to the present invention is applied. In the system shown in FIG. 6, the tag information adding device 10 according to the present invention is mounted on a vehicle V and used as a moving image database (DB) 10a, and constitutes a surrounding moving image and an in-vehicle moving image shot while the vehicle V is traveling. A search tag is automatically generated and added to any frame of a plurality of frames that are recorded while the vehicle V is traveling based on the current position and recorded as tagged moving image data. To do. The moving image data recorded in the tag information adding device 10 (moving image DB 10a) is uploaded to the server 1 and downloaded from the server 1 to the mobile phone 2 of the user Y, the PC (personal computer) 3, and the in-vehicle information terminal 4.

  The imaging means of the present invention is not limited to a video camera, but can be applied to a normal portable video camera, a digital still camera, and a camera with a built-in mobile phone. Also, the tag addition target video is not limited to a moving image, and can be applied to an image file composed of a plurality of still images.

  The present invention automatically generates tag information for searching for a user's desired image frame for a plurality of image frames constituting a moving image file or a still image file generated by being picked up by an image pickup means. And can be used for a video camera, a still camera, a mobile phone, and the like.

It is a block diagram which shows one Embodiment of the tag information addition apparatus which concerns on this invention. 3 is a flowchart for explaining the operation of the moving image processing apparatus of FIG. 1. It is explanatory drawing which shows moving image information. It is explanatory drawing which shows the moving image information when there is no audio | voice data. It is explanatory drawing which shows the moving image information when there exists audio | voice data. It is explanatory drawing which shows the system to which the tag information addition apparatus which concerns on this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Out-of-vehicle camera 12 In-vehicle camera 13 In-vehicle microphone 14 GPS part 15 Voice recognition device 16 Movie processing device 17 Movie recording device

Claims (6)

A tag information adding device that automatically generates and adds tag information for searching for a desired image frame of a user to a plurality of image frames that are captured and generated by an imaging unit,
A current position detecting means for detecting a current position of the imaging means during imaging by the imaging means;
Based on the current position detected by the current position detection means, using map data having a geographical name, a peripheral keyword extraction means for extracting a geographical name near the current position as a peripheral keyword;
Voice recognition means for recognizing a user's voice during imaging by the imaging means;
First tag information addition for adding the peripheral keyword extracted by the peripheral keyword extraction unit as tag information to the image frame at the time when the current position is detected when no voice is recognized by the voice recognition unit Means,
When a speech is recognized by the speech recognition means, a noun is extracted from the recognized speech, the extracted noun is compared with the peripheral keywords extracted by the peripheral keyword extraction means, and the matched noun is A tag information is added as tag information to the image frame at the time when the voice is recognized, and if it does not match, the noun or the peripheral keyword is added as tag information to the image frame at the time when the voice is recognized. Tag information adding means of
Provided tag information adding device. The first and second tag information adding means respectively add the same tag information to a predetermined number of image frames from the image frame at the time when the current position is detected and the image frame at the time when the sound is recognized. The tag information adding apparatus according to claim 1 . The tag information adding apparatus according to claim 1 or 2, wherein the plurality of image frames are moving image data photographed by a video camera used as the imaging unit. 4. The tag information adding device according to claim 1, wherein the plurality of image frames are still image files photographed by a still camera used as the imaging unit or a camera built in a mobile phone. 5. A tag information adding method for automatically generating and adding tag information for searching for a user's desired image frame to a plurality of image frames captured and generated by an imaging means,
A current position detecting step for detecting a current position of the imaging means during imaging by the imaging means;
Based on the current position detected in the current position detection step and map data, a peripheral keyword extraction step for extracting a geographical name near the current position as a peripheral keyword;
A voice recognition step of recognizing a user's voice during imaging by the imaging means;
First tag information addition for adding the peripheral keyword extracted in the peripheral keyword extraction step as tag information to the image frame at the time when the current position is detected when no voice is recognized in the voice recognition step Steps,
When a speech is recognized in the speech recognition step, a noun is extracted from the recognized speech, the extracted noun is compared with the peripheral keywords extracted in the peripheral keyword extraction step, and the matched noun is A tag information is added as tag information to the image frame at the time when the voice is recognized, and if it does not match, the noun or the peripheral keyword is added as tag information to the image frame at the time when the voice is recognized. Tag information addition step of
Provided tag information addition method. A computer program for automatically generating and adding tag information for searching for a user's desired image frame to a plurality of image frames captured and generated by an imaging means,
A current position detecting step for detecting a current position of the imaging means during imaging by the imaging means;
Based on the current position detected in the current position detection step and map data, a peripheral keyword extraction step for extracting a geographical name near the current position as a peripheral keyword;
A voice recognition step of recognizing a user's voice during imaging by the imaging means;
First tag information addition for adding the peripheral keyword extracted in the peripheral keyword extraction step as tag information to the image frame at the time when the current position is detected when no voice is recognized in the voice recognition step Steps,
When a speech is recognized in the speech recognition step, a noun is extracted from the recognized speech, the extracted noun is compared with the peripheral keywords extracted in the peripheral keyword extraction step, and the matched noun is A tag information is added as tag information to the image frame at the time when the voice is recognized, and if it does not match, the noun or the peripheral keyword is added as tag information to the image frame at the time when the voice is recognized. Tag information addition step of
Computer program provided.

Images