JP2016009262A - Moving image processing device, moving image processing program, and moving image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device, program, and method by which a user easily processes a moving image.SOLUTION: When first information is selected out of a plurality of pieces of information about moving image data, first related information that is related to the first information is displayed on a display unit 20. The moving image data is processed on the basis of the first information. In addition, the moving image data is processed on the basis of the first information and second information that is selected out of the first related information. When the second information is selected, a moving image based on either the second related information that is related to the second information or the processed moving image data is displayed.

Description

  The present invention relates to a moving image processing apparatus, a moving image processing program, and a moving image processing method.

  2. Description of the Related Art Conventionally, there is known a moving image search method in which an index understandable to human beings is assigned in units of frames and an arbitrary scene can be searched using the index. For example, in Patent Document 1, a computer automatically selects a plurality of words from a word database based on a physical quantity obtained by processing moving image contents, and assigns them to each frame as an index. A technique is disclosed in which a word corresponding to a scene to be searched is input when a search is performed, and a frame of a matching moving image is used as a candidate image.

JP-A-5-282380

  However, in the prior art, the indexes are not related to each other. Therefore, processing a moving image has been a troublesome operation for the user.

  The present invention has been made in view of such problems, and an object thereof is to allow a user to easily process a moving image.

  The moving image processing apparatus according to an aspect of the present invention includes a control unit that causes a display unit to display first related information related to the first information when first information is selected from a plurality of pieces of information related to moving image data. And a processing unit that processes the moving image data based on at least the first information.

  In the moving image processing method according to an aspect of the present invention, when first information is selected from among a plurality of pieces of information regarding moving image data, first related information related to the first information is displayed on a display unit, and at least the first information The moving image data is processed based on one information.

  A moving image processing program according to an aspect of the present invention includes a control step of causing a display unit to display first related information related to the first information when first information is selected from among a plurality of pieces of information related to moving image data. And a processing step of processing the moving image data based on at least the first information.

  According to the present invention, a user can easily process moving image data.

2 is a functional block diagram of the moving image processing apparatus 1 according to Embodiment 1. FIG. 3 is an example of an information table according to the first embodiment. 3 is an example of an information table according to the first embodiment. 4 is a flowchart illustrating an operation of the moving image processing apparatus 1 according to the first embodiment. 4 is an example of display on the display unit 20 according to the first embodiment. 4 is an example of display on the display unit 20 according to the first embodiment. 4 is an example of display on the display unit 20 according to the first embodiment. 4 is an example of display on the display unit 20 according to the first embodiment. 4 is an example of display on the display unit 20 according to the first embodiment. 6 is a functional block diagram of a moving image processing apparatus 1 according to Embodiment 2. FIG. 6 is a flowchart illustrating an operation of the moving image processing apparatus 1 according to the second embodiment. It is an example of the display of the display part 20 by Embodiment 2. FIG. It is an example of the display of the display part 20 by Embodiment 2. FIG. It is an example of the display of the display part 20 by Embodiment 2. FIG. It is a functional block diagram of the moving image processing apparatus 1 according to Embodiment 3. 14 is a flowchart illustrating an operation of the moving image processing apparatus 1 according to the third embodiment. It is an example of the information table by Embodiment 3. It is an example of the display of the display part 20 by Embodiment 3. FIG. It is a functional block diagram of the moving image processing apparatus 1 according to Embodiment 4. 10 is a flowchart illustrating an operation of the moving image processing apparatus 1 according to the fourth embodiment. It is an example of the display of the display part 20 by Embodiment 4. It is an example of the display of the display part 20 by Embodiment 4. It is an example of the display of the display part 20 by Embodiment 4. It is an example of the display of the display part 20 by Embodiment 4. It is an example of the display of the display part 20 by Embodiment 4. It is an example of the display of the display part 20 by Embodiment 4. It is an example of the display of the display part 20 by this invention.

The moving image processing apparatus according to the present invention will be described below with reference to the drawings and flowcharts.
Embodiment 1

  FIG. 1 is an example of a functional block diagram of a moving image processing apparatus 1 according to the present embodiment.

  The moving image processing apparatus 1 includes a moving image acquisition unit 10, a processing unit 11, and a control unit 12.

  The external device 2 includes a display unit 20 and a selection unit 21. Examples of the display unit 20 of the present embodiment include a display of a personal computer, a smartphone, a tablet, a digital camera, and other electronic devices. Examples of the selection unit 21 include a mouse, a keyboard, a microphone, a trackball, a camera, a light pen, and a touch panel.

  Under the control of the control unit 12, the video processing device 1 is connected to the external storage device such as a USB memory, a memory card, or a memory stick connected to the video acquisition unit 10 via the video acquisition unit 10 or via the Internet. Get a video file stored on the server. The moving image file in this embodiment includes moving image data, audio data, a signal for synchronizing moving image data and audio data, and the like. Text such as subtitles and metadata may be included. Furthermore, in this embodiment, it is assumed that a plurality of pieces of information relating to moving image data are associated in advance, and these pieces of information are included in the moving image file. That is, the moving image acquisition unit 10 acquires a moving image file including an information table in which a plurality of pieces of information related to moving image data are stored in association with each other as illustrated in FIG. 2A or 2B. The plurality of pieces of information related to the moving image data in the present embodiment includes various subjects such as the subject recorded in the moving image data, the operation of the subject, the size of the subject, the location, the time, the weather, the situation, the color, the sound luminance, and the edge. Information. In FIG. 2a or 2b, the information is stored as characters in the information table, but may be stored as an image or an icon.

  The control unit 12 causes the display unit 20 of the external device 2 to display at least one of the information stored in the information table acquired by the moving image acquisition unit 10. The selection unit 21 of the external device 2 selects information displayed on the display unit 20. Here, when there is information associated with the information displayed on the display unit 20, there is a display method in which information associated with the information is displayed on the display unit 20 when the information is selected. For example, the control unit 12 of the moving image processing apparatus 1 acquires the information selected by the selection unit 21 and causes the display unit 20 to display information associated with the acquired information.

  The processing unit 11 processes the moving image data acquired via the moving image acquisition unit 10 based on the information acquired from the selection unit 21 by the control unit 12. The processing of moving image data will be described in detail later.

  The control unit 12 decodes the moving image data processed by the processing unit 11 and causes the display unit 20 to reproduce the moving image. At this time, audio data, a signal for synchronizing the video data and audio data, and text such as captions and metadata processed together with the video data may be combined to reproduce the video. Only moving image data may be combined and reproduced as a moving image. The control unit 12 controls the moving image acquisition unit 10 and the processing unit 11 of the moving image processing apparatus 1. Further, the control unit 12 controls the display unit 20 and the selection unit 21 of the external device 2.

  Next, the operation of the moving image processing apparatus 1 according to the present embodiment will be described using the flowchart shown in FIG.

  (S1) The user turns on the switch of the moving image processing apparatus 1. In the present embodiment, the process shown in FIG. 3 is executed based on the user turning on the switch of the moving image processing apparatus 1, but the present invention is not limited to this. The process shown in FIG. 3 may be executed by displaying a menu screen on the display unit 20 of the external device 2 and operating the menu screen. Further, the operation shown in FIG. 3 may be automatically executed without necessarily requiring any user operation.

  (S2) The moving image acquisition unit 10 acquires a moving image file. The moving image file in the present embodiment includes moving image data, audio data, a signal for synchronizing moving image data and audio data, text such as subtitles and metadata, an information table, and the like. In the present embodiment, the information table is a table in which a plurality of pieces of information regarding moving image data are stored in association with each other in advance. FIG. 2A is an example of an information table corresponding to moving image data in which a scene “A-kun runs at an athletic meet” is recorded. FIG. 2B is an example of an information table corresponding to moving image data in which two scenes, a scene where “A-kun runs at an athletic meet”, a scene “B-kun eats at an athletic meet”, and two scenes are recorded. In the information table, a plurality of pieces of information relating to audio data, subtitles, text, and the like may be stored in association with each other in advance. This information table may be stored in advance in association with moving image data (external storage devices such as a USB memory, a memory card, and a memory stick, a server connected via the Internet, etc.). The apparatus 1 may create it. The case where the moving image processing apparatus 1 creates will be described in detail in the second embodiment.

  (S3) The control unit 12 displays at least one of the information stored in the information table acquired by the moving image acquisition unit 10 in the first area 201 of the display unit 20 of the external device 2 (FIG. 4a). For example, when an information table as shown in FIG. 2B is acquired, “A-kun”, “Athletic meet”, “Run”, “B-kun”, and “Eat” are displayed in the first area 201 of the display unit 20 as shown in FIG. The Examples of the display method include a method of displaying characters, images, icons, or the like corresponding to each information in the first area 201 of the display unit 20. In this embodiment, each information is displayed as a character. A method for displaying an image or icon corresponding to each information will be described in detail in the second embodiment.

  (S4) The user selects one piece of information displayed on the display unit 20 as the first information 401 via the selection unit 21 of the external device 2. The selection unit 21 may be selected by moving a cursor as shown in FIG. 4a using a mouse, or selected by a user using a keyboard, microphone, trackball, camera, light pen, touch panel, or the like. May be. If no information is selected, the process returns to S3 and continues to display information. If there is no selection from the user for a certain period of time, the processing may be terminated. If information is selected, the process proceeds to S5.

  (S5) The control unit 12 causes the display unit 20 to display information associated with the information selected by the user via the selection unit 21. For example, the control unit 12 acquires information selected by the selection unit 21 among the information displayed in the first area 201 as the first information 401. Further, the control unit 12 displays information associated with the first information 401 as the first related information 302 in the second area 202 of the display unit 20. In FIG. 4b, the same configurations as in FIG. FIG. 4B is an example in which “A-kun” is selected by the selection unit 21 from the information displayed in the first area 201 and the first related information 302 associated with “A-kun” is displayed in the second area 202. It is. This display example is a case where an information table as shown in FIG. “A-kun” is the first information 401. At this time, the first information 401 may be displayed with emphasis. For example, the character color of “A-kun” is changed, the size of the character is increased, the thickness of the character is increased, and the character is displayed with a movement such as rotation. In addition, information displayed in the first area other than the first information 401 (“athletic meet”, “run”, “B-kun”, “eating”) may be displayed lightly or not. If the 1st related information 302 is displayed, it will progress to S6. When the user selects the execution button 100 for instructing execution of the moving image data processing, the process may proceed to S8.

  (S6) The user selects one of the first related information 302 as the second information 402 via the selection unit 21 of the external device 2. The control unit 12 acquires information selected by the user as the second information 402 (FIG. 4c). In FIG. 4c, the same configurations as those in FIG. 4a or b are omitted from the numbers. If no information is selected, the process returns to S5 and continues to display the first related information 302. Further, if there is no selection from the user for a certain period of time, the process may be terminated, or the process may proceed to S8. If information is selected, the process proceeds to S7. When the user selects the execution button 100 for instructing execution of the moving image data processing, the process may proceed to S8.

  (S7) The user determines whether or not to further display information associated with the first information 401 and the second information 402. When displaying, it returns to S5. That is, the control unit 12 displays information associated with the first information 401 and the second information 402 as the second related information 303 in the third region 203 as illustrated in FIG. The information associated with the first information 401 and the second information 402 can be determined from the information table shown in FIG. 2b. Information associated with the second information 402 may be displayed. When there is no information to be displayed (when there is no information associated with the selected information), information indicating that there is no more information associated with the selected information may be displayed on the display unit 20. Good. When not displaying, it progresses to S8. When the user selects the execution button 100 for instructing execution of the moving image data processing, the process may proceed to S8.

  (S8) The processing unit 11 processes the moving image data acquired via the moving image acquisition unit 10 based on the information selected by the selection unit 21 (first information 401 and the like). For example, as illustrated in FIG. 4 d, it is assumed that the user selects “A-kun” as the first information 401, “Run” as the second information 402, and “Athletic meet” as the third information 403. The processing unit 11 acquires the scene “A-kun runs at the athletic meet” from the moving image data acquired through the moving image acquisition unit 10. Only the scene of at least the first information 401 “A-kun” may be acquired. You may make it acquire the scene of the 1st information 401 "A-kun" and the 2nd information 402 "run", ie, the "A-kun runs" scene. As an acquisition method, each information is associated with time information indicating where the information corresponds to the scene in the video data, and the corresponding scene is acquired from the video data based on the time information. A method is conceivable. For example, the scene corresponding to “A-kun” is 0 to 30 seconds, the scene corresponding to “Run” is 10 to 30 seconds and 40 to 50 seconds, and the scene corresponding to “Athletic meet” is 0 to 50 seconds. And In this case, when the moving image data is processed based on “A-kun”, “Run”, and “Athletic meet”, the processing unit 11 acquires moving image data in the range of 10 to 30 seconds. In addition, a frame based on the moving image data may be tagged with information related to the frame, and a frame in which “A-kun”, “Run”, and “Athletic meet” are all tagged may be acquired.

  (S9) As illustrated in FIG. 4D, the control unit 12 causes the moving image area 204 of the display unit 20 to reproduce a moving image based on the moving image data in the range acquired by the processing unit in S8. Alternatively, a representative image (thumbnail image) based on the moving image data in the range acquired by the processing unit may be displayed in the moving image region 204, and when the user selects a representative image via the selection unit 21, the moving image may be played back. The representative image may be the top image at the time of moving image reproduction, the image with the largest subject size, the image within the range where the subject movement is small, or the like. Instead of a representative image, an icon indicating reproduction may be used. Also, audio data corresponding to the moving image data in the range acquired by the processing unit, a signal for synchronizing the moving image data and audio data, and text such as subtitles and metadata may be reproduced together.

  As described above, according to the present embodiment, since each piece of information is associated with each other, the user can easily process moving image data. Further, since information associated with each information is displayed, it is possible to provide a display that is easy for the user to see. Furthermore, since the content of the moving image can be provided in characters, the user can easily grasp the content of the moving image.

  In the present embodiment, the information table is included in the moving image file, but is not limited thereto. For example, the moving image file includes an address indicating the storage location of the information table, and the information table may be used by referring to the address.

  In the present embodiment, the moving image processing apparatus 1 has acquired a moving image file including an information table, but is not limited thereto. The information table may not be acquired. For example, a frame based on moving image data may be displayed on the display unit 20 assuming that information related to the frame is tagged and information tagged to the same frame is associated with each other. As a result, even if there is no information table, information related to the information selected by the user can be displayed.

  In the present embodiment, the display unit 20 and the selection unit 21 are provided in the external device 2, but are not limited thereto. The display unit 20 and the selection unit 21 may be provided in the moving image processing apparatus 1. For example, the case where the moving image processing apparatus 1 uses the application which performs the process of the moving image processing apparatus 1 with a camera, a smart phone, a tablet, PC etc. can be considered.

  In the present embodiment, the moving image acquisition unit 10 acquires a moving image file from an external storage device or the like, but is not limited thereto. For example, the moving image processing apparatus 1 may store a moving image file in advance.

  In this embodiment, when the switch of the moving image processing apparatus 1 is turned on and a moving image file is acquired, at least one of the information stored in the information table is displayed on the display unit 20 of the external device 2, and the displayed information However, the present invention is not limited to this. The user may input the first information 401 from an input unit (not shown) such as a microphone or a keyboard. The moving image processing apparatus 1 converts sound (sound) acquired by the microphone into an electric signal, and displays character data as a result of analyzing the converted electric signal on the display unit 20 as the first information 401. For example, when the user wants to acquire the scene where “A-kun” is depressed from the moving image data, the user utters “A-kun” to the microphone. The moving image processing apparatus 1 analyzes the voice “A-kun” by voice recognition or the like, and acquires character data “A-kun”. The moving image processing apparatus 1 displays the acquired character data “A-kun” on the display unit 20 as the first information 401. After the first information 401 “A-kun” is displayed on the display unit 20, if there is information related to the first information 401 “A-kun”, the information related to “A-kun” is displayed as the first information. The information 302 may be displayed on the display unit 20. If there is no information associated with the first information 401 “A-kun”, the display unit 20 may display that there is no information associated with “A-kun”. If there is no scene where “A-kun” is moving in the moving image data, the display unit 20 may display that there is no scene where A-kun is moving in the moving image data. A plurality of pieces of information related to the moving image data may be displayed in advance on the display unit 20, and any one of the displayed pieces of information may be selected using a microphone as the first information 401. When the user utters “I want a scene where Mr. A is depressed” to the microphone, the video processing apparatus 1 uses voice recognition or the like to write character data “I want a scene where Mr. A is depressed”. And the noun “A-kun” is acquired by morphological analysis or the like. Thereafter, the noun “A-kun” may be displayed on the display unit 20 as the first information 401. When a plurality of nouns are acquired, a plurality of nouns may be displayed on the display unit 20. Instead of nouns, verbs, adjectives, and adjective verbs may be displayed. For example, when the user utters “I want a scene to run” to the microphone, the moving image processing apparatus 1 acquires the verb “run” as character data by voice recognition, morphological analysis, or the like, and displays the first information 401 as the first information 401. 20 may be displayed. The method for acquiring the verb “run” is the same as the method for acquiring the noun. In addition, the moving image processing apparatus 1 may display character data input from the keyboard by the user on the display unit 20 as the first information 401. The method for displaying the character data on the display unit 20 as the first information 401 by the moving image processing apparatus is the same as the method for displaying the first information 401 through the microphone. The moving image processing apparatus 1 may display the first information input from the user and the first related information related to the first information on the display unit 20.

  In this embodiment, the user selects information one by one in each area, but the present invention is not limited to this. A plurality of pieces of information may be selected, and information associated with each selected piece of information may be displayed on the display unit 20. For example, as shown in FIG. 5, “A-kun” and “eat” may be selected as the first information 401 among the information displayed in the first area 201. As a result, two scenes can be simultaneously acquired from one moving image data. As shown in FIG. 5, it is assumed that “A-kun” and “eat” are selected as the first information 401 and “Athletic meet” and “B-kun” are selected as the second information 402. In that case, a scene corresponding to “A-kun” and “athletic meeting” and a scene corresponding to “eat” and “B-kun” are acquired from one video data and displayed in the video area 204.

  In the present embodiment, the first area, the second area, the third area, and the moving image area remain displayed on the display unit, but are not limited thereto. For example, the first region may be deleted from the display unit 20 at the timing when the second region is displayed on the display unit 20. The first region and the second region may be deleted from the display unit 20 at the timing when the third region is displayed on the display unit 20. Other areas may be deleted from the display unit 20 at the timing when the moving image area is displayed on the display unit 20. Further, when the display unit 20 is not sufficiently large, the user may control whether each area is displayed on the display unit 20 or not displayed on the display unit 20 by performing an operation such as scrolling. .

  As a method for acquiring a corresponding scene from the moving image data, the processing unit 11 acquires moving image data in a range that matches the time information. In this case, all the frames based on the moving image data in the range that matches the time information may be acquired, or a part thereof may be acquired. Further, the processing unit 11 acquires a frame tagged with the selected information as a method of acquiring a corresponding scene from the moving image data. In this case, all the frames tagged with the selected information may be acquired, or a part thereof may be acquired. The frame to be acquired may be manually selected by the user, or may be automatically selected by the moving image processing apparatus 1. When selecting automatically, it may be obtained by thinning out every 5 frames or the like, or a frame that is well taken by detecting camera shake or blur may be obtained. The user may specify the playback time of the moving image data, and determine the number of frames so as to fall within that time.

  In this embodiment, the information is related to each other. For example, the information “A-kun”, “B-kun”, “Sit”, and “Eat” is acquired and associated from the scene of “A-kun sits, B-kun eats”. When the user selects information “A-kun” as the first information 401, “B-kun”, “sit”, and “eat” are displayed on the display unit 20 as the first related information 302. Thereafter, it is assumed that “eat” is selected as the second information 402 among the first related information 302 “B-kun”, “sit”, and “eat”. That is, it is assumed that “A-kun” is selected as the first information 401 and “eat” is selected as the second information 402. In this case, scenes related to “A-kun” and “eat” may be displayed on the display unit 20. For example, a scene where “A-kun runs, B-kun eats”, “A-kun laughs, C-kun eats” or the like may be displayed on the display unit 20. Alternatively, since the moving image data does not include the “A-you eat” scene, the moving image processing apparatus 1 may display information indicating that there is no “A-kun eats” scene on the display unit 20. Further, the moving image processing apparatus 1 may acquire an information table in which “A-kun” and “eat”, “B-kun”, and “sit” are not related. Further, as described in the second embodiment, when the moving image processing apparatus 1 creates an information table, the information table is created without associating “A-kun” with “eating” and “B-kun” with “sitting”. . Specifically, when the information “eating” is acquired from the frame, the information “eating” is the subject acquired from the vicinity of the portion where “eating” is acquired in the frame (in this case, Mr. A). Associate with. As a result, when the first information 401 “A-kun” is selected, “eat” is not displayed as the first related information 302. If there is a scene where “A-kun eats”, “eat” is displayed on the display unit 20 as the first related information 302, and if “eat” is selected as the second information 402, a scene “A-kun eats” is displayed. It is displayed on the display 20.

The display unit 20 displays a return button, and can return to the previous process when the user presses the return button.
<< Embodiment 2 >>

  In the first embodiment, the moving image processing apparatus acquires a moving image file including an information table associated with a plurality of pieces of information regarding moving image data. In the second embodiment, the moving image processing apparatus creates an information table. In the first embodiment, information is displayed on the display unit 20 in characters. In the second embodiment, information is displayed on the display unit 20 as an image or an icon. The main difference between the moving image processing apparatus 1 according to the first embodiment and the moving image processing apparatus 1 according to the second embodiment is that an information table creation unit is added. In the present embodiment, the same components and operations as those in the first embodiment are not described.

  FIG. 6 is an example of a functional block diagram of the moving image processing apparatus 1 according to the second embodiment. In the second embodiment, the same components as those in the first embodiment are represented by the same numbers as in the first embodiment.

  The moving image processing apparatus 1 includes a moving image acquisition unit 10, an information table creation unit 13, a processing unit 11, and a control unit 12.

  Under the control of the control unit 12, the video processing device 1 is connected to the external storage device such as a USB memory, a memory card, or a memory stick connected to the video acquisition unit 10 via the video acquisition unit 10 or via the Internet. Get the video file stored on the server. In the first embodiment, an information table in which information related to moving image data is associated with each other is included in the moving image file. However, in the present embodiment, the information table may not be included in the moving image file. The moving image file in this embodiment includes moving image data, audio data, a signal for synchronizing moving image data and audio data, and the like. Text such as subtitles and metadata may be included.

  The information table creation unit 13 acquires information from the moving image data included in the moving image file acquired via the moving image acquisition unit 10, and generates the information table by associating the acquired information with each other. A method for creating the information table will be described in detail later. Information may be acquired from text such as audio data, subtitles, and metadata included in the moving image file.

  The control unit 12 causes the display unit 20 of the external device 2 to display at least one of the information stored in the information table created by the information table creation unit 13. The selection unit 21 of the external device 2 selects information displayed on the display unit 20. The control unit 12 of the video processing device 1 acquires the information selected by the selection unit 21 and controls the display unit 20 based on the acquired information. Further, the control unit 12 reproduces a moving image based on the moving image data processed by the processing unit 11 on the display unit 20.

  The processing unit 11 processes the moving image data acquired via the moving image acquisition unit 10 based on the information acquired from the selection unit 21 by the control unit 12.

  The control unit 12 controls the moving image acquisition unit 10, the information table creation unit 13, and the processing unit 11 of the moving image processing apparatus 1. Further, the control unit 12 controls the display unit 20 and the selection unit 21 of the external device 2.

  Next, the operation of the moving image processing apparatus 1 according to the present embodiment will be described using the flowchart shown in FIG. Processes similar to those in the first embodiment have the same step numbers as those in the flowchart of FIG. 3 in the first embodiment. The description of the same processing as that of the first embodiment is omitted. Further, “A” in the flowchart shown in FIG. 7 means that “A” in the flowchart shown in FIG. 3 follows.

  (S2) The moving image acquisition unit 10 acquires moving image data. In the first embodiment, a moving image file including an information table in which information relating to moving image data is associated with each other is acquired. However, in the present embodiment, the information table may not be acquired. In the present embodiment, a moving image file including text, such as moving image data, audio data, a signal for synchronizing moving image data and audio data, subtitles, and metadata is acquired.

  (S21) The information table creation unit 13 selects a frame based on the moving image data from the moving image data acquired via the moving image acquisition unit 10. Let the selected frame be frame i. In this embodiment, it is assumed that one frame is selected from the first frame of the acquired moving image data, but the present invention is not limited to this.

  (S22) The information table creation unit 13 acquires information regarding the frame i selected in S21. Information related to frame i includes the subject moving in frame i, the motion of the subject, the size of the subject, the location where frame i was recorded, the time, weather, situation, color of frame i, brightness, edge, etc. And various information. As an information acquisition method, for example, the moving image processing apparatus 1 stores in advance an image of an object that can be a subject, and stores the characteristics of the stored image and the characteristics of the frame i (for example, color, brightness, edge, etc.). The subject can be specified by comparison. Also, by using a plurality of frames and frame rates, the distance and time that the subject has moved from frame i to frame i + n is grasped, and the movement of the subject (“run”, “walk”, “stop”, etc.) Information can be acquired. Also, the size of the subject can be acquired from the edge of the subject in frame i and the zoom rate of the lens. Further, the location and time at which the frame i is recorded can be acquired from the metadata. Further, by accessing the server, it is possible to acquire the weather and situation when the frame i is recorded. Furthermore, the status such as “quiet” or “lively” can be acquired from the voice data. In addition, information related to the frame i may be acquired using an existing technique other than those described here.

  (S23) The information table creation unit 13 creates an information table by associating the information acquired in S22 with each other. For example, when three pieces of information “A-kun”, “athletic meet”, and “run” are acquired as information related to frame i, an information table as shown in FIG. Next, it is assumed that three pieces of information “B-kun”, “athletic meet”, and “eat” are acquired in frame i + n. Since the information “Sports Day” has already been recorded in the information table, “B” and “Eat” are further associated with the information “Sports Day”. As a result, the information table becomes an information table as shown in FIG.

  (S24) If the frame i is not the last frame, the process returns to S21. If it is the last frame, the process proceeds to S3.

  (S3) The control unit 12 displays at least one of the information stored in the information table created by the information table creation unit 13 in the first area 201 of the display unit 20 of the external device 2. In the first embodiment, information is displayed on the display unit 20 as characters. In this embodiment, information is displayed on the display unit 20 as an image or an icon as shown in FIG. 8a. In the display example shown in FIG. 8a, characters are also displayed, but only images or icons may be displayed without displaying characters.

  8a to 8c are display examples of the display unit 20 in the second embodiment. The display contents similar to those in the first embodiment have the same numbers as in FIG. As shown in FIG. 8a, when “B-kun” is selected as the first information 401 from the information displayed in the first area 201, the image or icon of “B-kun” is highlighted. In FIG. 8a, the image or icon frame of “B-kun” is highlighted by making it thick, but it is not limited to this. For example, the display of images or icons other than “B-kun” may be lightened or not displayed. In addition, the image or icon of “Mr. B” can be illuminated, rotated, blinked, changed color, or displayed in large size. When the first information 401 is selected, the first related information 302 is displayed in the second area 202.

  As shown in FIG. 8b, when “eat” is selected as the second information 402 from the first related information 302, an image or icon of “eat” is highlighted. When the second information 402 is selected, the second related information 303 is displayed in the third area 203. As shown in FIG. 8c, when the third information 403 is selected from the second related information 303 and the user presses the execution button 100 instructing execution of the moving image data processing, the moving image processed in the moving image area 204 by the processing unit 11 A video based on the data is played. A representative image (thumbnail image) based on the processed moving image data may be displayed in the moving image area 204, and when the user selects a representative image via the selection unit 21, the moving image may be reproduced. The representative image may be the top image at the time of moving image reproduction, the image with the largest subject size, the image within the range where the subject movement is small, or the like. Instead of a representative image, an icon indicating reproduction may be used. Also, audio data corresponding to the moving image data in the range acquired by the processing unit, a signal for synchronizing the moving image data and audio data, and text such as subtitles and metadata may be reproduced together. In the present embodiment, moving image data is processed based on three types of information “B-kun”, “eat”, and “athletic meet”. Therefore, the moving image based on the moving image data of the scene “B-kun eats at the athletic meet” is reproduced. Alternatively, a representative image of the scene “B-kun eats at the athletic meet” is displayed, and when the user selects a representative image, a moving image is reproduced. Instead of the representative image, an icon indicating playback may be used. Further, since the size of the display unit 20 is limited, when there is no space for displaying information or moving image data in the display unit 20, the user gives an instruction to scroll the display unit 20 or the like, and FIG. The display may be changed from 8b and FIG. 8c.

  Further, the user may press the execution button 100 after selecting the first information 401. In that case, a scene in which the first information 401 “B-kun” is depressed is acquired. Furthermore, the user may press the execution button 100 after selecting the second information 402. In this case, a scene based on the first selection information “B-kun” and the second selection information “eat”, that is, a scene “B-kun eats” is acquired.

  As described above, according to the present embodiment, since each piece of information is associated with each other, the user can easily process moving image data. Since information associated with each information is displayed, it is possible to provide a display that is easy for the user to see. Moreover, since the content of the moving image is displayed as an image or an icon, the user can intuitively grasp the content of the moving image. Furthermore, since information can be acquired from moving image data, it is not necessary to acquire an information table.

  In the present embodiment, the moving image processing apparatus 1 includes the information table creation unit 13, but is not limited thereto. For example, the information table creation unit 13 may be provided in the external information processing unit 3 (not shown), and the moving image processing apparatus 1 may acquire the processing result from the external information processing unit 3.

  In the present embodiment, the information table creation unit 13 acquires information from the moving image data, but is not limited thereto. For example, the user may input information from an input unit (not shown).

In this embodiment, when acquiring information from moving image data, the frames are selected one by one. However, the present invention is not limited to this. For example, the frames may be selected every 5 frames or every 10 frames. The interval to be selected may be set to automatic or may be set by the user. In addition, by selecting a plurality of images instead of one by one, it is possible to acquire information on subject movement such as “run”.
<< Embodiment 3 >>

  The third embodiment is a moving image processing apparatus that allows a user to select information as if creating a sentence when selecting information. The main difference between the moving image processing apparatus 1 according to the first embodiment and the moving image processing apparatus 1 according to the third embodiment is that a display order determining unit is added. In the present embodiment, the same components and operations as those in the first embodiment are not described.

  FIG. 9 is an example of a functional block diagram of the moving image processing apparatus 1 according to the third embodiment. In the third embodiment, the same components as those in the first embodiment are represented by the same numbers as in the first embodiment.

  The moving image processing apparatus 1 includes a moving image acquisition unit 10, a display order determination unit 14, a processing unit 11, and a control unit 12. The display order determination unit 14 includes a part-of-speech specifying unit 14a and an information classification unit 14b.

  Under the control of the control unit 12, the video processing device 1 is connected to the external storage device such as a USB memory, a memory card, or a memory stick connected to the video acquisition unit 10 via the video acquisition unit 10 or via the Internet. The moving image data stored on the server is acquired. In the third embodiment, as in the first embodiment, the information table and the moving image data in which the information related to the moving image data is associated with each other in advance are acquired. Moreover, the moving image processing apparatus 1 may include the information table creating unit 13 as in the second embodiment, and the moving image processing apparatus 1 may create the information table.

  The display order determination unit 14 determines the order of information to be displayed on the display unit 20 of the external device 2. The part of speech specifying unit 14 a specifies the part of speech corresponding to each piece of information stored in the information table acquired via the moving image acquiring unit 10. The information classifying unit 14b classifies each information into a corresponding category based on the part of speech of each information specified by the part of speech specifying unit 14a.

  The control unit 12 transmits the information table acquired via the moving image acquisition unit 10 to the display order determination unit 14 and displays information on the display unit 20 of the external device 2 based on the display order determined by the display order determination unit 14. Let The selection unit 21 of the external device 2 selects information displayed on the display unit 20. The control unit 12 acquires the information selected by the selection unit 21 and further controls the display unit 20 based on the acquired information and the display order determined by the display order determination unit 14. Further, the control unit 12 causes the display unit 20 to reproduce a moving image based on the moving image data processed by the processing unit 11.

  The processing unit 11 processes the video data acquired by the control unit 12 via the video acquisition unit 10 based on the information acquired by the control unit 12 from the selection unit 21.

  Next, the operation of the moving image processing apparatus 1 according to the present embodiment will be described using the flowchart shown in FIG. Processes similar to those in the first embodiment have the same step numbers as those in the flowchart of FIG. 3 in the first embodiment. The description of the same processing as that of the first embodiment is omitted. Further, “A” in the flowchart shown in FIG. 10 means that “A” in the flowchart shown in FIG. 3 is continued.

  (S2) The moving image acquisition unit 10 acquires a moving image file. In the present embodiment, as in the first embodiment, the moving image file includes an information table. The moving image processing apparatus 1 may include the information table creating unit 13 as in the second embodiment, and the moving image processing apparatus 1 may create the information table.

  (S31) The part-of-speech specifying unit 14a acquires the information table included in the moving image file acquired by the moving image acquiring unit 10 via the control unit 12, and sets the part of speech corresponding to each piece of information stored in the information table. Identify. In the method of specifying the part of speech, the moving image processing apparatus 1 has a word dictionary in which a word and the part of speech of the word are associated, and the part of speech of the information is specified by comparing each information with the word dictionary. An external server has the word dictionary, and the part of speech may be specified by comparing the word dictionary with each piece of information by communicating with the external server by wire or wirelessly. In addition, parts of speech corresponding to each piece of information may be specified using methods other than those described here or existing techniques.

  (S32) The information classification unit 14b classifies each information into a corresponding category based on the part of speech specified by the part of speech specification unit 14a. In the present embodiment, the categories are “subject”, “predicate”, and “object or modifier”. For example, the information classification unit 14b classifies the noun into the first category “subject”, classifies the verb into the second category “predicate”, and classifies the other parts of speech into the third category “object or modifier”.

  (S3) The control unit 12 displays information classified into the first category “subject” in the first area 201 by the information classification unit 14b.

  (S <b> 5) The control unit 12 acquires information selected by the user via the selection unit 21 among the information displayed in the first area 201 as the first information 401. The control unit 12 causes the second area 202 to display information classified into the second category “predicate” among the information associated with the first information 401 as the first related information 302. When there is no information associated with the first information 401 and classified into the second category “predicate”, information indicating that may be displayed on the display unit 20. Then, it progresses to S7. When the user selects the execution button 100 for instructing execution of the moving image data processing, the process may proceed to S8.

  (S7) The user determines whether or not to display the next information further. When displaying, it returns to S5. That is, the control unit 12 uses the information associated with the first information 401 and the second information 402 as the second related information 303 as the information related to the third category “object or modifier” as the third region. 203 is displayed. If there is no information related to the first information 401 and the second information 402 and classified into the third category “object or modifier”, information indicating that is displayed on the display unit 20. Also good. When the next information is not further displayed, the process proceeds to S8. When the user selects the execution button 100 for instructing execution of the moving image data processing, the process may proceed to S8.

  As described above, according to the present embodiment, since the user can select information in the order of subject, predicate, object, or modifier, it is possible to select information as if creating a sentence and process video data. . For example, FIG. 12 shows a display example when there is an information table as shown in FIG. In the first area 201, information “A-kun”, “B-kun”, and “C-san” classified into the first category “subject” are displayed. Of the information “run”, “lively”, “laugh”, and “fun” associated with the first information 401 “A-kun”, the information “run” and “laugh” classified into the second category “predicate” is the second area. 202. Furthermore, among the information “fun” associated with the first information 401 “A-kun” and the second information 402 “laughing”, the information “fun” classified into the third category “object or modifier” is the first. 3 areas 203 are displayed. Since “A-kun” is selected as the first information 401, “laughing” is selected as the second information 402, and “fun” is selected as the third selection information, moving image data of the scene “A-kun laughs happily” is acquired, It is displayed in area 204. Thus, since information is displayed for each category, the user can easily select information. Further, since the information is displayed from the information classified into the first category “subject”, the user can easily select the information with a sense of creating a sentence familiar to everyday use.

  In this embodiment, information is displayed on the display unit 20 in the order of subject, predicate, object, or modifier, but the present invention is not limited to this. The display order may be specified by the user in a preferred order. In addition, instructions such as “Please select a subject”, “Please select a predicate”, “Please select an object or modifier” may be displayed on the display unit 20.

  In this embodiment, the nouns are classified as the first category “subject”, the verbs as the second category “predicate”, and the other parts of speech as the third category “object or modifier”. However, the present invention is not limited to this. For example, nouns indicating person names may be classified into the first category “subject”. The noun indicating the subject may be classified into the first category “subject”. Verbs or adjectives or adjective verbs may be classified into the second category “predicate”. Verbs or adjectives or adjective verbs indicating the state of the subject may be classified into the second category “predicate”. These classification methods may be specified by the user.

In the present embodiment, the categories are “subject”, “predicate”, and “object or modifier”, but the present invention is not limited to this. The type of part of speech may be a category, or “person”, “scenery”, “motion”, “location”, “time”, “others”, or the like may be a category. There can be any number of categories. When classified by “person”, “landscape”, “motion”, “location”, “time”, “other”, etc., for example, information classified as “person” or “landscape” is set as the first category and is stored in the first area 201. Information displayed and classified into “operation” is set as the second category and displayed in the second area 202, information classified as “location” or “time” or “others” is set as the third category, and the third area is displayed. 203 may be displayed. As a result, a noun indicating the subject is first displayed, and then a verb or adjective or adjective verb indicating the state of the subject is displayed. The order of display is not limited to this.
<< Embodiment 4 >>

  The fourth embodiment is a moving image processing apparatus capable of processing moving image data by creating a sentence with selected information and allowing the user to select the created sentence. As a result, the user can process moving image data without having to select information many times. In addition, by storing the created sentence, the user can easily use the sentence once used again. The main difference between the moving image processing apparatus 1 according to the first embodiment and the moving image processing apparatus 1 according to the fourth embodiment is that a sentence creation unit and a sentence storage unit are added. In the present embodiment, the same components and operations as those in the first embodiment are not described.

  FIG. 13 is an example of a functional block diagram of the moving image processing apparatus 1 according to the fourth embodiment. In the fourth embodiment, the same components as those in the first embodiment are represented by the same numerals as those in the first embodiment.

  The moving image processing apparatus 1 includes a moving image acquisition unit 10, a text creation unit 15, a text storage unit 16, a processing unit 11, and a control unit 12.

  The sentence creation unit 15 creates a sentence based on the information selected by the selection unit 21. A method of creating a sentence will be described in detail later.

  The text storage unit 16 stores the text created by the text creation unit 15.

  The control unit 12 causes the display unit 20 to display at least one of the information stored in the information table acquired via the moving image acquisition unit 10. The selection unit 21 selects information displayed on the display unit 20. The control unit 12 acquires the information selected by the selection unit 21 and further controls the display unit 20 based on the acquired information. In addition, the control unit 12 transmits the information selected by the selection unit 21 to the text creation unit 15. The sentence creation unit 15 creates a sentence based on the information transmitted from the control unit 12 and stores it in the sentence storage unit 16. The control unit 12 acquires the text stored in the text storage unit 16 and causes the display unit 20 to display the text. Further, the control unit 12 transmits the information selected by the selection unit 21 to the processing unit 11. Further, the control unit 12 causes the display unit 20 to display the moving image data processed by the processing unit 11.

  Next, the operation of the moving image processing apparatus 1 according to the present embodiment will be described using the flowchart shown in FIG. Processes similar to those in the first embodiment have the same step numbers as those in the flowchart of FIG. 3 in the first embodiment. The description of the same processing as that of the first embodiment is omitted.

  (S41) As shown in FIG. 15, the control unit 12 causes the display unit 20 to display content for asking the user whether to select information to process moving image data or to select text to process moving image data. The processing of moving image data by selecting information is the moving image processing method described in the first to third embodiments. That is, the user selects information and processes moving image data based on the selected information. The processing of moving image data by selecting a sentence is the moving image processing method of the present embodiment. That is, the text stored in the text storage unit is displayed on the display unit 20, and the moving image data is processed based on the text selected by the user. In this case, the user can save time and effort to select information, and the moving image data can be processed with one selection. In addition, by storing the created sentence, the user can easily use the sentence once used again. If the user selects information and selects to process moving image data, the process proceeds to S3. If the user selects a sentence and selects to process moving image data, the process proceeds to S44.

  When the user selects information and selects to process moving image data and proceeds to S3, the same processing as in the first embodiment is performed from S3 to S7. Thereafter, the process proceeds to S42, where a sentence is created based on the selected information, and the created sentence is stored in the sentence storage unit 16. When the sentence is stored, the moving image data is processed by the same processing as in the first embodiment, and a moving image based on the processed moving image data is reproduced on the display unit 20.

  (S42) The text creation unit 15 acquires the information selected by the selection unit 21 via the control unit 12, and creates a text based on the acquired information. For example, it is assumed that the order of displaying information is determined as in the third embodiment, and “A-kun”, “laughing”, and “fun” are selected as shown in FIG. In this case, the subject is “A-kun”, the predicate is “laughing”, and the object or modifier is “fun”. The sentence creation unit 15 creates a sentence “A-kun laughs happily” by combining an appropriate particle or auxiliary verb with each information or changing the ending of the object or modifier. Further, it is assumed that the order for displaying information is not determined as in the first embodiment, and “A-kun”, “Run”, and “Athletic meet” are selected as shown in FIG. In this case, the sentence creation unit 15 specifies the part of speech of each information, and determines whether the noun is a person name for information in which the part of speech is identified as a noun. If there is information corresponding to the person name in the acquired information, the sentence creation unit 15 uses the information corresponding to the person name as the subject, the verb as a predicate, the other part of speech as the modifier, and the noun other than the person name as the object. And The sentence creation unit 15 creates a sentence “A-kun runs at an athletic meet” by combining an appropriate particle or auxiliary verb with each information or by transforming the ending of the object or modifier. If a sentence is created, it will progress to S43.

  (S43) The text storage unit 16 stores the text created by the text creation unit 15 in S42. When the sentence is stored, the process proceeds to S8. The moving image data is processed in the same manner as in the first embodiment, and a moving image based on the processed moving image data is reproduced on the display unit 20.

  (S44) If the user selects a sentence in S41 and selects to process moving image data, the process of S44 is performed. The control unit 12 causes the display unit 20 to display the text stored in the text storage unit 16. If there is no sentence stored in the sentence storage unit 16, the user may be notified of this. The notification method may be displayed on the display unit 20 or may be notified by sound. Processing may be terminated.

  (S45) The user selects one of the sentences displayed on the display unit 20 via the selection unit 21. If no text is selected, the process returns to S44 and continues to display the text. If there is no selection from the user for a certain period of time, the processing may be terminated. If a sentence is selected, the process proceeds to S46.

  (S46) The display unit 20 displays a content asking the user whether to process the moving image data with the selected text or to modify the selected text to process the moving image data. When the moving image data is processed with the selected text, the process proceeds to S8, where the moving image data is processed in the same process as in the first embodiment. When the moving image data is processed by partially correcting the selected sentence, the process proceeds to S47.

  (S47) The user uses the selection unit 21 to select a portion of the selected sentence to be corrected. The control unit 12 acquires the part to be corrected selected by the selection unit 21 and displays information instead. The user selects information intended by the user from the displayed information. When there is no information intended by the user in the displayed information, the user may input from an input unit (not shown). For example, it is assumed that the user selects a sentence “A-kun runs at an athletic meet” and corrects this sentence. As shown in FIG. 16, the user selects “A-kun” as a location to be corrected. Since “A-kun” is a person name, a person name other than “A-kun” stored in the information table is displayed in the display area 5 as information replacing “A-kun”. The user selects information intended by the user from the alternative information displayed in the display area 5. If the intended information is not displayed, it may be input from the input unit 6.

  (S48) The text storage unit 16 stores the text corrected in S47. When the sentence is stored, the process proceeds to S8.

  (S8) The processing unit 11 processes the moving image data acquired via the moving image acquiring unit 10. When the information is selected in S41 and the moving image data is processed, the processing unit 11 acquires the information via the moving image acquisition unit 10 based on the information (first information 401 and the like) selected by the selection unit 21 as in the first embodiment. Processed video data. When a sentence is selected in S41 and the selected sentence is further corrected, the moving image data is processed based on the sentence corrected in S47. That is, a scene corresponding to information included in the corrected sentence is acquired from the moving image data. When a sentence is selected in S41 and the selected sentence is not corrected, the moving image data is processed based on the sentence selected in S45. That is, the scene corresponding to the information included in the selected sentence is acquired from the moving image data.

  As described above, according to the present embodiment, the user can process moving image data without having to select information many times by selecting a sentence stored in the sentence storage unit 16. In addition, by storing the created sentence, the user can easily use the sentence once used again.

  In this embodiment, the moving image processing apparatus 1 automatically determines a subject, a predicate, a modifier, and an object, automatically combines a proper particle or auxiliary verb, and transforms the ending of the object or modifier. Created but not limited to it. It may be performed manually by the user. For example, the user selects information that becomes a subject, predicate, object, or modifier. A particle or auxiliary verb that connects a subject, predicate, object, or modifier is input by an input unit (not shown). Alternatively, a particle or an auxiliary verb is displayed on the display unit 20, and the user selects an appropriate one from the displayed particles or auxiliary verbs. When a sentence is automatically created, if it is determined that there is no subject or verb and the sentence creation unit 15 cannot create the sentence, the user may be notified of this.

  In the present embodiment, all the sentences created by the sentence creating unit 15 are stored in the sentence storage unit 16, but the present invention is not limited to this. For example, the control unit 12 causes the display unit 20 to display the text created by the text creation unit 15. Only when the user gives an instruction to store the displayed text in the text storage unit 16, the control unit 12 may store the created text in the text storage unit 16.

  In the present embodiment, the text stored in the text storage unit 16 is displayed on the display unit 20. As the display order, the order of frequency of use, the order of created date, the order of long sentences, and the like can be considered.

  In this embodiment, although the case where there was only one sentence was demonstrated, it is not limited to it. For example, it is conceivable to acquire two scenes from moving image data by connecting two sentences using a conjunction. This will be specifically described with reference to FIG. First, as shown in FIG. 17a, it is assumed that the user selects a sentence “A-kun runs at an athletic meet”. When the user wants to process moving image data with a plurality of sentences, the conjunction button 7 is pressed. When the conjunction button 7 is pressed, a plurality of conjunctions are displayed on the display unit 20, and the user selects the intended conjunction "and" (FIG. 17b). When the user selects a conjunction, the second sentence candidate is displayed on the display unit 20 (FIG. 17c). From the displayed second sentence candidates, the user selects the intended second sentence “B-kun eats at the athletic meet.” Then, a sentence “A-kun runs at an athletic meet and B-kun eats at an athletic meet” can be made (FIG. 17d). When the user gives an instruction to process the video data with this sentence, the video processing device 1 acquires from the video data the scene “A you run at the athletic meet” and the scene “B you eat at the athletic meet” from the video data. They are connected and displayed on the display unit 20. Depending on the conjunction used at this time, an effect may be added between the two scenes. For example, since the conjunction “and” is sequential, the moving image processing apparatus 1 determines that the two scenes are similar scenes, and has the effect of giving a connection between the two scenes such as a fade. It is inserted between two scenes and displayed on the display unit 20. In the case of a conjunctive conjunction such as “by the way”, an effect reminiscent of a scene change such as a page turning effect may be inserted between the two scenes. The correspondence between conjunctions and effects may be set in advance or specified by the user.

  A display example as shown in FIG. 18 may be considered as the display unit 20 of the present invention. The moving image area 204 is divided into a plurality of areas. In FIG. 18, the area is divided into an area 8, an area 9, and an area 10. These divided areas display scenes corresponding to the information displayed in the first area 201, the second area 202, and the third area 203 below each area. For example, a scene “A-kun runs at an athletic meet” is displayed in the area 8. In the area 9, a scene “B-kun runs at an athletic meet” is displayed, and an area 10 displays a scene “C-kun runs at an athletic meet”. The information displayed in the first information 201, the second information 202, or the third information 203 may be changed as the information displayed by the user sliding the display unit 20.

  In the present invention, a moving image file including moving image data, audio data, a signal for synchronizing moving image data and audio data, and the like is acquired via the moving image acquisition unit 10, but the present invention is not limited thereto. For example, moving image data may be acquired. In that case, the moving image processing apparatus 1 processes moving image data.

  As mentioned above, although embodiment of this invention has been explained in full detail using drawing and a flowchart, a concrete structure is not restricted to this embodiment, The design etc. of the range which does not deviate from the summary of this invention are included. .

  Each unit in the embodiment of the present invention may be realized by dedicated hardware, or may be realized by a memory and a microprocessor.

  The control unit 12 according to the embodiment of the present invention is configured by a memory and a CPU (central processing unit), and the function is realized by loading a program for realizing the function of the control unit 12 into the memory and executing the program. It may be allowed.

  Further, a program for realizing the function of the control unit 12 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed, whereby the processing by the control unit 12 is performed. May be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

Claims (17)

When the first information is selected from among a plurality of pieces of information regarding the video data, a control unit that causes the display unit to display first related information related to the first information;
And a processing unit that processes the moving image data based on at least the first information. The control unit displays the first related information on the display unit when the first information is selected from the plurality of pieces of information acquired from the moving image data. Video processing device. The moving image processing according to claim 1, wherein the processing unit processes the moving image data based on the first information and the second information selected from the first related information. apparatus. When the second information is selected, the control unit causes the display unit to display a moving image based on the second related information related to the second information or the moving image data processed by the processing unit. The moving image processing apparatus according to claim 3. When the first information is selected, the control unit highlights the first information to be displayed on the display unit, and when the second information is selected, the control unit highlights the second information on the display unit. The moving image processing apparatus according to claim 3, wherein the moving image processing apparatus is displayed. 6. The control unit according to claim 1, wherein when the moving image data is processed by the processing unit, the control unit displays a moving image based on the moving image data processed by the processing unit on the display unit. The moving image processing apparatus described. The control unit causes the display unit to display information classified into the first category among the plurality of information, and when the first information is selected from the information classified into the first category, The moving image processing apparatus according to claim 1, wherein the first related information classified into a second category is displayed on the display unit. The information classified into the first category is a noun indicating a subject,
The video processing apparatus according to claim 7, wherein the control unit displays a noun indicating the subject on the display unit. The control unit includes a noun indicating the subject classified into the first category among the plurality of information, and a verb or adjective indicating the state of the subject classified into the second category among the plurality of information, or The moving image processing apparatus according to claim 8, wherein an adjective verb is displayed on the display unit. When the first information is selected from the plurality of information acquired from the moving image data or the plurality of information acquired from a frame related to the moving image data, the control unit displays the first related information. The moving image processing apparatus according to claim 1, wherein the moving image processing apparatus is displayed on a screen. The moving image processing apparatus according to claim 3, wherein the processing unit acquires a frame in which the first information and the second information are acquired from the moving image data. The plurality of information is associated with time information corresponding to a frame from which the plurality of information is acquired,
The processing unit acquires a frame corresponding to the time information associated with the first information and the time information associated with the second information from the moving image data. Item 12. A moving image processing apparatus according to Item 11. A sentence creation unit for creating a sentence based on at least the first information;
The moving image processing according to claim 1, wherein when the sentence created by the sentence creating unit is selected, the processing unit processes the moving image data based on the selected sentence. apparatus. A sentence storage unit for storing the sentence created by the sentence creation unit;
The control unit causes the display unit to display the sentence stored in the sentence storage unit,
The moving image processing according to claim 13, wherein the processing unit processes the moving image data based on the selected sentence when at least one of the sentences displayed on the display unit is selected. apparatus. The control unit displays at least one conjunction on the display unit,
The said process part processes the said moving image data based on the selected said conjunction, when at least 1 is selected from the said conjunctions displayed on the said display part. Video processing device. A control method for displaying first related information related to the first information on a display unit when the first information is selected from the plurality of pieces of information regarding the video data;
A moving image processing method comprising: processing the moving image data based on at least the first information. When the first information is selected from among a plurality of pieces of information related to the moving image data, a control step of causing the display unit to display first related information related to the first information;
And a processing step for processing the moving image data based on at least the first information.