JP2006166407A - Imaging device and its control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device and its control method, wherein a speech in a moving image can be recognized and a frame image suitable for the speech can be selected for display and output. <P>SOLUTION: The imaging device comprises a speech recognizer 105 for recognizing a speech included in the moving image, and an image selector 107 for selecting the frame image of a print candidate based on the end of the speech recognized by the speech recognizer 105. A frame image 108 selected by the image selector 107 and a text image 110 showing the speech recognized by the speech recognizer 105 are composed by an image composition part 111 for output. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus capable of reproducing a moving image and a control method thereof.

  Digital cameras capable of shooting and playing back moving images are commercially available, and by using such moving image shooting / playback functions, it is possible to record and play back moving images similar to video cameras.

  In addition, the digital camera is provided with a function capable of printing the captured image by transferring it to a printer or the like. In the case of a normal still image, a desired image can be printed by selecting the desired image and outputting it to a printer. On the other hand, in the case of a moving image, the images change continuously, and it is not easy to determine which frame image is optimal for printing.

In Patent Document 1, a discontinuous point (cut point) of a moving image is detected, a silent part of the sound is detected as a sound break, and a part that is a sound break and is a cut point of the moving image is detected as a whole break. It is described that a representative frame is selected and displayed in a list for each of the parts thus separated. Further, in Japanese Patent Laid-Open No. 2004-228688, when audio information is input together with image information, when the printing of the image is instructed, the audio portion is recognized as speech, and the recognized result is converted into a character code to be printed. It is described to expand to.
JP-A-9-214879 JP 2000-301806 A

However, according to the above-described conventional method, when a moving image is printed, which frame image is selected and printed depends on the operation of the user, and the frame image thus selected and voice recognition are performed. The character code does not always match, and it is not easy for the user to select a frame image in a moving image desired by the user.

  An object of the present invention is to solve the above-mentioned problems of the prior art.

  A feature of the present invention is to provide an imaging apparatus capable of recognizing sound in a moving image, selecting a frame image suitable for the sound, and displaying / outputting it, and a control method thereof.

  The present invention also provides an imaging apparatus capable of selecting a frame image using a voice input by a user as a search keyword, synthesizing a voice image corresponding to the selected frame image, and printing the image, and a control method thereof. It is in.

An imaging device according to one embodiment of the present invention includes the following configuration. That is,
An imaging device capable of playing a video,
A recognition means for recognizing the audio contained in the video,
A selection unit that selects a frame image of a print candidate based on a voice break recognized by the recognition unit;
Output control means for combining and outputting the frame image selected by the selection means and text information indicating the voice recognized by the recognition means.

An imaging device according to one embodiment of the present invention includes the following configuration. That is,
An imaging device capable of playing a video,
Speech recognition means for recognizing input speech information;
A recognition means for recognizing the audio contained in the video,
Comparing means for comparing the voice information voice-recognized by the voice recognition means with the voice information voice-recognized by the recognition means;
Extraction means for extracting frame images determined to be coincident by the comparison means;
Output control means for combining and outputting the frame image extracted by the extraction means and text information indicating the audio information.

An imaging device according to one embodiment of the present invention includes the following configuration. That is,
An imaging device capable of playing a video,
A recognition means for recognizing the audio contained in the video,
A selection unit that selects a frame image of a print candidate based on a voice break recognized by the recognition unit;
It is determined whether or not to divide text information indicating the speech recognized by the recognition unit, and in the case of division, an image selection unit that selects a frame image corresponding to the speech corresponding to the divided text;
When the text information is not divided, the frame image selected by the selection unit and the text information indicating the speech recognized by the recognition unit are combined and output,
In the case of dividing the text information, there is provided output control means for synthesizing and outputting the frame image selected by the image selection means and the text information corresponding to the divided text.

An imaging device control method according to an aspect of the present invention includes the following steps. That is,
A method for controlling an imaging apparatus capable of reproducing a movie,
A recognition process for recognizing the audio contained in the video,
A selection step of selecting a frame image of a print candidate based on the voice break recognized in the recognition step;
And an output control step of synthesizing and outputting the frame image selected in the selection step and text information indicating the speech recognized in the recognition step.

An imaging device control method according to an aspect of the present invention includes the following steps. That is,
A method for controlling an imaging apparatus capable of reproducing a movie,
A speech recognition process for recognizing input speech information;
A recognition process for recognizing the audio contained in the video,
A comparison step for comparing the voice information recognized in the voice recognition step with the voice information recognized in the recognition step;
An extraction step of extracting frame images determined to match in the comparison step;
And an output control step of synthesizing and outputting the frame image extracted in the extraction step and text information indicating the audio information.

An imaging device control method according to an aspect of the present invention includes the following steps. That is,
A method for controlling an imaging apparatus capable of reproducing a movie,
A recognition process for recognizing the audio contained in the video,
A selection step of selecting a frame image of a print candidate based on the voice break recognized in the recognition step;
It is determined whether or not to divide the text information indicating the speech recognized in the recognition step, and when dividing, an image selection step of selecting a frame image corresponding to the speech corresponding to the divided text;
If the text information is not divided, the frame image selected in the selection step and the text information indicating the voice recognized in the recognition step are combined and output,
An output control step of combining and outputting the frame image selected in the image selection step and the text information corresponding to the divided text when dividing the text information;
It is characterized by having.

  According to the present invention, it is possible to recognize sound in a moving image, select a frame image suitable for the sound, and display / output it.

  In addition, according to the present invention, there is an effect that a frame image can be selected using the voice input by the user as a search keyword, and the voice image corresponding to the selected frame image can be synthesized to print the image.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims, and all the combinations of features described in the embodiments are not necessarily essential to the solution means of the invention. .

  FIG. 1 is a functional block diagram showing a functional configuration of the imaging apparatus according to the embodiment of the present invention. This imaging apparatus will be described in the case of a digital camera, a digital video camera, or the like in this embodiment.

  In the figure, a moving image file 101 is stored in a storage medium such as a memory card or a tape. The moving image file demultiplexer 102 separates an audio signal and an image signal from, for example, an MPEG file stored as a moving image file. The image decoder 103 inputs and decodes the image signal separated by the demultiplexer 102. The audio decoder 104 decodes the audio signal separated by the demultiplexer 102 and outputs it to the audio recognition unit 105. The voice recognition unit 105 recognizes the decoded voice signal. The result of the speech recognition is sent to the sentence / sentence recognition unit 106, and a document / sentence break is detected. Based on the sentence and phrase breaks thus detected, the image selection unit 107 selects a frame image corresponding to the sentence and phrase breaks from among a plurality of frame images decoded by the image decoder 103. The selection criteria for selecting a frame image by the image selection unit 107 will be described in detail later. The frame image selected in this way becomes the selected image 108.

  On the other hand, the sentence extracted by the sentence and phrase recognition unit 106 is converted into text data by the text conversion unit 109. The text data thus converted is subjected to pattern development based on the character font corresponding to the character code to become a text image 110. The text image 110 and the selected image 108 are combined by the image combining unit 111. The synthesized image is displayed or transmitted to the printer 112 for printing. The microphone 113 is used to input an acoustic signal such as voice.

  FIG. 2 is a diagram illustrating a print example of the image printed by the printer 112 in this way.

  Here, the text image 110 is combined with the selected selection image 108 as shown by the balloon 200 and printed.

  FIG. 3 is a block diagram illustrating a configuration of the imaging apparatus according to the present embodiment.

  The control unit 301 controls the overall operation of the imaging apparatus, and controls the CPU 330, the ROM 331 storing a control program executed by the CPU 330, and various data when the CPU 330 executes control processing. A RAM 332 for temporarily storing the data is provided. The control target in the control unit 301 is to capture an image in the imaging unit 303 based on a trigger such as a shutter from the input unit 302, store the image data, and transmit the captured image data to the outside via the wired communication unit 304. In other words, it includes processing such as displaying an image captured by the imaging unit 303 and stored in the storage medium 313, and displaying an image being captured by the imaging unit 303 on the display unit 306.

  The imaging unit 303 includes an imaging device (CCD), an image buffer memory, and the like. The captured image is stored in a temporary buffer memory, and then stored in a storage medium 313 such as a memory card under the control of the control unit 301. Alternatively, it can be transmitted to the outside via a wired or wireless communication unit 304, 308. The input unit 302 includes switches and buttons operated by the user, and is used to perform operations such as operations using a UI, power supply, shutter, and camera zoom. A battery 309 supplies power to the entire imaging apparatus. In some cases, power may be supplied to the outside via the connector 305, or power may be supplied from the outside. The power supply monitoring unit 310 measures the battery capacity of the battery 309 and notifies the control unit 301 of it.

  The wired communication unit 304 serves as a controller for the connector 305, monitors the connection status of the connector 305, and notifies the control unit 301 of the monitoring result. The connector 305 includes a plurality of connection terminals, and transmits and receives image data to and from other terminals (for example, a PC and a printer). The connector 305 includes a power line and the like. The wireless communication unit 308 has a function related to wireless communication, receives a signal received from the control unit 301, modulates the signal into an RF signal, and transmits the signal. The received RF signal is demodulated and passed to the control unit 301. The voice processing unit 311 processes the voice signal input from the microphone 113, recognizes the voice, and outputs it. The microphone 113 is also used for inputting an acoustic signal during video recording. The storage / reproduction control unit 312 controls writing of data to the storage medium 313 and reading of data from the storage medium 313. The reproduced audio signal is sent to the audio processing unit 311 to be recognized.

[Embodiment 1]
FIG. 4 is a flowchart for explaining processing by the imaging apparatus according to Embodiment 1 of the present invention. A program for executing this processing is stored in the ROM 331 and executed under the control of the CPU 330.

  First, in step S1, a selected time stamp is determined. This sets a selection criterion for deciding which image to select from images to be selected.

FIG. 5 is a diagram showing an example of a menu screen displayed on the display unit 306 when this selection time stamp is selected.
(1) The middle image of the images in the selection target period is selected.

When a sentence or phrase break is detected based on speech recognition, it is specified to select a frame image that is approximately in the center of the image period corresponding to the sentence or phrase.
(2) Select an image after a certain time from the beginning of the target period.

When a sentence or phrase break is detected based on speech recognition, it is specified to select an image after a certain time from the beginning of the image period corresponding to the sentence or phrase.
(3) Select the first sounded image of the target period.

When a sentence or phrase break is detected based on speech recognition, it is designated to select the first sounded image in the image period corresponding to the sentence or phrase.
(4) Select an image with the smallest motion vector.

It is specified that an image with the smallest motion vector is selected within an image period corresponding to the sentence or phrase when a sentence or phrase break is detected based on speech recognition. In this case, for example, only an intra frame may be selected.
(5) Select an optimal image from the shooting parameters.

  When a sentence or phrase break is detected based on speech recognition, it is specified that an image with appropriate shooting parameters is selected within an image period corresponding to the sentence or phrase. Whether or not the shooting parameter is appropriate is determined by, for example, analyzing the luminance distribution in the image, whether the image is well exposed, whether the white balance is appropriate, or whether the focusing is optimal (image It is conceivable to determine whether or not the focus is out of focus.

  Any of these five items can be selected using the input unit 302.

  Thus, in step S1, the selected selection time stamp is stored in the RAM 332. Next, in step S3, a moving image is input, and in step S4, the voice included in the input moving image is recognized. Next, in step S5, based on the speech recognition, the phrase / sentence break is recognized, and the recognized phrase / sentence is converted into text. In step S7, the frame image closest to the selected time stamp selected in step S1 and stored in step S2 is selected. In step S8, the character image converted to text in step S6 and the frame image selected in step S7 are synthesized. In step S9, the combined image is stored in the RAM 332. In step S10, it is checked whether or not it is the final image of the moving image. When not changing, it progresses to step S3 and the next moving image is input, but when changing the time stamp for selection, it returns to step S1 and performs the above-mentioned process.

  In step S10, if it is the end of the moving image, the process proceeds to step S12, and the image synthesized in step S8 is displayed on the display unit 306. Here, if there are a plurality of composite images, a plurality of images are displayed on the display unit 306. In step S13, an image of the frame selected by the user is output to the printer 112 to execute printing.

  As described above, according to the first embodiment, it is possible to select or print or display an optimal image from the moving image reproduction period.

  Furthermore, the selected image can be output and printed by synthesizing the content of the sound related to the image as a balloon with text data as shown in FIG.

[Embodiment 2]
Next, a second embodiment of the present invention will be described. In the second embodiment, words included in the moving image are input from the microphone 113 as selection items for selecting a desired image. Then, a voice that matches the input voice is emitted and the image at the time is determined as the selected image.

  FIG. 6 is a flowchart for explaining processing in the imaging apparatus according to the second embodiment. A program for executing this processing is stored in the ROM 331 of the control unit 301 and is executed under the control of the CPU 320.

  First, reproduction of a moving image is started in step S21, and the user's voice is input from the microphone 113 in step S22. When the voice is input, the input voice is recognized next in step S23, and the voice information of the moving image recognized in step S24 is compared with the voice information in step S25. As a result of this comparison, when there is no matching audio, the process returns to step S21 and the same processing is repeated for the next moving image.

  If a matching voice is found in step S25, the process proceeds to step S26, and the frame image at that time is selected. When the selected image is determined in this way, the voice is converted into text, and the text data and the selected frame image are synthesized in step S27. Thereby, for example, an image in which characters are added to the image by a balloon as shown in FIG. 2 is generated. In step S28, the composite image is displayed on the display unit 306, and the user is inquired about whether or not it is possible. If the user inputs OK, the process proceeds to step S29, and the composite image is determined as a print or output image. If OK is not input, the process returns to step S21 and the same operation is performed again.

[Modification of Embodiment 2]
FIG. 7 is a flowchart for explaining a modification of the processing in the imaging apparatus according to the second embodiment. A program for executing this processing is stored in the ROM 331 of the control unit 301 and is executed under the control of the CPU 320. The In this modification, the selection of a frame image for photography is the same as that of the second embodiment described above in that the voice input by the user is used as a search keyword. In the first embodiment, the selected time stamp is used, and also in the reproduction of the moving image, the entire moving image is reproduced and the frame image is selected, or the image period including the image in the composition extracted in step S12 of the first embodiment ( The point of selecting a desired frame image by reproducing a moving image for each phrase unit (corresponding to a phrase unit and a sentence unit) is different from the configuration of the second embodiment.

  First, in step S31, a moving image to be reproduced and a selection time stamp are selected. This selection time stamp is the same as that in the first embodiment. Here, when selecting a moving image, it is specified whether to select all the moving images to be reproduced or to reproduce the image period including the frame image extracted in step S12 described above. be able to. When the moving image to be extracted is selected in this way, the process proceeds to step S32, and voice input by the user is performed. When inputting the voice, the user sets the digital camera in a predetermined mode, and then inputs the voice through the microphone 112 while pressing a specific button (for example, a function key). The user's voice input in this way is recognized as a voice in step S33 and the result is stored in the RAM 332. Next, in step S34, the reproduction of the moving image designated in step S1 is started. In step S35, the voice information included in the moving image is recognized as in step S4 in FIG. Next, in step S35, the voice information that has been voice-recognized and stored in step S33 matches the voice-recognized result in step S35, and the voice is based on the selected time stamp specified in step S31. It is determined whether or not a frame image whose information matches can be selected. If a frame image that meets these conditions can be selected, the process proceeds to step S37. If not, the process returns to step S34 to execute the above-described processing.

  If a frame image can be selected in step S36, the voice information is converted into text in step S37, and the text data is developed into a character image using a character font. In step S38, a character image is combined with the frame image, and the combined image is displayed on the display unit 306. Next, in step S39, it is determined by the user whether or not the composite image is satisfactory by the user. If “OK” is input as satisfactory, the process proceeds to step S40, and the composite image is output to the printer 112. Print. On the other hand, if the user is not satisfied, the process proceeds to step S31, where the moving image is designated again and the selected time stamp is designated.

  In step S38, a plurality of candidate frame images close to the designated selection time stamp may be extracted and displayed, and the user may select a desired frame image from among the plurality of candidate frame images.

[Embodiment 3]
Next, a third embodiment of the present invention will be described. In the third embodiment, a sentence or a phrase is divided as means for selecting and printing an optimal combination of sound and image. As a result, the text can be printed or displayed together with the image as a text or phrase having an appropriate number of characters.

  FIG. 8 is a diagram for explaining an example of the relationship between a moving image frame and audio.

  Reference numerals 801 to 806 denote a series of image frames constituting a moving image. Reference numerals 807 and 808 denote sounds synchronized with the series of image frames (801 to 806). In the case of the above-described embodiment, these series of voices are recognized as sentences or phrases as shown by 807 and 808. Furthermore, image frames 809 and 810 are selected from the respective sections based on the designated selection time stamp.

  In FIG. 9, an image frame 801 is selected as a frame corresponding to the voice “watching”, and an image with a balloon 901 including text indicating the voice “watching!” Is printed and output on the image. Yes. In this way, the sound is converted into text, and an image in which the speech balloon indicating the sound is combined with the selected image frame is printed.

  In particular, the third embodiment will be described in the case where the text portion of the print image shown in FIG. 10 is subdivided.

  FIG. 11 is a diagram for explaining an example of dividing the text representing the voice 808 in the third embodiment, and the same parts as those in FIGS. 8 and 9 are denoted by the same symbols.

  Reference numerals 910 and 911 denote texts obtained by dividing the text representing the voice 808 into two parts. Such text division may be performed according to an instruction from an operator or may be performed automatically.

  12 and 13 are flowcharts for explaining processing by the imaging apparatus according to Embodiment 3 of the present invention.

  In FIG. 12, first, a moving image is input in step S <b> 41, and in step S <b> 42, the sound recorded with the moving image is recognized by the sound processing unit 311. Next, in step S43, the recognized voice is expressed as text (sentence / sentence), and in step S44, the recognition result is converted into text data. Next, in step S45, based on a preset time stamp selection method, a predetermined image frame is selected from the moving image corresponding to the recognized sound. In step S46, the text image converted to text in step S44 and the image of the image frame selected in step S45 are combined. In step S47, the image synthesized in step S46 and the text / sentence data converted into text are stored. In step S48, it is determined whether or not it is the last frame of the moving image.

  If it is determined in step S48 that the frame is the last frame of the moving image, the process proceeds to step S49, the composite image created and stored in step S47 is read, and the list is displayed on the display unit 306. Next, in step S50, the operator is inquired whether each composite image can be left as it is, or whether to edit the composite image. If this is acceptable, the process proceeds to step S51, and the selected composite image is printed. On the other hand, if it is selected in step S50 to edit the composite image, the process proceeds to step S61 (FIG. 13).

  FIG. 13 is a flowchart for explaining the editing process. First, in step S61, using the input unit 302, an image to be edited is selected from the list of composite images displayed on the display unit 306. Next, in step S62, one of the selected composite images is displayed, and the process proceeds to step S63. In step S 63, whether to end editing (OK), to divide a sentence / sentence that is text, or to change the time stamp selection method is selected by an instruction from the input unit 302. If it is instructed to finish editing, the process proceeds to step S49 (FIG. 12), and a list of composite images is displayed.

  On the other hand, when the subdivision of the sentence / phrase is performed, the process proceeds to step S64, and the sentence / phrase corresponding to the text of the selected composite image is divided. Next, in step S65, an image frame that is a basis of the composite image is selected again from the divided moving images in the target section based on the set time stamp selection method. Next, in step S66, the text divided in step S64 and the image frame reselected in step S65 are synthesized. In step S67, the synthesized image and the divided text are stored. In step S62, the divided text and the recombined image are displayed.

  If the change of the time stamp selection method is selected in step S63, the process proceeds to step S68 to display a time stamp selection method selection screen. Next, when the selection method is changed in step S69, it is stored. Next, in step S70, an image frame to be synthesized is selected from the frame of the moving image section that is the target of the audio based on the newly set time stamp selection method. In step S71, the selected frame is re-synthesized with the text indicating the voice. At this time, the text is the same text as before the image frame is selected again. In step S72, the composite image and text are stored. In step S 62, the recombined image is displayed on the display unit 306.

  FIG. 14 is a diagram showing an example of an image frame divided and selected by a series of processes (FIGS. 12 and 13) according to the third embodiment, and the same parts as those in FIGS. Is shown.

  Here, reference numerals 910 and 911 denote the results of subdivision of sentences and phrases ("Dad, Dad got on") indicating voice. Here, the text representing the voice 808 is divided into two texts 910 and 911. Further, image frames 804 and 805 are selected as frames corresponding to the texts 910 and 911 from the series of image frames 803 and 804 and 805 and 806 corresponding to the divided texts 910 and 911, respectively. .

  15 and 16 are diagrams showing examples of images printed by combining the divided texts 910 and 911 corresponding to the image frames 804 and 805, respectively. In this image data, voice 808 is displayed by balloons 1501 and 1502 including text corresponding to each voice.

  In the above, an example in which the operator determines the subdivision of the sentence / phrase in the editing flow has been described. However, the number of characters in the sentence / phrase per image frame, the area on the layout of the character image on the text composite image, and the like. Based on the size or the like, the subdivision of the sentence / sentence, the reselection of the image frame in the corresponding section, and the image recombination processing may be automatically performed until the composite image is optimized.

  In this case, it is preferable to change the size of the segment to be divided according to the size of the area where the character can be laid out. For this purpose, it is only necessary to recognize a main subject and set a balloon area for laying out characters according to the size of the other area. The segment division position can be set according to the number of characters to be laid out in the balloon. This makes it easier to recognize the characters in the balloon and create a composite image that includes easy-to-read text.

  FIG. 17 is a diagram showing an example of a setting screen displayed on the display unit 306 when selecting whether to automatically divide a sentence / sentence according to the present embodiment.

  The figure shows a state in which the automatic division mode is set (“ON” is selected).

  Although the embodiments of the present invention have been described in detail above, the present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device.

  In the present invention, a software program that realizes the functions of the above-described embodiments is supplied directly or remotely to a system or apparatus, and the computer of the system or apparatus reads and executes the supplied program code. In some cases, it can be achieved by In that case, as long as it has the function of a program, the form does not need to be a program. Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. That is, the present invention includes a computer program itself for realizing the functional processing of the present invention. In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  As a storage medium for supplying the program, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card ROM, DVD (DVD-ROM, DVD-R) and the like. As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a storage medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different home page. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the claims of the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who satisfy predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program may be part of the actual processing or The functions of the above-described embodiment can also be realized by performing all the processing and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a functional block diagram which shows the function structure of the imaging device which concerns on embodiment of this invention. It is a figure which shows the printing example which printed the synthesis | combination from the imaging device which concerns on this Embodiment with the printer. It is a block diagram which shows the structure of the imaging device which concerns on this Embodiment. It is a flowchart explaining the process by the imaging device which concerns on Embodiment 1 of this invention. It is a figure which shows the example of a menu screen displayed on a display part when selecting the selection time stamp which concerns on this Embodiment 1. FIG. 10 is a flowchart for describing processing in the imaging apparatus according to the second embodiment. 12 is a flowchart for describing a modification of processing in the imaging apparatus according to the second embodiment. It is a figure explaining an example of the relationship between a moving image frame and an audio | voice. It is a figure which shows the example of a print which synthesize | combined and printed the text and image frame which show a sound. It is a figure which shows the other example of a print which synthesize | combined and printed the text which shows an audio | voice, and an image frame. In Embodiment 3, it is a figure explaining the example which divides | segments the text showing an audio | voice. , It is a flowchart explaining the process by the imaging device which concerns on Embodiment 3 of this invention. It is a figure which shows an example of the image frame divided | segmented and selected by the series of processes (FIG. 12, FIG. 13) concerning this Embodiment 3. FIG. , In Embodiment 3, it is a figure which shows the example of an image which synthesize | combined and printed each divided text corresponding to each image frame. It is a figure which shows the example of a setting screen displayed on a display part, when selecting whether the division | segmentation of the sentence / sentence concerning this Embodiment is performed automatically.

Claims (42)

An imaging device capable of playing a video,
A recognition means for recognizing the audio contained in the video,
A selection unit that selects a frame image of a print candidate based on a voice break recognized by the recognition unit;
Output control means for combining and outputting the frame image selected by the selection means and text information indicating the speech recognized by the recognition means;
An imaging device comprising:   2. The imaging apparatus according to claim 1, wherein the selection unit selects an image at a substantially center of a target image within a period corresponding to a length of a phrase up to the voice break.   The imaging apparatus according to claim 1, wherein the selection unit selects a frame image at the time of the first sound of a target image within a period corresponding to a length of a phrase up to the voice break.   The imaging apparatus according to claim 1, wherein the selection unit selects a frame image after a predetermined time from the beginning of a target image within a period corresponding to a length of a phrase up to the speech break.   The said selection means selects the frame image with the magnitude | size of a motion vector the minimum among the target images in the period corresponding to the length of the phrase to the said audio | voice division | segmentation, The Claim 1 characterized by the above-mentioned. Imaging device.   The imaging apparatus according to claim 1, wherein the selection unit includes a unit that acquires a shooting parameter of a frame image, and selects the frame image of the print candidate based on the shooting parameter. An imaging device capable of playing a video,
Speech recognition means for recognizing input speech information;
A recognition means for recognizing the audio contained in the video,
Comparing means for comparing the voice information voice-recognized by the voice recognition means with the voice information voice-recognized by the recognition means;
Extraction means for extracting frame images determined to be coincident by the comparison means;
Output control means for synthesizing and outputting the frame image extracted by the extraction means and text information indicating the audio information;
An imaging device comprising:   The imaging apparatus according to claim 7, wherein the extraction unit extracts a frame image satisfying a predetermined condition from a plurality of frame images determined to match by the comparison unit.   9. The imaging apparatus according to claim 8, wherein the frame image satisfying the predetermined condition is a frame image at a substantially center of the target image within a period corresponding to a length of a phrase up to the voice break. . 9. The frame image satisfying the predetermined condition is a frame image at the time of the first sound of the target image within a period corresponding to a length of a phrase up to the break of the sound. Imaging device.   9. The frame image satisfying the predetermined condition is a frame image after a predetermined time from the beginning of a target image within a period corresponding to a length of a phrase up to the voice segment. Imaging device.   The frame image satisfying the predetermined condition is a frame image having a minimum motion vector size among target images within a period corresponding to a length of a phrase up to the voice segment. Item 9. The imaging device according to Item 8. The imaging apparatus according to claim 8, wherein the frame image satisfying the predetermined condition is determined based on a shooting parameter of the frame image. An imaging device capable of playing a video,
A recognition means for recognizing the audio contained in the video,
A selection unit that selects a frame image of a print candidate based on a voice break recognized by the recognition unit;
It is determined whether or not to divide text information indicating the speech recognized by the recognition unit, and in the case of division, an image selection unit that selects a frame image corresponding to the speech corresponding to the divided text;
When the text information is not divided, the frame image selected by the selection unit and the text information indicating the speech recognized by the recognition unit are combined and output,
Output control means for combining and outputting the frame image selected by the image selection means and the text information corresponding to the divided text when dividing the text information;
An imaging device comprising:   15. The imaging apparatus according to claim 14, wherein the selection unit selects an image at a substantially center of the target image within a period corresponding to a length of a phrase up to the voice break.   15. The imaging apparatus according to claim 14, wherein the selection unit selects a frame image at the time of the first sound of a target image within a period corresponding to a length of a phrase up to the voice break.   15. The imaging apparatus according to claim 14, wherein the selection unit selects a frame image after a predetermined time from the beginning of a target image within a period corresponding to a length of a phrase up to the speech break.   The said selection means selects the frame image with the magnitude | size of a motion vector the minimum among the object images in the period corresponding to the length of the clause to the said audio | voice division | segmentation, The said image is characterized by the above-mentioned. Imaging device.   The imaging apparatus according to claim 14, wherein the selection unit includes a unit that acquires a shooting parameter of a frame image, and selects the frame image of the print candidate based on the shooting parameter.   The image selection means determines whether or not to divide the text information indicating the sound based on whether or not the number of characters of the clause until the voice break recognized by the recognition means is larger than a predetermined number of characters. The imaging apparatus according to claim 14.   15. The image selection unit determines whether or not to divide according to whether the characters of the phrase up to the voice break recognized by the recognition unit fit in the synthesized image. Imaging device. A method for controlling an imaging apparatus capable of reproducing a movie,
A recognition process for recognizing the audio contained in the video,
A selection step of selecting a frame image of a print candidate based on the voice break recognized in the recognition step;
An output control step of synthesizing and outputting the frame image selected in the selection step and text information indicating the voice recognized in the recognition step;
A method for controlling an imaging apparatus, comprising:   23. The method of controlling an imaging apparatus according to claim 22, wherein, in the selection step, an image at a substantially center of a target image within a period corresponding to a length of a phrase up to the voice break is selected.   23. The control of the imaging apparatus according to claim 22, wherein, in the selection step, a frame image at the time of the first sound of the target image within a period corresponding to a length of a phrase up to the speech break is selected. Method.   23. The control of an imaging apparatus according to claim 22, wherein, in the selection step, a frame image after a predetermined time from the beginning of a target image within a period corresponding to a length of a phrase up to the speech break is selected. Method.   23. The frame selection method according to claim 22, wherein in the selection step, a frame image having a minimum motion vector size is selected from target images within a period corresponding to a length of a phrase up to the speech segment. Control method of imaging apparatus.   23. The method of controlling an imaging apparatus according to claim 22, wherein in the selection step, shooting parameters of a frame image are acquired and the frame image of the print candidate is selected based on the shooting parameters. A method for controlling an imaging apparatus capable of reproducing a movie,
A speech recognition process for recognizing input speech information;
A recognition process for recognizing the audio contained in the video,
A comparison step for comparing the voice information recognized in the voice recognition step with the voice information recognized in the recognition step;
An extraction step of extracting frame images determined to match in the comparison step;
An output control step of synthesizing and outputting the frame image extracted in the extraction step and text information indicating the audio information;
A method for controlling an imaging apparatus, comprising:   30. The control of an imaging apparatus according to claim 28, wherein in the extraction step, a frame image satisfying a predetermined condition is extracted from a plurality of frame images determined to be coincident in the comparison step. Method.   30. The imaging apparatus according to claim 29, wherein the frame image satisfying the predetermined condition is a frame image at a substantially center of a target image within a period corresponding to a length of a phrase up to the voice break. Control method.   30. The frame image satisfying the predetermined condition is a frame image at the time of the first sound of the target image within a period corresponding to a length of a phrase up to the break of the sound. Method for controlling the imaging apparatus.   30. The frame image satisfying the predetermined condition is a frame image after a predetermined time from the beginning of a target image within a period corresponding to a length of a phrase up to the voice segment. Method for controlling the imaging apparatus.   The frame image satisfying the predetermined condition is a frame image having a minimum motion vector size among target images within a period corresponding to a length of a phrase up to the voice segment. Item 30. A method for controlling an imaging apparatus according to Item 29. 30. The method of controlling an imaging apparatus according to claim 29, wherein the frame image satisfying the predetermined condition is determined based on a shooting parameter of the frame image. A method for controlling an imaging apparatus capable of reproducing a movie,
A recognition process for recognizing the audio contained in the video,
A selection step of selecting a frame image of a print candidate based on the voice break recognized in the recognition step;
It is determined whether or not to divide the text information indicating the speech recognized in the recognition step, and when dividing, an image selection step of selecting a frame image corresponding to the speech corresponding to the divided text;
If the text information is not divided, the frame image selected in the selection step and the text information indicating the voice recognized in the recognition step are combined and output,
An output control step of combining and outputting the frame image selected in the image selection step and the text information corresponding to the divided text when dividing the text information;
A method for controlling an imaging apparatus, comprising:   36. The method of controlling an imaging apparatus according to claim 35, wherein the selecting step selects an image at a substantially center of a target image within a period corresponding to a length of a phrase up to the voice break.   36. The control of an imaging apparatus according to claim 35, wherein the selecting step selects a frame image at the time of the first sound of the target image within a period corresponding to a length of a phrase up to the voice break. Method.   36. The control of an imaging apparatus according to claim 35, wherein the selection step selects a frame image after a predetermined time from the beginning of a target image within a period corresponding to a length of a phrase up to the voice break. Method.   36. The method according to claim 35, wherein the selecting step selects a frame image having a minimum motion vector size among target images within a period corresponding to the length of a phrase up to the speech break. Control method of imaging apparatus.   36. The method of controlling an imaging apparatus according to claim 35, wherein the selecting step includes a step of acquiring a shooting parameter of a frame image, and the frame image of the print candidate is selected based on the shooting parameter.   In the image selection step, it is determined whether or not to divide the text information indicating the voice based on whether or not the number of characters of the clause until the voice break recognized in the recognition step is larger than a predetermined number of characters. 36. The method of controlling an imaging apparatus according to claim 35.   36. In the image selection step, it is determined whether or not to divide according to whether the characters of the phrase up to the voice break recognized in the recognition step fit in the combined image. Method for controlling the imaging apparatus.

Images