JP4904691B2 - Camera device and photographing method - Google Patents

Description

  The present invention relates to a camera device having an audio shutter function and a photographing method.

2. Description of the Related Art Conventionally, camera devices have a voice shutter function that performs a series of shooting operations including autofocusing and exposure triggered by recognition of the voice of a registered command word (recognition target word). Is described in Patent Document 1 below.
JP-A-11-194392

  However, in the above technique, for example, since the focusing operation and the exposure operation are performed after the recognition of the command word “high, cheese” is completed, the substantial shooting operation is performed after the user issues the command word. There was a problem that a slight time lag occurred before it was performed.

  The present invention has been made in view of such a conventional problem, and a camera device capable of almost eliminating a time lag between issuing a command word for shooting and performing an actual shooting operation. And an imaging method.

In order to solve the above-mentioned problem, the invention of claim 1 is a camera device having an automatic focus adjustment function for performing a series of photographing operations consisting of a plurality of steps triggered by voice input of a predetermined command word. Voice recognition means for recognizing input voice, command word storage means for storing the predetermined command word, a plurality of utterance stages set in the command words stored in the command word storage means, and the series Corresponding information storage means for storing correspondence information indicating a correspondence relationship with the operation at each stage in the shooting operation of the above, and whenever the speech recognition stage recognized by the voice recognition means reaches each utterance stage of the command word, Control means for sequentially starting the operation of each stage in the series of photographing operations corresponding to the utterance stage indicated by the stage information stored in the correspondence information storage means, The multi-stage operation in the photographing operation includes a focusing operation and an exposure operation by the automatic focus adjustment function, and the voice recognition means includes an acoustic model including a noise component generated during the automatic focus adjustment operation in the photographing operation and an acoustic that does not include it. and a model, and change in the acoustic model including the noise component in response to the recognition of the instruction word instructing automatic focusing, an acoustic model that does not include in the series of photographing operations, instructs the automatic focusing The voice in the next utterance stage after the utterance stage of the command word is recognized .

  In such a configuration, when a predetermined command word is inputted by voice, the operation of each stage in a series of photographing operations is started in order from the stage before the voice recognition of the command word is completed.

  Further, in the invention of claim 2, the operation at any stage in the series of photographing operations is composed of a plurality of operations performed continuously.

  In the invention of claim 4, a plurality of instruction words are stored in the instruction word storage means, and a plurality of instruction words stored in the instruction word storage means are stored in the correspondence information storage means. It is assumed that a plurality of correspondence information indicating a correspondence relationship between each utterance stage set for each and each stage operation in each of a plurality of series of photographing operations is stored.

  According to a fifth aspect of the present invention, there is provided a registration means for storing a word composed of speech recognized by the voice recognition means in the command word storage means as a new command word, and the command word storage by the registration means. Generating means for generating new correspondence information indicating a correspondence relationship between a plurality of utterance stages in the new command word stored in the means and the operations of each stage in the series of photographing operations, and storing the correspondence information in the correspondence information storage means And provided.

According to the invention of claim 5, there is provided a photographing method in a camera device having an automatic focus adjustment function for performing a series of photographing operations consisting of a plurality of steps with a voice input of a predetermined command as a trigger. Each time the recognized speech recognition stage reaches each utterance stage of the plurality of utterance stages set in the command word, the series of photographing associated with each utterance stage and a step of starting the operation of each step in the operating sequence, the comprises a focusing operation and the exposure operation by the automatic focusing function on the operation of the plurality of stages in series of photographing operations, sequentially recognizing process speech, the In response to the recognition of the instruction word instructing automatic focus adjustment in a series of shooting operations, the acoustic model including the noise component is changed from the acoustic model not including the noise component, and automatic focus adjustment is instructed The speech of the next utterance step utterance stages of the instruction word, and a method of recognizing.

  According to such a method, when a predetermined command word is inputted by voice, the operation of each stage in a series of photographing operations is started in order from the stage before the voice recognition of the command word is completed.

According to the invention of claim 6, the audio to be input to the computer of the camera device having the automatic focus adjustment function for performing a series of photographing operations consisting of a plurality of steps triggered by the audio input of a predetermined command word. Each time a speech recognition stage recognized by the speech recognition means reaches each utterance stage of a plurality of utterance stages set in the command word, and is associated with each utterance stage. And a process for causing each part of the apparatus to sequentially start the operation of each stage in the series of shooting operations, and the plurality of stages of the series of shooting operations include a focusing operation and an exposure operation by an automatic focus adjustment function. wherein, the sequential process of recognizing the speech recognition means speech includes the noise component in response to the recognition of the instruction word instructing an automatic focus adjustment in the series of photographing operations The sound model, and change from the acoustic model which does not include the audio of the next utterance step utterance stages of the instruction word instructing automatic focusing, and the recognized program.

  As described above, in the present invention, when a predetermined command word is inputted by voice, the operation of each stage in a series of shooting operations is started in order from the stage before the voice recognition of the command word is completed. . Therefore, it is possible to eliminate almost all the time lag between the time when the command word for shooting is issued and the time when the actual shooting operation is performed.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic electrical configuration of a digital camera according to the present invention.

  This digital camera has an audio shutter function, and includes a key input unit 1, a focusing unit 2, an exposure unit 3, an image input unit 4, an image compression unit 5, an image storage unit 6, an image display unit 7, a microphone. 8 includes a voice input unit 9, an A / D conversion unit (A / D) 10, a work memory 11, a voice recognition unit 12, and a program memory 13, except for the key input unit 1 and the microphone 8. ˜13 is a configuration driven and controlled by the control unit 14.

  The key input unit 1 includes various keys used by the user for operating the digital camera, such as a power key, a shutter key, a mode switching key for shooting / playback mode, and an operation key used for setting various functions. When any key is operated, it is detected by the control unit 14. The focusing unit 2 includes a focus motor that drives a focus lens in an optical system (not shown) to a position corresponding to the subject distance and a drive circuit thereof in the shooting mode.

  The exposure unit 3 includes an image pickup device such as a CCD for picking up a subject image formed by the optical system, a driving circuit for the image pickup device, and an analog image pickup signal output from the image pickup device. A digital imaging signal is output including a converter and the like. The image input unit 4 includes various signal processing circuits for performing various signal processing on the digitally converted imaging signal. The image compression unit 5 is composed of a circuit that compresses the image data processed by the image input unit 4 and decompresses the compressed image data. The image storage unit 6 includes various memory cards that store compressed image data. The image display unit 7 includes a liquid crystal display, a driving circuit thereof, and the like, and displays a subject image signal-processed by the image input unit 4 and a recorded image read from the image storage unit 6 on the liquid crystal display. Let

  The voice input unit 9 is composed of an amplifier that amplifies the voice input from the microphone 8 and other voice processing circuits, and outputs the processed voice signal. The A / D converter 10 converts an analog audio signal output from the audio input unit 9 into a digital signal. The work memory 11 is a RAM that sequentially stores converted audio signals (audio data), and stores various data that the control unit 14 generates and uses when controlling each unit as needed.

  The voice recognition unit 12 is equal to or higher than the data accumulation speed for input voices sequentially stored in the work memory 11 in the shooting standby state when the voice shutter function is turned on (when delayed). The acoustic model recorded in the program memory 13 is used at a speed that means that the feature is extracted and the Viterbi algorithm is used for recognition processing. The recognition result is sequentially sent to the control unit 14.

  The control unit 14 is mainly composed of peripheral circuits including a CPU and an input / output interface. The program memory 13 is a non-volatile memory such as an EEPROM, and data such as the above-described acoustic model used by the speech recognition unit 12 for speech recognition, and various programs for causing the control unit 14 to control each unit, for example, Control programs such as AE (automatic exposure) and AF (automatic focus adjustment), and a program for causing the control unit 14 to function as the control means, registration means, and generation means of the present invention are stored.

  The program memory 13 is an instruction word storage means and correspondence information storage means of the present invention, and the program memory 13 stores the above-described programs and data together with data constituting the registration table T shown in FIG. ing. The registration table T includes correspondence information 102 indicating a relationship between a plurality of types of reserved words (speech patterns) 101, a plurality of utterance stages set in each reserved word 101, and operations at each stage in a series of photographing operations. Consists of The reserved word 101 is a command word that can be used at the time of photographing using the voice shutter function, and the reserved word 101 has a plurality of utterance stages corresponding to the contents.

  The number of utterance stages is three, except that the reserved word 101 “high, cheese” is two stages, up to “high” (first utterance stage) and “... cheese” (second utterance stage). It is. In this embodiment, there are three types of operations that can be set corresponding to each utterance stage: automatic focus adjustment, exposure, and recording, and the number of operations is the number of reserved words 101 “HIGH, TRIMAS, MOUICHAI”. The three operations of recording, exposure, and recording corresponding to the three utterance stages ("... In addition, among the acoustic models described above, noise (optical system and motor drive sound) generated by automatic focus adjustment is superimposed on the acoustic models of the “cheese” and “trimus” portions included in the plurality of reserved words 101. HMM (Hidden Markov Model, Hidden Markov Model) trained by PCM is prepared.

  Next, the operation according to the present invention of the digital camera having the above configuration will be described. FIG. 3 is a flowchart showing an operation in the sound shutter mode when the sound shutter function is set to ON.

  In the voice shutter mode, for example, voice input processing from the microphone 8 is started in response to the user pressing the shutter key (step SA1). Then, the input new syllable part is sequentially recognized by the voice recognition unit 12, and the voice pattern being recognized is successively accumulated (step SA2). The speech recognition method in the speech recognition unit 12 is as follows.

  The reserved word 101 described above is set in the program memory 13 with the grammar shown in FIG. The figure is an example of “high, cheese”. “None” is a silent model representing silence, and is a grammar that can be handled with or without “silence” before and after the utterance of “high, cheese” and after “high”. Here, in order to simplify the description, an acoustic model (HMM) with syllables as a unit will be described. Note that the acoustic model may be the word HMM “yes cheese”, or may be an HMM in a finer unit such as phonemes, semiphones, and triphones.

Here, it is assumed that the silence model is a one-state model and the syllable model is a three-state model.
As an example, “Ha” explains.
What is the first state of “ha”? The second state of the HMM model “ha”, which has a high probability of outputting the acoustic features of the “h” consonant “h” part. What is the third state of the HMM model “ha” that has a high probability of outputting acoustic features of “ha”? A model that is learned as an HMM model that has a high probability of outputting the acoustic features of the vowel “a” portion of “ha” It is.

The cumulative likelihood calculation is performed as shown in FIG.
For example, the cumulative likelihood of the first state “I” at time N is
A = (cumulative likelihood of the third state of “ha” at time N−1) × (output probability that is the first state where the feature value at time N is “yes”) × (from the third state of “ha” State transition probability to end “ha”)
B = (cumulative likelihood of the first state of “I” at time N−1) × (output probability that the feature value at the time is the first state of “I”) × (from the first state of “I” Transition to the first state of “I” (loop transition) state transition probability)
A and B obtained as follows are compared, and the one with the larger probability is set as the cumulative likelihood of the first state of “I” at time N.

  In this way, the cumulative likelihood for all states is updated every time. For example, if the time N is the moment immediately after uttering “Chi” after uttering “I” when uttering “Yes cheese”, the cumulative likelihood is the third state of “I”, “ The likelihood of the “none” model between the third state of “I” and the first state of “Chi” should be the highest.

  However, the problem here is when an irrelevant utterance or noise is picked up, and there is also a point with the highest cumulative likelihood at that time. It is conceivable that the likelihood becomes high. Therefore, it is necessary to calculate the reliability of the utterance. For syllables that are not used in “yes cheese” (for example, “A”, “U”, “N”, etc.), the output probability of each frame is calculated, and the value of the model with the highest output probability in each frame is multiplied (if Viterbi) log, so add them). If the utterance was really “yes”, the model with the highest likelihood in each frame is likely to be in a state of “yes” or “yes”, so the cumulative likelihood of “yes” The difference with is less. The reliability is obtained based on this. Of course, the reliability calculation method does not necessarily need to be this method.

  With the above method, the speech recognition unit 12 has the highest cumulative likelihood of the final state of “I” or the “no” model between “I” and “Chi”, and the reliability is equal to or greater than the threshold value. At this time, a signal is sent to the control unit 14 indicating that the utterance of “Yes” has ended. Similarly, when the likelihood of the final state of “Z” or the cumulative likelihood of the “No” model that follows it is the highest and the reliability is equal to or greater than the threshold value, the utterance of “Yes cheese” has ended. A signal indicating a signal indicating is sent.

  On the other hand, while the speech recognition unit 12 performs speech recognition, the control unit 14 stores the sequentially recognized speech patterns in the internal memory, and each time the above signal is sent from the speech recognition unit 12, the speech being recognized. The pattern and the contents of the registration table T are compared to identify candidate reserved words 101 and their utterance stages (step SA3), and the recognition status information indicating them is updated (step SA4). FIG. 6 shows the contents of the recognition status information 103 at a certain time, and FIG. 6A shows the contents at the time when “high” is recognized.

  If the candidate for the reserved word 101 remains (YES in step SA5), the control unit 14 determines whether or not the process proceeds to the next utterance stage (step SA6). If the process has not yet proceeded to the next utterance stage (NO in step SA6), the processes in and after step S2 are repeated. When the process proceeds to the next utterance stage, that is, when it is determined that the previous utterance stage has ended (YES in step SA6), the current reserved word candidate corresponding to the remaining reserved word candidate is determined based on the recognition status information 103 at that time. A process for performing the staged operation is performed (step SA7). That is, the content of the recognition status information 103 is the content shown in FIG. 6A, and automatic focus adjustment is started when the user speaks up to “Yes”. Thereafter, until the current utterance stage is the final stage of the reserved word (candidate) being recognized (NO in step SA8), the process returns to step AS2 and the above-described processing is repeated.

  Thereafter, the content of the recognition status information 103 becomes, for example, as shown in FIG. 6B. When the user speaks up to “Yes, take a picture”, exposure is performed in step SA7. Is as shown in FIG. 6C, for example. When the user speaks up to "High, Trimus, OK", recording is performed in Step SA7. Eventually, if the reserved word candidate disappears while repeating the above process (NO in step SA5), or if the current utterance stage is the final stage of the reserved word 101 being recognized (YES in step SA8), the recognition status information 103 Is cleared (step SA9). Thereafter, the process returns to step AS2, and the subsequent processing is repeated from the beginning.

  As described above, in the voice shutter mode, a reserved word (command word) registered in advance is recognized by voice and a series of shooting operations are performed, but the recognition is performed during the recognition before the entire reserved word is recognized. Every time the stage reaches the utterance stage set as a reserved word, the operation required for shooting is started. Therefore, the time lag between when the user issues a reserved word and when exposure, which is a substantial photographing operation, is performed can be almost eliminated.

  In addition, since a plurality of reserved words having different contents of a series of shooting operations are registered, the user can instruct a shooting operation having different overall operation contents by using a plurality of reserved words. In addition, since the “cheese” and “trimus” portions, that is, the portions that indicate the exposure immediately after the autofocus adjustment are recognized by using an acoustic model including a noise component caused by the autofocus adjustment, the above portions are automatically focused. Even if it is uttered during adjustment, a high recognition rate can be ensured for that portion.

  On the other hand, FIG. 7 is a flowchart showing a processing procedure of the control unit 14 in a command registration mode prepared in advance in the digital camera.

  When the command registration mode is set, the control unit 14 displays a command input method selection screen on the LCD screen of the image display unit 7 and allows the user to select an input method (step SB1). In this embodiment, two types of “selection” and “free” are prepared as input methods. If the user selects “select” (YES in step SB2), any word is selected as a command element from each utterance stage portion of the reserved word (see FIG. 2) already registered (step SB3). If the user selects “freedom” (NO in step SB2), an arbitrary word is input as a command element by a predetermined key operation (step SB4). An arbitrary word input method is performed, for example, by displaying a character selection screen such as 50 sounds on the LCD screen of the image display unit 7 and selecting a plurality of characters by key operation.

  Subsequently, three types of operations at the time of photographing that can be designated, that is, “automatic focus adjustment”, “exposure”, and “recording” in the present embodiment are displayed, and a command element selected or input from among them is displayed. One or a plurality of operations are selected (step SB5). Then, after incrementing the number of utterance stages (step SB1), if there is no command input end instruction (NO in step SB7), the process returns to step SB1, and the command element of the next generation stage is given to the user by the same processing as described above. Select or input and select the corresponding action.

  Then, after selecting one or a plurality of actions in step SB6, if there is a command input end instruction (YES in step SB7), “,” (in the utterance stage) between the selected or input command elements. (Separator) is automatically inserted to generate a new reserved word (step SB8). Although not shown in the figure, when there is one command element selected or input by the user, the command element is used as a reserved word as it is. Then, the registration table T is updated by adding the generated new reserved word (step SB9).

  Therefore, the user can freely add a command word that can be used at the time of shooting in the above-described voice shutter mode by selecting the command registration mode.

  In the above description, the case where the present invention is applied to a digital camera as a dedicated machine has been described. However, the present invention is not limited to this, and the present invention can be applied to other camera devices such as a camera-equipped mobile phone and a silver salt camera. Can also be adopted.

It is a block diagram which shows schematic structure of the digital camera which concerns on this invention. It is a conceptual diagram which shows the data memorize | stored in the program memory. It is a flowchart which shows the operation | movement in an audio | voice shutter mode. It is a figure which shows the grammar of the reserved word currently recorded. It is a figure which shows the cumulative likelihood calculation method during speech recognition. It is the figure which showed the example of a change of the recognition status information in voice recognition. It is a flowchart which shows the process sequence of the control part in command registration mode.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Key input part 2 Focusing part 3 Exposure part 7 Image display part 8 Microphone 9 Voice input part 10 A / D conversion part 11 Work memory 12 Voice recognition part 13 Program memory 14 Control part 101 Reserved word 102 Corresponding information T Registration table

Claims (6)

A camera apparatus having an automatic focus adjustment function for performing a series of photographing operations consisting of a plurality of steps using a voice input of a predetermined command as a trigger,
Speech recognition means for recognizing input speech;
Command word storage means for storing the predetermined command word;
Correspondence information storage means for storing correspondence information indicating a correspondence relationship between a plurality of utterance stages set in the instruction word stored in the instruction word storage means and the operation of each stage in the series of photographing operations;
Each time the speech recognition stage recognized by the speech recognition means reaches each utterance stage of the command word, the series of series corresponding to the utterance stage indicated by the stage information stored in the correspondence information storage means Control means for sequentially starting the operation of each stage in the shooting operation,
The multi-stage operation in the series of photographing operations includes a focusing operation and an exposure operation by an automatic focus adjustment function, and the voice recognition unit includes an acoustic model including a noise component generated during the automatic focus adjustment operation in the photographing operation; An acoustic model that does not include, and in response to the recognition of the instruction word instructing the automatic focus adjustment in the series of shooting operations, the acoustic model that includes the noise component is changed from the acoustic model that does not include the automatic focus adjustment. A camera device characterized by recognizing a voice in a utterance stage next to a utterance stage of the command word instructing .   The camera apparatus according to claim 1, wherein the operation at any stage in the series of photographing operations includes a plurality of operations performed in succession.   A plurality of command words are stored in the command word storage means, and each utterance stage set in each of the plurality of command words stored in the command word storage means in the correspondence information storage means, The camera apparatus according to claim 1, wherein a plurality of correspondence information indicating a correspondence relationship with each step of the series of photographing operations is stored. Registration means for storing a word composed of speech recognized by the voice recognition means in the command word storage means as a new command word;
The registration unit generates new correspondence information indicating a correspondence relationship between a plurality of utterance stages in the new command word stored in the command word storage unit and each stage operation in the series of photographing operations, and the correspondence The camera device according to claim 1, further comprising a generation unit that stores the information in the information storage unit. An imaging method in a camera device having an automatic focus adjustment function for performing a series of imaging operations consisting of a plurality of steps using a voice input of a predetermined command as a trigger,
The step of recognizing the input voice sequentially,
Each time the recognized speech recognition stage reaches each utterance stage of the plurality of utterance stages set in the command word, the operation of each stage in the series of photographing operations associated with each utterance stage is started in order. Including the steps of:
The operations in a plurality of stages in the series of photographing operations include a focusing operation and an exposure operation by an automatic focus adjustment function, and the step of sequentially recognizing voice recognizes a command word instructing automatic focus adjustment in the series of photographing operations. In response to the above, the acoustic model including the noise component is changed from the acoustic model not including the noise component, and the speech of the utterance stage next to the utterance stage of the command word instructing the automatic focus adjustment is recognized. Shooting method. A computer having a camera device having an automatic focus adjustment function for performing a series of photographing operations consisting of a plurality of steps using a voice input of a predetermined command as a trigger
A process for causing the voice recognition means to sequentially recognize the input voice;
Each stage in the series of photographing operations associated with each utterance stage each time the speech recognition stage recognized by the voice recognition means reaches each utterance stage of a plurality of utterance stages set in the command word Process to start each part of the device in order,
The multi-stage operation in the series of shooting operations includes a focusing operation and an exposure operation by an automatic focus adjustment function, and the process of sequentially recognizing the voice to the voice recognition means instructed the automatic focus adjustment in the series of shooting operations. Recognizing the speech in the utterance stage next to the utterance stage of the instruction word instructed to perform automatic focus adjustment by changing from the acoustic model not including the noise component to the acoustic model including the noise component in response to the recognition of the instruction word A program characterized by

Images