JP2009232227A - Imaging device and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device and an imaging method by which favorable image data can be obtained when exposure operation is performed more than once to perform multiple exposure imaging operation for creating a piece of image data. <P>SOLUTION: The imaging device performs exposure operation more than once to create a piece of image data. Notification operation is started at least once during a multiple exposure period from the start of a first exposure period when the first exposure operation is performed until the end of the nth exposure period when the last exposure operation is performed. In this manner, since a user can be notified that multiple exposure imaging is under execution, the imaging device can be prevented from being moved in a large way during the multiple exposure period. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus capable of performing a multiple exposure imaging operation in which an exposure operation is performed a plurality of times to create one image data, and an imaging method thereof.

  Imaging devices represented by digital still cameras that create still image data from input optical images and digital video cameras that can create both moving and still image data have become widespread in recent years. ing.

  When creating image data of a still image using the above-described imaging device, the obtained image data may be deteriorated because the optical image moves during the optical image input period (exposure period). It becomes a problem. This problem occurs mainly due to a so-called camera shake that causes the user to shake the imaging apparatus without keeping it at a certain position during the exposure period.

In order to solve this problem, various image pickup apparatuses that obtain good image data by reducing the influence of camera shake during the exposure period have been proposed. In particular, Patent Document 1 proposes an imaging apparatus that performs a plurality of exposure operations, shortens each exposure period, and synthesizes a plurality of obtained image data to create one image data. By performing such processing, it is possible to reduce the influence of blurring of the imaging apparatus with respect to each exposure period as much as possible, and to create good image data by synthesis.
JP 2007-104029 A

  However, when the image data is created by performing the exposure operation a plurality of times as described above, the imaging devices need to be directed in substantially the same direction in all the exposure periods. In particular, if the imaging device is moved greatly before all exposure operations performed a plurality of times are completed, the optical images input during each exposure period will differ greatly, and a plurality of obtained image data will be synthesized. The problem becomes difficult.

  In view of such problems, the present invention provides an imaging apparatus and an imaging method capable of obtaining good image data when performing a multiple exposure imaging operation in which a plurality of exposure operations are performed to create one image data. For the purpose.

  In order to achieve the above object, an imaging apparatus according to the present invention includes a plurality of images created by the imaging unit in an imaging apparatus including an imaging unit that performs an exposure operation for creating image data based on an input optical image. An image processing unit that creates one image data based on the data, and a notification unit that performs a notification operation when the plurality of image data are generated by the imaging unit, the imaging unit including the exposure operation When creating the plurality of image data by performing a plurality of times, from the start of the exposure operation performed first among the exposure operations performed a plurality of times until the end of the exposure operation performed last, The notification unit starts the notification operation at least once during a multiple exposure period, which is a period of.

  In the imaging apparatus having the above-described configuration, the notification unit may perform a long notification operation performed during a period including at least two periods during which the exposure operation is performed as one of the notification operations.

  With this configuration, the notification operation is performed so as to cover a period in which a plurality of exposure operations are performed. And it becomes possible to alert | report to a user about the period when several exposure operation | movement is performed. Therefore, at least a plurality of image data can be made good, and image data created based on these image data can be made good.

  Further, if the long notification operation is performed, a period during which the notification operation is performed becomes long, so that it is easy for the user to recognize that a plurality of pieces of image data are created. Therefore, it is possible to suppress the image pickup apparatus from being moved greatly during the multiple exposure period.

  Further, in the imaging apparatus having the above-described configuration, the number of the plurality of image data created by the imaging unit necessary for creating the one image data by the image processing unit, that is, the number of times of performing the exposure operation is variable. It doesn't matter. Further, the period during which the exposure operation is performed may be variable. Further, at least one of the number of the plurality of image data created and the period during which the exposure operation is performed may be determined based on the input optical image.

  With this configuration, it is possible to appropriately change the number of image data created and the period during which the exposure operation is performed in accordance with surrounding imaging conditions. Therefore, it is possible to improve each of the plurality of image data.

  In the imaging device having the above-described configuration, the notification unit may perform the long notification operation by processing one notification signal. With this configuration, even if the number of image data created or the period for performing the exposure operation changes as described above and the period for performing the long notification operation changes, it is possible to easily follow and notify. It becomes.

  Further, in the imaging device having the above-described configuration, the notification unit outputs the voice to perform the long notification operation, and extends and outputs at least a part of the notification sound shorter than the period during which the long notification operation is performed. Thus, the long notification operation may be performed.

  By comprising in this way, the alert | reported audio | voice will be extended and output. Therefore, it can be made easy to convey to the user that the multiple exposure period is long.

  Further, in the imaging device having the above-described configuration, the notification unit outputs the sound to perform the long notification operation, and echoes and outputs a notification sound shorter than a period in which the long notification operation is performed. The long notification operation may be performed.

  By configuring in this way, the voice to be notified becomes natural. In addition, since the notification sound is simply echoed using a filter or the like, the processing becomes simple.

  Further, in the imaging apparatus having the above-described configuration, the notification unit outputs voice and performs the long notification operation, and at the same time, repeatedly outputs at least part of the notification sound shorter than the period during which the long notification operation is performed. The long notification operation may be performed.

  By comprising in this way, the alert | reported audio | voice will be repeated. Therefore, it is possible to easily convey the image of the multiple exposure imaging operation to the user.

  In the imaging apparatus having the above-described configuration, the long notification operation may be performed during a period including all of the multiple exposure periods. With this configuration, it is possible to notify the user during all the multiple exposure periods. Therefore, it is possible to reliably suppress the image pickup apparatus from being largely moved during the multiple exposure period.

  In the imaging apparatus having the above-described configuration, the long notification operation is performed at least from the start of the period in which the second exposure operation is performed from the beginning to the end of the period in which the second exposure operation is performed from the end. It may be performed in a period including this period. With this configuration, it is possible to notify the user over almost the entire multiple exposure period, and it is also possible not to notify during a period when the imaging apparatus is difficult to move. As a result, it is possible to suppress notification for an unnecessarily long time, and to save power and improve the impression given to the user.

  In the imaging device having the above-described configuration, the notification unit may perform the long notification operation by displaying predetermined information.

  In the imaging device having the above configuration, the notification unit may use, as the predetermined information, an image based on the image data created by the imaging unit immediately before the imaging unit creates the plurality of image data. It may be displayed.

  By comprising in this way, it becomes possible to appeal to a user's vision that it is a multiple exposure period. Therefore, it is possible to clearly notify the user that it is the multiple exposure period.

  Furthermore, it is also possible to combine the above-described voice notification operation and the notification operation by displaying predetermined information. Thus, it becomes possible to appeal to the user more strongly by combining the notification operations of a plurality of different methods.

  In the imaging device having the above-described configuration, the notification unit may perform an end notification operation performed during a period including the end of the multiple exposure period as one of the notification operations.

  With this configuration, it is possible to clearly notify the user of the end of the multiple exposure period. In particular, when performing in addition to other notification operations such as a long notification operation, it is preferable to notify with a sound or display different from other notification operations because the difference becomes clearer.

  The imaging method according to the present invention is created by a first step of creating a plurality of image data by performing an exposure operation for creating image data a plurality of times based on an input optical image, and the first step. A second step of creating one image data based on the plurality of image data, and is performed lastly from the start of the exposure operation performed first among the exposure operations performed a plurality of times in the first step. The notification operation is started at least once during the multiple exposure period, which is a period until the end of the exposure operation.

  According to the present invention, the notification operation is started at least once during the multiple exposure period in which a plurality of exposure operations are performed. Therefore, it is possible to notify the user about the multiple exposure period. Therefore, it is possible to suppress the user from moving the imaging apparatus greatly during the multiple exposure period.

  For example, if notification is given at the beginning of the multiple exposure period, it is difficult to perform a large amount of movement during the multiple exposure period when the user thinks that the operation of the imaging apparatus has failed because the notification operation is not easily performed. It becomes possible to do. Further, if notification is given at the end of the multiple exposure period, it is possible to prevent the imaging apparatus from being moved greatly by assuming that the multiple exposure period has already ended. On the other hand, if the notification is made in the middle of the multiple exposure period, both the effect of informing in the early stage and the effect of informing in the final stage can be obtained.

<Configuration of imaging device>
First, the configuration of an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of the imaging apparatus. Here, a digital video camera capable of creating both moving image data and still image data will be described as an imaging device. However, only a still image data such as a digital still camera is created. It does not matter.

  As shown in FIG. 1, the imaging device 1 includes an imaging unit 2 that creates image data from an input optical image. The image pickup unit 2 includes an image sensor 3 formed of a solid-state image pickup device such as a CCD (Charge Coupled Devices) or a CMOS (Complementary Metal Oxide Semiconductor) sensor that converts incident light into an electric signal, and an optical image of a subject. And a lens unit 4 that adjusts exposure, focus, and the like.

  Further, the imaging apparatus 1 includes an AFE (Analog Front End) 5 that converts analog image data output from the image sensor 3 into digital and adjusts gain, and a microphone that collects sound and converts it into an electrical signal. 6, an image processing unit 7 that performs various image processing such as gradation correction processing on image data that is a digital signal output from the AFE 5 and generates control data for controlling the imaging unit 2, and a microphone 6 The audio processing unit 8 converts the analog audio data output from the digital to digital and performs various audio processing such as noise removal, and performs encoding processing such as JPEG (Joint Photographic Experts Group) compression on the still image data A compression processing unit 9 that performs compression coding processing such as MPEG (Moving Picture Experts Group) compression method on image data and audio data of moving images An external memory 10 that records the compressed encoded signal compressed by the compression processing unit 9, a driver unit 11 that records and reads the compressed encoded signal in the external memory 10, and an external memory in the driver unit 11. And a decompression processing unit 12 that decompresses and decodes the compression-coded signal read out from 10.

  The imaging device 1 also includes an image output circuit unit 13 that converts image data obtained by decoding by the expansion processing unit 12 into an analog signal for display on a display device (not shown) such as a display, and the expansion processing unit 12. The audio output circuit unit 14 that converts the audio data obtained by decoding into an analog signal to be reproduced by an audio reproduction device (not shown) such as a speaker, and the fact that the image data is recorded is indicated to the user And an informing unit 15 for informing.

  The imaging apparatus 1 also stores a CPU (Central Processing Unit) 16 that controls the overall operation of the imaging apparatus 1 and a memory 17 that stores each program for performing each process and temporarily stores data when the program is executed. A timing generator (TG) that outputs a timing control signal for matching the operation timing of the operation unit 18 to which an instruction from the user is input, such as a button for starting imaging and recording, a button for adjusting imaging conditions, and the like. ) Unit 19, a bus line 20 for exchanging data between the CPU 16 and each block, and a bus line 21 for exchanging data between the memory 17 and each block.

  The external memory 10 may be anything as long as it can record image data and audio data. For example, a semiconductor memory such as an SD (Secure Digital) card, an optical disk such as a DVD, a magnetic disk such as a hard disk, or the like can be used as the external memory 10. Further, the external memory 10 may be detachable from the imaging device 1.

  In addition, in the case of a digital still camera that does not create moving image data, a portion that processes input audio may not be provided. For example, the microphone 6 and the audio processing unit 8 may be omitted. However, the part relevant to the alerting | reporting part 15 shall be provided. For example, the imaging apparatus 1 illustrated in FIG. 1 includes the notification unit 15 and the audio output circuit unit 14.

<Basic operation of imaging device>
Next, the basic operation of the imaging apparatus 1 will be described with reference to FIG. First, in the imaging device 1, the CPU 16 controls the shutter and diaphragm of the lens unit 4 and various lens positions, and the exposure and focus are controlled. Then, the optical image incident and projected from the lens unit 4 is converted into image data as an electrical signal by being photoelectrically converted in the image sensor 3. Then, image data is output from the image sensor 3 to the AFE 5 in synchronization with the timing control signal input from the TG unit 19.

  The image data converted from an analog signal to a digital signal and adjusted in gain by the AFE 5 is input to the image processing unit 7. The image processing unit 7 converts input image data into image data composed of a luminance signal and a color difference signal, and performs various image processing such as gradation correction and contour enhancement. The memory 17 operates as a frame memory, and temporarily holds image data when the image processing unit 7 performs processing.

  Further, the image processing unit 7 creates control data used when automatically adjusting the focus and exposure in the lens unit 4 based on the input image data. Then, based on this control data, the CPU 16 controls the lens unit 4 and automatically adjusts focus and exposure. When the user manually adjusts focus, exposure, and the like, the CPU 16 controls the lens unit 4 based on an instruction input from the user via the operation unit 18.

  On the other hand, the microphone 6 creates sound data by collecting sound and converting it into an electrical signal. The created sound data is input to the sound processing unit 8 and converted into a digital signal, and various sound processing such as noise removal and adjustment of the gain of the sound data is performed.

  Then, both the image data output from the image processing unit 7 and the audio data output from the audio processing unit 8 are input to the compression processing unit 9 and compressed by the compression processing unit 9 using a predetermined compression method. At this time, the image data and the audio data are temporally associated with each other so that the image and the sound are not shifted during reproduction. The compressed image data and audio data are recorded in the external memory 10 via the driver unit 11. On the other hand, when image data of a still image is recorded, the image data is compressed by a predetermined compression method and recorded in the external memory 10 without being associated with audio data.

  The series of operations described above is started when the user inputs an instruction to start recording image data or audio data to the operation unit 18 or is instructed to start recording by a program such as a timer. The end of recording of image data and audio data is the same, and is performed by a user instruction or a program. At this time, the notification unit 15 notifies the user of the start or end of creation of image data and audio data to be recorded. The details of the notification operation of the notification unit 15 will be described later.

  The imaging apparatus 1 is capable of performing a multiple exposure imaging operation when creating still image data. For example, when performing addition-type image stabilization processing as the multiple exposure imaging operation, the imaging unit 2 performs an exposure operation a plurality of times and outputs a plurality of created image data one by one. The plurality of output image data are combined by the image processing unit 7 into one image data. The details of the addition-type image stabilization processing will be described later.

  The compressed encoded signal recorded in the external memory 10 is read out to the decompression processing unit 12 based on a user instruction. The decompression processing unit 12 decompresses and decodes the compressed encoded signal to obtain image data and audio data. The obtained image data is input to the image output circuit unit 13, and the audio data is input to the audio output circuit unit 14. Then, it is converted into an analog signal in the image output circuit unit 13 and the audio output circuit unit 14, converted into a format reproducible in the display device and the audio reproduction device, and output.

  Note that the display device and the audio reproduction device may be integrated with the imaging device 1 or may be separate and connected using terminals and cables provided in the imaging device 1. It does not matter as long as

  Moreover, although the alerting | reporting part 15 shall notify to a user with an audio | voice, it is good also as notifying to a user with light or another method in addition to an audio | voice (or instead of an audio | voice). For example, notification may be performed by performing a predetermined display on the display device, or notification may be performed by lighting or blinking an LED (Light Emitting Diode) lamp. Further, the notification unit 15 may be provided with a dedicated sound reproduction device, and the sound reproduction device may perform notification. When notifying by performing a predetermined display on the display device, a signal output from the notification unit 15 is input to the image output circuit unit 13. Further, when notification is made using an LED lamp or a dedicated sound reproduction device, a signal is input from the notification unit 15 to each device.

  Also, a so-called preview mode in which the image data created by the imaging unit 2 is displayed on the display device one by one without being recorded in the external memory 10 before the image data is recorded, and the composition of the image to be recorded is confirmed by the user. It does not matter as having. In the preview mode, the image data output from the image processing unit 7 may be output to the image output circuit unit 13 without being compressed.

<Additive image stabilization processing>
Next, an example of the addition-type image stabilization processing executed by the above-described imaging device 1 in FIG. 1 will be described with reference to the drawings. FIG. 2 is a schematic diagram of image data that is subjected to addition-type image stabilization processing. FIGS. 2A and 2B show examples of image data 30 and 31 obtained in different exposure periods, and FIG. 2C shows image data 30 shown in FIGS. 2A and 2B. , 31 are arranged in an overlapping manner.

  The black areas shown in FIGS. 2A and 2B are obtained based on predetermined processing (for example, a determination result based on the color and luminance of each pixel, a determination result of recognition of a subject such as a face, and the like). The feature regions 30a and 31a extracted through the process of dividing the data into feature regions and non-feature regions). Further, it is assumed that the imaging device 1 does not move greatly when the image data 30 and 31 shown in FIGS.

  When the imaging apparatus 1 does not move greatly and the exposure operation is performed in the same direction, the feature region 30a of the image data 30 shown in FIG. 2A and the image data 31 shown in FIG. The feature region 31a has a similar shape. However, the position of the feature region 30a in the image data 30 shown in FIG. 2A and the position of the feature region 31a in the image data 31 shown in FIG. It is different.

  In such a case, as shown in FIG. 2C, the composition is performed so that the feature regions 30a and 31a in the respective image data 30 and 31 are overlapped. In the case of the example shown in FIG. 2, the image data 31 shown in FIG. 2B shown by a broken line is moved upward by one pixel and left by one pixel with respect to the image data 30 shown in FIG. The regions 30a and 30b can be overlapped.

  Then, a predetermined synthesis coefficient based on the positional relationship shown in FIG. 2C is added to each of the image data 30 and 31 in FIGS. 2A and 2B, and weighted addition is performed. As a result, image data is synthesized.

  Note that the synthesis coefficient may be determined for each pixel, for example. Further, whether or not the pixel is included in the feature areas 30a and 31a, and whether or not there is a pixel that overlaps the pixel of other image data when the image data is arranged at the position shown in FIG. The value of the synthesis coefficient may be changed based on this. In addition, the value of the synthesis coefficient may be changed based on the direction and size of the feature regions 30a and 30b that are moved to overlap, that is, the direction and size of movement of the imaging device 1.

  In addition, the number of exposure operations may be increased or decreased based on the imaging conditions. For example, the overall exposure time may be determined based on the aperture of the aperture, the brightness of the image data, and the like, and the single exposure time may be determined based on the focal length. Then, the number of times of performing the exposure operation may be determined from the entire exposure time and one exposure time.

  In addition, as an example of a multiple exposure imaging operation in which multiple exposure operations are performed to create one image data, the addition-type image stabilization processing has been described. However, the present invention is not limited to other methods as long as it is a multiple exposure method. It can be applied even if it exists.

<Notification action>
(Single exposure imaging operation)
Next, a notification operation of the notification unit 15 when creating still image data to be recorded in the external memory 10 of FIG. 1 will be described. Here, when creating single image data, a single-exposure imaging operation that performs an exposure operation only once, and when performing single-exposure imaging operation that performs multiple exposure operations when creating one image data, are performed. This will be described in comparison with the case where First, a notification operation example of the notification unit 15 when performing a single exposure imaging operation will be described with reference to the drawings.

  FIG. 3 is a timing chart illustrating an example of a notification operation of the notification unit when performing a single exposure imaging operation. The shutter trigger in FIG. 3 is a signal output from the CPU 16 in FIG. 1 when an instruction to record an input optical image as still image data is issued by a user or a program. is there. Based on this signal, the imaging apparatus 1 performs an operation for creating still image data to be recorded in the external memory 10.

  The image sensor in FIG. 3 indicates the drive timing of the image sensor 3 in FIG. In particular, a period during which the image sensor 3 performs photoelectric conversion of the projected optical image is indicated as an “exposure period”, and a period during which image data created by the image sensor 3 is read out to the AFE 5 is indicated as a “read period”. .

  Also, the notification (voice) in FIG. 3 indicates the timing of the notification operation performed by the notification unit 15 in FIG. 1 for the user. In particular, FIG. 3 shows a case where the user is notified by voice, and as an example, a case where a notification sound of “Cashan” (a sound easily recalled as an operation sound of the imaging apparatus) is shown.

  The mechanical shutter in FIG. 3 is a shutter provided in the lens unit 4 in FIG. 1 and determines whether an optical image projected on the image sensor 3 is acceptable. If it is in the open state, an optical image is projected onto the image sensor 3, and if it is in the closed state, it indicates that no optical image is projected onto the image sensor 3.

  In the example shown in FIG. 3A, the exposure period of the image sensor is started according to the input of the shutter trigger, and the mechanical shutter is opened. At this time, the output of the notification sound of the tone “Cashan” is started in accordance with the start of the exposure period. Then, the mechanical shutter is closed at the end of the exposure period of the image sensor, and then the reading period of the image sensor is started. Image data is output from the image sensor during this readout period.

  FIGS. 3B and 3C show an example in which the notification operation is performed at a timing different from that in FIG. 3 (b), 3 (c) and FIG. 3 (a) differ only in the timing of the notification operation, and the operation timings of the other parts are the same. Therefore, detailed description of operations other than the notification operation is omitted.

  In the example shown in FIG. 3B, the output of the notification sound of the tone color “Cashan” is started at the end of the exposure period of the image sensor. In the example shown in FIG. 3C, the output of the notification sound of the tone “Cashan” is started at the end of the reading period of the image sensor.

  Moreover, as shown in FIG. 4, it is also possible to notify by an image. FIG. 4 is a timing chart showing the operation of the notification unit when performing a single exposure imaging operation. Although the notification method and the notification timing are different between FIG. 3 and FIG. 4, the operation timing of other parts is the same. Therefore, detailed description of operations other than the notification operation is omitted.

  The notification (image) in FIG. 4 indicates the timing of the notification operation performed by the notification unit 15 in FIG. 1 for the user. In particular, the case of notifying the user by an image is shown. “Black screen display” indicates that a black screen is displayed without displaying an image on the display device. “Preview image display” indicates that the image data created by the image sensor is displayed on the display device immediately before the shutter trigger is input. For example, in the preview mode performed before creating image data of a still image for recording, the image data displayed last among the image data displayed to the user is continuously displayed on the display device after the shutter trigger is input. Etc.

  In the example shown in FIG. 4A, black screen display is started at the start of the exposure period. In the example shown in FIG. 4B, preview image display is performed at the start of the exposure period, and black screen display is performed at the end of the exposure period.

  In the case of performing the single exposure imaging operation, the exposure period is one time, and it is not necessary to synthesize image data. Therefore, FIGS. 3A to 3C and FIGS. 4A and 4B described above. As shown in the example, the notification may be made by any method and timing. In particular, if the single-exposure imaging operation is performed, or if the configuration is such that the user is notified quickly, it is unlikely that the user greatly moves the imaging device during a single exposure period. Therefore, it can suppress that an imaging device is moved largely. However, it tends to be reflected in image data that can be influenced by small movements such as camera shake during the exposure period.

(Multiple exposure imaging operation)
In the single exposure imaging operation, it is difficult to reduce the influence of small movements such as camera shake as described above. On the other hand, in the multiple exposure imaging operation described below, it is possible to reduce the influence of camera shake by performing an exposure operation multiple times with a shorter exposure period.

  However, since the multiple exposure imaging operation performs the exposure operation a plurality of times, the entire exposure period tends to be long. For this reason, it is easy for the user to move the imaging apparatus greatly during the multiple exposure imaging operation. Therefore, in the imaging apparatus according to the present embodiment, the notification operation as in the following example is performed so that the user can sufficiently recognize the multiple exposure imaging operation, and the imaging apparatus is moved greatly during the multiple exposure imaging operation. Suppress.

[First embodiment]
A first embodiment of a notification operation when performing a multiple exposure imaging operation will be described with reference to the drawings. FIG. 5 is a timing chart showing the first embodiment of the operation of the notification unit when performing the multiple exposure imaging operation, and corresponds to FIG. 3 showing the single exposure imaging operation. In addition, the same name is attached | subjected about the part similar to FIG. 3, and the detailed description is abbreviate | omitted.

  As shown in FIG. 5, when a multiple exposure imaging operation is performed, an exposure operation and a reading operation are alternately repeated in the image sensor after a shutter trigger is input. Specifically, the first exposure operation is performed in the “first exposure period” after the shutter trigger is input, and then the first read operation is performed in the “first readout period”. Then, after the first readout period, the second exposure operation is performed in the “second exposure period”, and then the second readout operation is performed in the “second readout period”. The same applies thereafter, and the n-th exposure operation is performed in the “n-th exposure period”, and then the n-th read operation is performed in the “n-th read period”. With the above operation, n pieces of image data are obtained, and one image data is created based on the n pieces of image data, thereby completing the multiple exposure imaging operation. Note that n is an integer of 2 or more.

  Further, a period from the start of the first exposure period of the image sensor to the end of the nth exposure period is referred to as a “multiple exposure period”. Further, the mechanical shutter is opened during each of the first to nth exposure periods, and is closed during the other periods.

  In this example, the above-described notification sound of the tone “Cashan” is output at an arbitrary timing during the multiple exposure period. FIG. 5 shows the case where the notification sound starts to be output from the end of the multiple exposure period, that is, from the end of the nth exposure period.

  If notification is performed as shown in FIG. 5, the notification is performed at the end of the multiple exposure period. Therefore, it is possible to notify the user that the multiple exposure period, that is, the period during which the imaging apparatus cannot be moved greatly, has ended. Therefore, it is possible to suppress the user from moving the imaging apparatus greatly during the multiple exposure period. In the example shown in FIG. 5, the notification sound output ends during the n-th reading period. However, the notification sound output may end after the n-th reading period ends.

  Further, even when starting the output of the notification sound so that the end of the multiple exposure period and the end of the notification sound overlap, the user is made to recognize the end of the multiple exposure period as in the example shown in FIG. It becomes possible. Further, even if the output of the notification sound is started from an arbitrary position between the output start position of the notification sound in this case and the output start position of the notification sound (at the end of the multiple exposure period) shown in FIG. The end of the multiple exposure period is included in the period for outputting the notification sound. Therefore, it is possible to notify the user of the end of the multiple exposure period.

  In the above-described example, the end of the multiple exposure period is notified to the user to suppress the image pickup apparatus from being moved greatly. However, the notification is started at the beginning or middle of the multiple exposure period. In addition, it is possible to prevent the imaging apparatus from being moved greatly. For example, if notification is started in the early stage, it is possible to prevent the user from moving the imaging device greatly during the multiple exposure period assuming that the user has failed to operate the imaging device because the notification sound is not easily output. It becomes possible. On the other hand, when the notification is started in the middle, the user thinks that the operation has failed and moves the imaging apparatus greatly during the multiple exposure period, and the user moves the imaging apparatus greatly by assuming that the multiple exposure period has already ended. Both can be suppressed.

  Further, for example, the output of the notification sound may be started at a plurality of locations in the multiple exposure period, such as in the early and middle stages, the middle stage, and the final stage of the multiple exposure period. If notification is made a plurality of times in this way, it becomes possible to notify the user of the presence or absence of the multiple exposure imaging operation more reliably.

[Second Embodiment]
A second embodiment of the notification operation when performing the multiple exposure imaging operation will be described with reference to the drawings. FIG. 6 is a timing chart showing the second embodiment of the operation of the notification unit when performing the multiple exposure imaging operation, and corresponds to FIG. 5 showing the first embodiment. In addition, the same name is attached | subjected about the part similar to FIG. 5, and the detailed description is abbreviate | omitted. In the following description, for convenience of explanation, it is assumed that “Cashan”, which is the tone color of the notification sound, is separable into three parts “Ka”, “Sha” and “N”.

  Operations other than the notification operation in this example, that is, operations such as a shutter trigger, an image sensor, and a mechanical shutter are the same as those in the first embodiment. Therefore, detailed description of these operations is omitted. Further, the point that the output of the notification sound is started during the multiple exposure period is the same as in the first embodiment.

  In this example, notification is performed by outputting a long notification sound. The long notification sound is a notification sound having a length including at least two exposure periods. In the example shown in FIG. 6, a notification sound is output from the end of the first exposure period to the end of the nth exposure period. That is, a long notification sound having a length including n−1 exposure periods is output.

  Moreover, in the example shown in FIG. 6, the output period is lengthened by deforming a predetermined notification sound shorter than the period in which the long notification sound is output. For example, the long notification sound is output by a method of extending and outputting a sound (“sha”) that is intermediate in time to the tone color (“cashan”) of a predetermined notification sound.

  If the notification sound is output so that a plurality of exposure periods are included as in this example, at least the plurality of exposure periods for which the notification sound is output is a multiple exposure period for the user. It is possible to notify. Therefore, it is possible to suppress the image pickup apparatus from being moved greatly during a plurality of exposure periods in which notification is performed, and it is possible to obtain a plurality of good image data. Therefore, it is possible to improve one piece of image data created based on a plurality of obtained image data.

  Further, if notification is performed by outputting a long notification sound, the user can easily recognize that the multiple exposure imaging operation is being performed. In particular, when the notification sound is extended and output, it is easy for the user to recall that the multiple exposure period is long. Therefore, it is possible to suppress the image pickup apparatus from being moved greatly during the multiple exposure period.

  In addition, since only existing sound data for notification sound is adjusted and output, it is not necessary to provide notification sound dedicated to the multiple exposure imaging operation. In particular, in the case where the multiple exposure period is made variable according to the imaging conditions and the notification period is also made variable, in any case, the method for adjusting the existing audio data as in this example Can also respond.

  In addition, you may combine this example and 1st Example. That is, the notification sound output in the first embodiment may be the long notification sound of this example. With this configuration, the effect of the first embodiment can be further enhanced. That is, it is possible to further suppress the image pickup apparatus from being greatly moved.

  Further, it is preferable that the long notification sound is output so that the entire multiple exposure period is included, because the user can be encouraged not to move the imaging apparatus during the entire multiple exposure period.

  On the other hand, as shown in FIG. 6, it is not output at a predetermined exposure period (for example, the first exposure period) in the early stage, or is not output at a predetermined exposure period (for example, the nth exposure period) at the end, The output may not be performed during a predetermined exposure period at the beginning and end. At this time, the user recognizes that the notification sound is not output for a short period from the time when the shutter trigger is input to the start of the notification sound output, or for a short period from the end of the notification sound output to the end of the multiple exposure period. Therefore, it is difficult to move the image pickup apparatus greatly. Therefore, even with such a configuration, it is possible to suppress the user from moving the imaging apparatus greatly during the entire multiple exposure period. In addition, the notification sound is not output for a long time, and the power saving and the impression given to the user can be improved. Similarly, the notification sound may not be output for the second exposure period and the (n-1) th exposure period.

[Third embodiment]
A third embodiment of a notification operation when performing a multiple exposure imaging operation will be described with reference to the drawings. FIG. 7 is a timing chart showing the third embodiment of the operation of the notification unit when performing the multiple exposure imaging operation, and corresponds to FIG. 5 and FIG. 6 showing the first and second embodiments. In addition, the same name is attached | subjected about the part similar to FIG.5 and FIG.6, and the detailed description is abbreviate | omitted.

  In this example, a long notification sound is output by echoing the notification sound. In the example shown in FIG. 7, the notification sound is output from the end of the first exposure period to the end of the nth exposure period. The point of outputting such a long notification sound is the same as in the second embodiment, and the notification period is also the same as in the second embodiment. Therefore, the effect obtained by outputting the long notification sound can be obtained in this example as well as the second embodiment.

  As in this example, it is possible to output a natural notification sound by echoing the notification sound and outputting it as a long notification sound. In addition, since the notification sound is simply echoed using a filter or the like, the processing becomes simple. In addition, if the configuration is such that the existing notification sound is adjusted and output as in the second embodiment, it is not necessary to provide a notification sound dedicated to the multiple exposure imaging operation, and even if the notification period is variable, it can be easily handled. can do.

  In addition, you may combine this example and 1st Example. That is, the notification sound output in the first embodiment may be the long notification sound of this example. With this configuration, the effect of the first embodiment can be further enhanced. That is, it is possible to further suppress the image pickup apparatus from being greatly moved.

[Fourth embodiment]
A fourth embodiment of a notification operation when performing a multiple exposure imaging operation will be described with reference to the drawings. FIG. 8 is a timing chart showing the fourth embodiment of the operation of the notification unit when performing the multiple exposure imaging operation, and corresponds to FIGS. 5 to 7 showing the first to third embodiments. In addition, the same name is attached | subjected about the part similar to FIGS. 5-7, and it abbreviate | omits about the detailed description.

  In this example, a short notification sound shorter than a long notification sound is repeated to constitute a long notification sound. Further, in the example shown in FIG. 8, a long notification sound is output from the end of the first exposure period to the end of the nth exposure period. The point of outputting the long notification sound is the same as in the second and third embodiments, and the period of notification of the long notification sound is also the same as in the second and third embodiments. Therefore, the effect obtained by outputting the long notification sound can be obtained in this example as well as the second and third examples.

  In this example, at the end of the multiple exposure period, an end notification sound that is a notification sound different from the short notification sound constituting the long notification sound is output. In the example shown in FIG. 8, the short notification sound is obtained by shortening the end notification sound.

  As in this example, by outputting a short notification sound as a long notification sound, the user can easily recognize that the multiple exposure imaging operation is being performed. In particular, if the notification sound is shortened and output repeatedly, the image of the multiple exposure imaging operation is easily transmitted to the user. Therefore, it is possible to more effectively act on the user not to move the imaging apparatus greatly during a plurality of exposure periods.

  In addition, after the multiple exposure period ends, by outputting an end notification sound different from the short notification sound constituting the long notification sound, it is possible to clearly notify the user that the multiple exposure period has ended. .

  Similarly to the second and third embodiments, when the existing notification sound is adjusted to output the short notification sound and the end notification sound, it is not necessary to provide a notification sound dedicated to the multiple exposure imaging operation. Even if the notification period is variable, it can be easily handled.

  An existing notification sound may be used as the end notification sound. Further, the short notification sound may be shortened by thinning out the existing notification sound, or only a part (for example, the first half) of the existing notification sound may be output. Moreover, short notification sounds may be output continuously or may be output at equal intervals.

  Further, as long as the end notification sound and the end of the multiple exposure period overlap, the timing for outputting the end notification sound may be different from the case shown in FIG. Moreover, it is good also as a structure which does not output an end notification sound, and it is good also as a structure which outputs an end notification sound in another Example.

  Moreover, it may be configured so that the end of the long notification sound and the start of the end notification sound are continuous, and the user may be notified of the long notification sound and the end notification sound as a series of notification sounds. In addition, the long notification sound and the end notification sound may be configured to be discontinuous, and the user may be notified as separate notification sounds.

  Moreover, you may combine this example and 1st Example. That is, the notification sound output in the first embodiment may be the long notification sound of this example. With this configuration, the effect of the first embodiment can be further enhanced. That is, it is possible to further suppress the image pickup apparatus from being greatly moved. Further, the end notification sound of this example may be output during the same period as the notification sound of FIG. 5 shown for the first embodiment.

[Fifth embodiment]
A fifth embodiment of the notification operation when performing the multiple exposure imaging operation will be described with reference to the drawings. FIG. 9 is a timing chart showing the fifth embodiment of the operation of the notification unit when performing the multiple exposure imaging operation, and FIG. 4 showing the single exposure imaging operation and the diagrams showing the first to fourth embodiments. 5 to FIG. In addition, the same name is attached | subjected about the part similar to FIG.4 and FIGS.5-8, and the detailed description is abbreviate | omitted.

  In this example, notification is performed by displaying a preview image during a period similar to the period in which the long notification sound of the second to fourth embodiments is output, and thereafter, black screen display is performed. In the example shown in FIG. 9, the preview image is displayed from the end of the first exposure period to the end of the nth exposure period. And since it is the same as that of 2nd-4th Example about the point which performs alerting | reporting for a long period, the effect acquired by performing alerting | reporting for a long period is also acquired in this example like 2nd-4th Example. Can do.

  Further, as in the fourth embodiment, a black screen display different from the preview image display is performed at the end of the multiple exposure period. Therefore, the effect obtained by performing the notification operation different from the notification operation performed for a long period at the end of the multiple exposure period can be obtained in this example as well as the fourth embodiment.

  If the preview image display is performed for a long period as in this example, it is possible to appeal to the user's vision that it is a multiple exposure period. Therefore, it is possible to clearly notify the user that it is the multiple exposure period.

  In addition to the preview image (or in place of the preview image), characters and marks such as “during multiple exposure period” may be displayed on a display device or the like. If the display is performed in this way, it becomes possible to clearly notify the user. The same applies to the period during which black screen display is performed, and characters and marks such as “end of multiple exposure period” may be displayed for a predetermined period.

  In addition, although it is possible to perform notification only in this example, the notification operation may be performed by a plurality of different methods by combining this example and the first to fourth examples. And it becomes possible to alert | report more strongly with respect to a user by combining and performing several different alerting | reporting operation | movement.

[Modification]
In the first to fifth embodiments described above, the notification unit performs notification by image display and sound output. However, in addition to (or instead of) these methods, other methods may be used. It does not matter as a notification. For example, notification may be performed by blinking or turning on an LED lamp for notification. In this case, for example, notification may be performed by using a plurality of LEDs or controlling a blinking cycle. However, the notification operation is performed during the same period as the notification operation shown in the first to fifth embodiments.

  Further, the mechanical shutter needs to be open during the exposure period of the image sensor, and is preferably closed during the reading period in order to suppress deterioration of image data during reading. Therefore, it does not matter whether it is open or closed during other periods. Moreover, it is good also as a structure which is not equipped with a mechanical shutter, and it does not matter even if it always opens during a multiple exposure period.

  In addition, the present invention can be applied to any electronic device as long as it can create still image data. For example, the present invention may be applied to a mobile phone or a PDA (Personal Digital Assistant).

  Moreover, about the imaging device 1 of FIG. 1 shown about embodiment of this invention, you may perform operation | movement of the alerting | reporting part 15 by control apparatuses, such as a microcomputer and CPU16. Further, all or part of the functions realized by these control devices are described as a program, and the program is executed on a program execution device (for example, a computer) to realize all or part of the functions. It doesn't matter.

  The imaging apparatus 1 in FIG. 1 is not limited to the above-described case, and can be realized by hardware or a combination of hardware and software. Further, when the imaging apparatus 1 is configured using software, a block diagram of a part realized by software represents a functional block diagram of the part.

  As mentioned above, although embodiment in this invention was described, the range of this invention is not limited to this, It can add and implement various changes in the range which does not deviate from the main point of invention.

  The present invention relates to an imaging apparatus capable of performing a multiple exposure imaging operation in which an exposure operation is performed a plurality of times to create one image data, and an imaging method thereof. In particular, the present invention is preferably applied to an imaging apparatus and an imaging method capable of executing addition-type image stabilization that combines a plurality of image data to create one image data.

These are block diagrams which show the structure of the imaging device in embodiment of this invention. FIG. 6 is a schematic diagram of image data on which an addition-type image stabilization process is performed. These are the timing charts which show about the alerting | reporting operation example of the alerting | reporting part in the case of performing single exposure imaging operation. These are timing charts which show operation | movement of the alerting | reporting part in the case of performing single exposure imaging operation. These are the timing charts shown about 1st Example of operation | movement of the alerting | reporting part in the case of performing multiple exposure imaging operation. These are the timing charts shown about 2nd Example of operation | movement of the alerting | reporting part in the case of performing multiple exposure imaging operation. These are the timing charts shown about 3rd Example of operation | movement of the alerting | reporting part in the case of performing multiple exposure imaging operation. These are the timing charts shown about 4th Example of operation | movement of the alerting | reporting part in the case of performing multiple exposure imaging operation. These are the timing charts shown about 5th Example of operation | movement of the alerting | reporting part in the case of performing multiple exposure imaging operation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Imaging part 3 Image sensor 4 Lens part 5 AFE
6 Microphone 7 Image processing unit 8 Audio processing unit 9 Compression processing unit 10 External memory 11 Driver unit 12 Decompression processing unit 13 Image output circuit unit 14 Audio output circuit unit 15 Notification unit 16 CPU
17 Memory 18 Operation section 19 TG section 20 Bus 21 Bus 30, 31 Image data 30a, 31a Characteristic area

Claims (9)

In an imaging apparatus including an imaging unit that performs an exposure operation for creating image data based on an input optical image,
An image processing unit that creates one image data based on a plurality of image data created by the imaging unit;
A notification unit that performs a notification operation when the plurality of image data is created by the imaging unit;
When the imaging unit creates the plurality of image data by performing the exposure operation a plurality of times,
During the multiple exposure period, which is a period from the start of the exposure operation performed first among the exposure operations performed a plurality of times to the end of the exposure operation performed last.
The imaging device, wherein the notification unit starts the notification operation at least once.   The imaging apparatus according to claim 1, wherein the notification unit performs, as one of the notification operations, a long notification operation performed in a period including at least two periods in which the exposure operation is performed.   The notification unit performs the long notification operation by outputting voice and performing the long notification operation, and extending and outputting at least a part of a notification sound shorter than a period during which the long notification operation is performed. The imaging apparatus according to claim 2.   The notification unit performs the long notification operation by outputting sound and performing the long notification operation, and by echoing and outputting a notification sound shorter than a period during which the long notification operation is performed. The imaging apparatus according to claim 2.   The notification unit performs the long notification operation by repeatedly outputting at least a part of a notification sound shorter than a period during which the long notification operation is performed, while performing the long notification operation by outputting sound. The imaging apparatus according to claim 2, characterized in that:   The imaging apparatus according to claim 2, wherein the notification unit performs the long notification operation by displaying predetermined information.   The information section displays an image based on the image data created by the imaging section as the predetermined information immediately before the image data is created by the imaging section. 6. The imaging device according to 6.   The imaging according to any one of claims 1 to 7, wherein the notification unit performs, as one of the notification operations, an end notification operation performed during a period including the end of the multiple exposure period. apparatus. A first step of creating a plurality of image data by performing an exposure operation for creating image data based on an input optical image a plurality of times;
A second step of creating one image data based on the plurality of image data created by the first step,
During the multiple exposure period which is a period from the start of the exposure operation performed first among the exposure operations performed a plurality of times in the first step to the end of the exposure operation performed last.
An imaging method characterized by starting a notification operation at least once.

Images