JP4686402B2 - Camera, playback device, and playback control method - Google Patents

Description

  The present invention relates to a voice recording / playback technique for a camera, and more particularly to a voice recording or voice playback technique for a camera capable of simultaneously shooting a plurality of moving images.

  A conventional video camera shooting method will be briefly described. FIG. 16 is a diagram for explaining a conventional photographing method. First, FIG. 16A is a view of a state in which a conventional video camera is set as seen from the back. FIG. 16B is a simple flowchart showing an operation procedure. The user aligns the camera with the subject (step A1). The camera zoom is adjusted in accordance with the size of the subject (step A2). Then, shooting is started by a shooting start instruction (step A3). Waiting for the shooting end instruction, the shooting ends (step A4). FIG. 16C shows an example of a captured image. For example, during shooting of an image such as C-1, it is often performed that framing is changed as in C-2 or zooming as in C-3. In many cases, this frequent operation results in a shake state due to the movement of the user's hand. In addition, such an image may be uncomfortable during viewing. In addition, due to this frequent operation, operation sounds such as zoom sounds may be annoying, and if zooming control is missed, there is a difficulty that it will immediately lead to failure.

  In order to solve such a problem, there has been proposed an imaging apparatus that simultaneously captures two images having different angles of view, regardless of optical means (Patent Document 1). This imaging device is a device that reads out signals from different areas of the imaging device and records them as moving image signals.

On the other hand, the audio played during video playback also plays an important role. Therefore, there is also a proposal for a video camera that can improve sound collecting performance under shooting conditions in which an audio subject moves within a shooting frame (Patent Document 2). In Patent Document 2, a gain control unit as a microphone directivity control unit automatically controls the gain of a stereo microphone line based on an output of a motion detection unit that calculates image data and detects a moving direction of a subject. It is described that the direction of sound collection directivity is corrected.
JP 2006-42399 A JP 2000-4493 A

  A camera that simultaneously captures two screens as proposed in Patent Document 1 has great convenience. In order to further enhance the effect of the two-screen shooting, sound recorded simultaneously with the image is also important. However, since the conventional audio recording method is constant regardless of the image, the output audio is the same even if the entire screen and the up screen are switched during reproduction. For example, in the method of recording audio according to the position of the moving subject as in Patent Document 2, the reproduced audio is the same even when the screen is switched.

It is an object of the present invention to provide a playback apparatus that performs audio playback corresponding to playback of a plurality of moving images, a playback control method thereof, and the like.

In order to achieve the above object, the playback device according to the first aspect of the present invention specifies , as a moving image captured by one camera, a full-area image where the entire subject is imaged and an image of a partial area of the overall image. An image reproduction unit for switching whether to display the whole area image larger than the designated part image or to display the designated part image larger than the whole area image at the time of multi-display for simultaneously displaying two moving images of the partial image; An audio playback unit that plays back an audio signal recorded on at least two channels in response to shooting of a moving image, and a directivity correction that corrects the directivity of the audio signal to be played back according to the designated portion image displayed above and parts, with the audio reproduction unit, when the designation unit image is displayed larger than the entire image by the image reproduction unit, an audio signal corrected by the directional correction unit Raw and, when the entire image is displayed larger than the specified unit image by the image reproduction unit is for reproducing an audio signal that is not the correction by the directional correction unit.

According to a second aspect of the present invention, there is provided a camera according to the first aspect of the present invention, a moving image recording unit that simultaneously records the two moving images of the whole area image and the designated portion image, and a recording direction of the designated portion image. A direction recording unit and an audio signal input unit for inputting audio signals of at least two channels, and the directivity correction unit corrects the directivity according to the shooting direction.

The reproduction control method according to the third invention, an image reproduction unit for displaying a moving image, and audio reproduction unit for reproducing audio signals recorded in at least two channels corresponding to the shooting of the moving image, the In a playback control method for a camera including a directivity correction unit that corrects the directivity of an audio signal to be played back, as a moving image shot by one camera, a whole area image obtained by shooting the entire subject and one of the whole area images the multi-display time for displaying two video images of the specified portion image is a region of the part at the same time, switches or the or the designated portion image the entire image to display larger than the designated unit images displayed larger than the entire image When the image reproduction unit is controlled so that the designated image is displayed larger than the entire area image by the image reproduction unit, the directivity correction unit corrects the image. Play audio signal, when displaying the entire image larger than the specified unit images by the image reproducing unit, controls the audio reproducing unit to reproduce the audio signal that is not the correction by the directional correction unit To do.

The present invention can provide a playback apparatus that performs audio playback corresponding to playback of a plurality of moving images, a playback control method thereof, and the like.

The best mode for carrying out the present invention will be described below with reference to the drawings.
The embodiment will be described with reference to FIGS. FIG. 1 is an overall block diagram of a camera 1 to which the present invention is applied. The camera 1 is provided with a photographic lens 2, an image sensor 3, an AFE (analog front end) 4, a selection unit 5, an image processing unit 8, and a recording medium 20. The taking lens 2 forms an image of the incident subject 25 on the image sensor 3. The image sensor 3 is composed of a CCD or a CMOS, and converts the formed subject image into an electrical signal. The AFE 4 reads an image signal from the image sensor 3 and outputs the read image signal as digital image data by performing AGC processing, CDS processing, AD conversion, or the like. The AFE 4 also controls driving of the image sensor 3. The selection unit 5 selects and outputs the image data from the AFE 4 to the two subsequent image processing units according to the type.

  The image processing unit 8 performs various processing on the image data and records it on a recording medium. The image processing unit 8 includes a first image processing unit 6a, a second image processing unit 6b, a first recording unit 7a, and a second recording unit 7b. The selection unit 5 outputs a signal of an area of the entire screen, which is image data thinned out according to a predetermined rule, to the first image processing unit 6a. The selection unit 5 outputs image data limited to a predetermined area to the second image processing unit 6b. The first image processing unit 6a and the second image processing unit 6b are provided with a compression unit, an expansion unit, and the like, and perform processing such as color correction, signal compression, and expansion on the image data. The compression unit performs, for example, JPEG (Joint Photographic Expert Group) compression or MPEG (Motion Picture Experts Group) compression for recording image data on a recording medium. The decompressing unit performs a process of restoring the compressed image data.

  That is, an image of the entire shooting area (hereinafter referred to as an entire image or an entire image) is input to the first image processing unit 6a. The second image processing unit 6b receives a part of the image of the photographing region (hereinafter referred to as a designated part image or a part image). The first recording unit 7a records the entire image processed by the first image processing unit 6a on the recording medium 20. Similarly, the second recording unit 7b records the image of the portion subjected to the image processing by the second image processing unit 6b on the recording medium 20.

  The camera 1 is provided with a display control unit 9, a display unit 12, and a touch panel 13. The display control unit 9 performs control for displaying the image processed by the first image processing unit 6 a and the second image processing unit 6 b on the display unit 12. The display unit 12 is composed of, for example, an LCD, and displays a monitor image at the time of shooting, and displays a decompressed recorded image at the time of reproduction. The touch panel 13 is mounted on the surface of the display unit 12 and directly detects an operator's touch operation on the screen of the display unit 12.

  Further, the camera 1 is provided with an AF unit 14, a microphone unit 15, an ADC unit 21, an audio recording / reproducing unit 17, a right speaker 18R, and a left speaker 18L. The AF unit 14 includes an actuator and a position encoder, and controls the movement of the focus lens in the photographing lens 2. The AF unit 14 detects the in-focus position based on the contrast value of the image from the image processing unit 8 and moves the focus lens.

  The microphone unit 15 is an audio signal input unit that inputs an audio signal, and is used to detect and record audio during shooting. The microphone unit 15 is provided with a right microphone 15R and a left microphone 15L in the microphone unit 15, and can record while changing the balance depending on the position of the subject. The ADC unit 21 digitizes and outputs analog signals from the right microphone 15R and the left microphone 15L.

The audio recording / playback unit 17 compresses the audio signal digitized by the ADC unit 21 and records it on the recording medium 20. Then, the audio recording / reproducing unit 17 reads and expands the audio signal from the recording medium 20 at the time of reproduction, and converts it into an analog signal. The voice recording / reproducing unit 17 is further provided with a directivity correcting unit 17a. The directivity correction unit 17a corrects the sound output from the speaker so as to have directivity that matches the captured image (imaging direction θ). This is done at the time of either voice recording or voice playback. Further, during multi-screen shooting (shooting the entire area image and the designated portion image), the directivity correcting portion 17a corrects the directivity only on the designated portion image. The speakers 18R and 18L output the audio signal reproduced by the recording / reproducing unit 17 as audio.
Details of the directivity correction unit 17a will be described with reference to FIGS.

  Further, the camera 1 is provided with an MPU 10, a ROM 11, an operation unit 16, a time unit 22, and a direction recording unit 19. The MPU 10 is a control unit that controls the entire camera 1 such as shooting and reproduction according to a program. The selection unit 5 and the touch panel 13 are also controlled by the MPU 10. The ROM 11 is a non-volatile and recordable memory such as a flash ROM, and stores a control program for performing camera processing. The operation unit 16 notifies the MPU 10 of the photographer's instruction. Switches 16 a and 16 b are provided as a representative example of the operation unit 16. The switch 16a is for instructing the designated portion image capturing. The designated part image will be described later. The switch 16b is for release instruction. The time unit 22 is a time generation unit for attaching a time stamp to an image or sound during recording. Based on the accompanying time data, each image and sound are reproduced in synchronization during reproduction. The direction recording unit 19 records the switching result of the shooting direction θ. The recording destination is, for example, the ROM 11. The shooting direction will be described with reference to FIG.

  The camera 1 is a camera of a type that receives the subject image 25 with the imaging element 3 via the photographing lens 2 and converts it into an electrical signal and records it. The imaging element 3 used here is, for example, 10 million. It is more desirable that the pixel is a high pixel of the pixel class and a CMOS sensor or the like. That is, it is desirable that the imaging device 3 used here adopts the latest technology that enables high-speed all-pixel readout from 60 frames to 100 frames per second.

  On the other hand, with such a high-pixel and high-speed readout sensor, there is currently no display device that uses all the pixels during reproduction. For example, even a so-called high-definition TV (HDTV) has a resolution equivalent to 1 million pixels. In other words, shooting using all the pixels (10 million pixels) becomes over-spec when considering reproduction. Therefore, in the present invention, an arbitrary part is read and only a necessary part is read and recorded. Such reading out of a predetermined area, thinning out, addition of pixel signals, and the like are also good at the CMOS sensor. Therefore, in the present invention, such a function is effectively used.

  The video camera according to the present invention makes effective use of a larger amount of information by using the latest high-definition high-speed image sensor as described above, reducing the number of movable parts by conventional optical zoom. Thus, it is possible to capture a smooth moving image with a changing motion.

  FIG. 2 shows an example of an area frame changing operation at the time of photographing using the camera 1 of the present invention. FIG. 2A is a view of the camera 1 held by the user's hand as seen from the back. A display surface of the display unit 12 is disposed on the back surface of the camera 1. On the screen of the display unit 12, a square area frame 12a is displayed so as to be superimposed on the monitor image. The area frame 12a is a frame that is displayed to indicate the position and size of the designated portion image that is partially captured. The user holds the switch 16a and the switch 16b while operating with the right hand, and looks at the display unit 12 to determine the composition and shooting timing. While the switch 16b (release) is being pressed, the entire area image displayed on the display unit 12 is recorded in the memory by the first image processing unit 6a and the recording unit 7a. When the position and size of the area frame 12a are designated with the left hand, the touch panel detects it and the designated portion image in the area frame 12a is recorded by the second image processing unit 6b and the recording unit 7b.

  Here, there are various ways of determining and changing the area frame 12a. As an example, as shown in FIG. 2B, several frame patterns 12a-1 are displayed in advance, and depending on which one is touched. An area frame may be selected. Alternatively, the size may be changed according to the touched position and the number of times. Further, in order not to be recorded accidentally even if the touched position is wrong, the portion (designated portion) may not be recorded unless the switch 16a is pressed.

  FIG. 3A is a diagram showing a comparison between an imaging area captured on the entire area and an imaging area captured on the designated portion. The entire imaging area 3c is an area where the entire area is captured. A predetermined designated portion imaging region 3d of all the imaging regions 3c is a region where the designated portion is imaged by the image processing unit 6b. That is, if only the designated portion imaging region 3d is recorded, the same effect as that obtained by electronic zooming is obtained. FIG. 3B is a diagram of a pixel array schematically showing pixels to be thinned out and read. Of all the pixels constituting the entire imaging region 3c, for example, only the hatched pixel 3e is thinned out and read out at the time of whole area photographing. At this time, the thinning rate is automatically set so that the resolution at the time of reproduction is sufficient. That is, if it is compatible with HDTV (High Definition Television), the thinning rate is set so as to be about 1 million pixels.

  FIG. 4 is a flowchart showing the procedure of the two-screen shooting process in the camera 1. This process is mainly executed by the MPU 10 according to a program. Assume that the camera 1 is set to the shooting mode. Then, it is determined whether or not the user has operated the switch 16b (step S11). When the switch 16b is operated (YES in step S11), first, whole area shooting and recording are started (step S12). The recording performed here is normal recording, and the directivity correction processing described later is not performed.

  Next, it is determined whether or not the user has operated the touch panel 13 (step S13). If there is no operation on the touch panel (step S13 NO), it is next determined whether or not the switch 16a is operated (step S14). The switch 16a is a switch for instructing photographing of the designated portion image. If the user operates the switch 16a (step S14 YES), it is next determined whether the area frame 12a is displayed (step S15). If the area frame 12a is designated (YES in step S15), designated part photographing for photographing the range designated by the frame is started (step S16).

  Dedicated recording is performed at the same time as the designated part shooting starts. In recording at the time of photographing the designated portion, a sound corresponding to the area frame 12a is recorded (step S17). Details of the area frame-corresponding voice recording will be described later with reference to a flowchart (subroutine) in FIG. Here, since the whole area shooting is continuously performed, two moving image shootings are simultaneously performed. As for audio recording, it is desirable to simultaneously perform the area frame compatible audio recording and normal audio recording in which LR is recorded in stereo as it is. This is because the sound can be switched according to the screen switching during reproduction. Of course, it is possible to use only the in-frame compatible audio recording.

  If the area frame 12a is not selected, a warning is displayed (step S23), and the process returns to step S11. On the other hand, if there is an operation on the touch panel (step S13 YES), the area frame 12a is changed. First, when the touch-operated position is deviated from the center of the screen, the deviation is calculated. The angle between the center of the biased designated portion image and the center of the screen is recorded as the shooting direction θ (step S19). This is performed by the direction recording unit 19. Subsequently, the area frame 12a is displayed around the touched place (step S20). Subsequently, it is determined whether the touch has been made a plurality of times (step S21). If touched a plurality of times (YES in step S21), the size of the area frame 12a is changed according to the number of touches (step S22). For example, it is made smaller according to the number of times.

  FIG. 5 shows a screen of a variation example of the area frame 12a. FIG. 5 is a diagram illustrating an example of a change in the size of the frame area 12a displayed on the display unit 12 by a touch operation. FIG. 5A shows the size of the first area frame 12a and the touch operation on the area frame 12a. Then, the area frame 12a is gradually reduced as shown in FIGS. 5A to 5B by a plurality of touch operations.

  Returning to FIG. On the other hand, if it is determined that the operation of the switch 16b has been released during shooting (NO in step S11), the shooting of both the entire area and the designated portion is completed (step S18), and the system waits for the operation of the switch 16b (shooting restart). In addition, after step S21 and step S22, the process proceeds to step S11, and photographing is continued unless an end instruction is given. Through the above flowchart, the entire area image and the designated portion image in the range designated at the timing designated by the user are recorded.

    Next, a modification of the above operation is shown in FIG. FIG. 6 is a flowchart for explaining a modified example of the procedure of the photographing process explained in FIG. In the flowchart of FIG. 4, when the switch 16b (release) is pressed, the entire area shooting is always performed. However, it is also convenient to allow only the designated portion to be shot without shooting the entire area. Therefore, in this modification, the entire area shooting and the specified area shooting are performed independently.

  First, it is determined whether or not the switch 16b is operated (step S31). If the switch 16b is operated (YES in step S31), the whole area shooting is continued if the whole area shooting is being performed, and the whole area shooting is started if the entire area shooting is not being performed (step S32). On the other hand, if the switch 16b is not operated (NO in step S31), the entire area shooting is not performed or is ended (step S33), and the process proceeds to step S34. Subsequently, it is determined whether or not the switch 16a is operated (step S34). If the switch 16a has been operated (step S34 YES), the designated portion photographing for photographing the portion designated by the area frame 12a is started or continued (step S35). On the other hand, if the switch 16b is not operated (NO in step S34), the designated portion photographing is not performed or is ended (step S36). Then, the process returns to step S31. As a result, it is possible to arbitrarily perform both or any one of the whole area photographing and the designated portion photographing. It also leads to saving of recording media.

  FIG. 7 is a schematic view of the camera 1 as viewed from above in order to show the arrangement of microphones and the like. The left side is the subject side. A photographing lens 2 is provided at the front center of the camera 1 body. The optical axis Z is the optical axis of the taking lens 2 and becomes the center of the screen. A right microphone 15R is arranged on the right side (upper side in the figure) of the optical axis Z of the camera 1 body, and a left microphone 16L is arranged on the left side (lower side in the figure). A speaker 18RL is disposed on the back of the camera 1. As described above, the direction of the designated portion image with respect to the optical axis Z is indicated by the shooting direction θ. Here, the sign of the shooting direction θ is + (plus) when the designated portion image is on the right side when viewed from the photographer, and − (minus) when it is on the left side.

  The audio recording corresponding to the area frame described in step S17 in FIG. 4 will be described. FIG. 8 is a flowchart for explaining a subroutine for recording audio for area frames. This process is mainly performed by the audio recording / reproducing unit 17 and the directivity correcting unit 17a in accordance with an instruction from the MPU 10. First, the photographing direction θ is detected from the result recorded in advance in step S19 of FIG. 4 (step S41). The directivity correction unit 17a performs the directivity correction process with respect to the shooting direction θ on the audio signal R input from the right microphone 15R and the audio signal L input from the left microphone 15L (step S42). 9, 10, 11, and 12 are block diagrams respectively illustrating the four types of directivity correction units 17 a that perform directivity correction (step S <b> 42). Hereinafter, in FIGS. 9, 10, 11, and 12, the designated portion image is located on the left side and the L signal is emphasized.

  First, FIG. 9 shows a main block diagram for explaining the first type directivity correction unit 17a. The first type directivity correction unit 17a is a method of emphasizing and recording during monaural recording. The speaker from which sound is output is a monaural speaker 18M. The directivity correction unit 17a performs enhancement processing on the L signal and records it on the recording medium 20.

  The directivity correction unit 17a includes an adder 1, an adder 2, an adder 3, an adder 4, and a multiplier. The audio signals detected by the right microphone 15R and the left microphone 15L are A / D converted by the ADC 21 and input to the directivity correction unit 17a as the R signal and the L signal. The R signal and the L signal are calculated by the adder 1 as (R−L). Then, the difference between (R−L) is weighted according to the imaging direction θ detected by the multiplier, and changes in R and L are more dynamically emphasized. Amplification in the multiplier (the number of additions) is controlled to be changed by the MPU 10 in accordance with the shooting direction θ. The R signal is further added by the adder 2 and the L signal is added by the adder 3. The calculated L and R signals are added by the adder 4, and the LR mix signal is recorded on the recording medium 20. Thereby, the L signal is further emphasized and recorded.

  At the time of reproduction, the audio signal emphasized and recorded above is read from the recording medium 20, converted into an analog signal by a DAC (digital-to-analog converter) in the audio recording / reproducing unit 17, and output from the monaural speaker 18M.

  The width of the camera is limited due to its portability, and when the distance between the right microphone 15R and the left microphone 15L cannot be increased, the difference between the left and right sounds is small. Even in such a case, the difference between left and right can be further emphasized by taking the signal of the difference between R and L as described above. For example, if the absolute value of θ is increased, (R−L) amplification is performed, and then the addition result of the adder 4 is output to the monaural speaker 18M. In the above description, the configuration in the case where the L side is emphasized has been described. However, when the designated portion image is on the right side, the polarity of the RL signal input to the adder 1 in FIG. It is sufficient to switch to the circuit to be used.

  Moreover, although the above block diagram is a block diagram of audio recording corresponding to the designated portion image, it is also possible to record the audio corresponding to the designated portion image and the audio corresponding to the entire image in separate files. This can be realized by executing the audio recording process for the entire image in parallel.

  FIG. 10 is a main block diagram for explaining the second type directivity correction unit 17a. The second type directivity correction unit 17a is a method that performs output with voice enhancement during reproduction. The directivity correction unit 17a outputs the L signal after enhancement processing during reproduction. The directivity correction unit 17a includes an adder 1, an adder 2, an adder 3, an adder 4, and a multiplier. The directivity correcting unit 17 a is provided in the audio recording / reproducing unit 17. The speaker from which sound is output is the monaural speaker 18M.

  Audio signals from the right microphone 15R and the left microphone 15L are recorded on the recording medium 20 as they are without directivity correction. One of the recorded LR signals is recorded as the other additional data. Further, it is assumed that the shooting direction θ is also recorded on the recording medium 20 in association with the corresponding image by the direction recording unit 19 at the time of shooting.

  Then, in the same manner as described in the above block diagram 10, the adder 1, the adder 2, the adder 3, the adder 4, and the multiplier are switched according to the shooting direction θ. The mixed audio signal is converted into an analog signal by the DAC in the audio recording / reproducing unit 17 and output from the monaural speaker 18M.

  In any of the first and second type directivity correction units 17a, the difference in sound from each direction can be emphasized by the difference in sound between the left and right, so that stereo recording can be performed with a small device such as a camera. Even when performing, it is possible to reproduce sound having directivity characteristics according to the shooting direction. It should be noted that the above-described voice enhancement reproduction process is not limited to being mounted on the camera 1 but may be naturally mounted on an image reproduction apparatus that is not equipped with a photographing system.

  FIG. 11 is a main block diagram for explaining the third type directivity correction unit 17a. The third type directivity correction unit 17a is applied to a camera equipped with a stereo speaker, and is a method of performing emphasis recording during stereo recording. In the above two examples, the example of monaural recording and monaural speaker has been described. However, with the configuration shown in FIG. 11, it is possible to correspond to stereo recording and stereo speakers 18R and 18L. The outputs of the right microphone 15R and the left microphone 15L are each AD converted by the ADC 21, subjected to directivity correction processing by the directivity correction unit 17 a in the audio recording / playback unit 17, and recorded on the recording medium 20. At the time of reproduction, the emphasized R signal and L signal are read from the recording medium 20, converted into analog signals by the DAC in the audio recording / reproducing unit 17, and output from the stereo speakers 18R, 18l.

  An adder 1, an adder 2, an adder 3, and a multiplier are provided in the directivity correcting unit 17a in the audio recording / reproducing unit 17. The R signal and the L signal are calculated by the adder 1 as (R−L). Then, the difference of (R−L) is weighted by the multiplier, and the change between R and L is more dynamically emphasized. Amplification in the multiplier (the number of additions) is controlled to be changed by the MPU 10 in accordance with the shooting direction θ. The adder 2 further adds the R signal, and the adder 3 adds the L signal. The signal output from the adder 2 is recorded on the recording medium 20 as a new R signal. Further, the signal output from the adder 2 is recorded on the recording medium 20 as a new L signal.

  FIG. 12 is a main block diagram for explaining the fourth type directivity correction unit 17a. The fourth type directivity correction unit 17a is applied to a camera equipped with a stereo speaker, and is a method of performing audio-emphasized output during reproduction. The outputs of the right microphone 15R and the left microphone 15L are each AD-converted by the ADC 21 and recorded on the recording medium 20 as LR separate audio data. At the time of reproduction, the LR audio data recorded on the recording medium 20 is read out. The audio data is subjected to directivity correction processing by the directivity correction unit 17a in the audio recording / reproducing unit 17 and is output from the right speaker 18R and the left speaker 18L.

  An adder 1, an adder 2, an adder 3, and a multiplier are provided in the directivity correcting unit 17a in the audio recording / reproducing unit 17. The R signal and the L signal are calculated by the adder 1 as (R−L). Then, the difference of (R−L) is weighted by the multiplier, and the change between R and L is more dynamically emphasized. Amplification in the multiplier (the number of additions) is controlled to be changed by the MPU 10 in accordance with the shooting direction θ. The R signal is further added by the adder 2 and the L signal is added by the adder 3. The signal output from the adder 2 is D / A converted as a new R signal and output from the right speaker 18R. Further, the signal output from the adder 2 is D / A converted as a new L signal and output from the left speaker 18L. In this type, the sound is played back in stereo, so you can enjoy more realistic sound. It should be noted that the above-described voice enhancement reproduction process is not limited to being mounted on the camera 1 but may be naturally mounted on an image reproduction apparatus that is not equipped with a photographing system.

  Next, the sound effect during reproduction display on the display unit 12 will be described with reference to FIGS. Enhanced recording or enhanced reproduction that produces this effect is realized by the circuits described with reference to FIGS. First, FIG. 13 is an example at the time of so-called multi-display described in the embodiment in which both the entire area image and the designated portion image are displayed simultaneously. FIG. 13A is a full-area image screen, and FIG. 13B is a designated portion image screen. (A) and (B) are switched at any time and reproduction is performed. When the whole area image of (A) is displayed, the RL sound is reproduced evenly. On the other hand, when the designated portion image (B) is displayed, the right person is reproduced in close-up, so that the right sound (R) is emphasized and reproduced.

  Here, the directivity correction processing for the sound of the designated portion image may be performed at the time of recording as in the first or third type, or may be performed at the time of reproduction as in the second or fourth type. . In addition, when the recording is performed at the time of recording, it is assumed that two audio recordings, that is, area frame compatible audio recording and normal recording are recorded. When performed at the time of reproduction, only the normally recorded stereo sound is required. As described above, since the sound is switched in conjunction with the screen switching, the scene switching is performed more realistically.

  The following example is an example in which a plurality of captured images are all designated portion images. In the embodiment, the entire area image and the designated portion image have been described as the two moving images captured simultaneously. However, it is possible to use two designated portion images as images that are simultaneously shot. In this example, two designated-portion images (partial images) photographed at the same time are reproduced.

  FIG. 14 shows the entire screen imaged. Assume that two images surrounded by a broken line are simultaneously recorded on a subject as shown in FIG. The broken line area in the upper right of FIG. 14 is the image reproduced in FIG. The broken line area in the lower left of FIG. 14 is an image reproduced in FIG. That is, different positions on the screen are photographed at the same time, and are switched during reproduction as shown in FIGS. 14 (A) and 14 (B). When the right image as shown in FIG. 14A is reproduced, the R sound that is the right sound is emphasized and reproduced. Further, as shown in FIG. 5B, when the left image is reproduced, the L sound that is the left sound is emphasized and reproduced.

  Here, the directivity correction processing for the sound of the designated portion image may be performed at the time of recording as in the first or third type, or may be performed at the time of reproduction as in the second or fourth type. . In addition, when the recording is performed at the time of recording, two area frame-corresponding voice recording data are recorded as voice recording. At the time of reproduction, only the normally recorded stereo sound is required. As described above, since the sound is switched in conjunction with the switching of the screen, the scene can be switched more realistically.

  As described above, it is possible to perform a realistic and dramatic reproduction effect by performing such audio reproduction. Further, enhancement during reproduction may be performed by zooming.

  Note that part or all of the processing of the MPU 10 described in the above embodiment may be configured by hardware. Further, the directivity correction unit 17a may be configured by software. The specific configuration is a design matter. Each control process by the MPU 10 is realized by a software program stored in the ROM 11 being supplied to the MPU and operating according to the supplied program. Therefore, the software program itself realizes the function of the MPU, and the program itself constitutes the present invention. A recording medium for storing the program also constitutes the present invention. As a recording medium, besides a flash memory, an optical recording medium such as a CD-ROM or DVD, a magnetic recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Similarly, the audio recording / playback unit 17 may be entirely or partially configured by software. In the embodiment, an example in which the present invention is applied to a digital camera has been described. However, the present invention is not limited thereto, and may be applied to, for example, a camera unit of a mobile phone.

  Furthermore, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

1 is an overall block diagram of a digital camera 1 to which the present invention is applied in an embodiment. The figure which shows the example of change operation of the area frame at the time of imaging | photography in embodiment. The figure which contrasted and showed the imaging area imaged whole area | region and the imaging area image | photographed partially in embodiment. 4 is a flowchart illustrating a two-screen shooting process of the camera 1 in the embodiment. The figure which shows the example of a change of the magnitude | size of the frame area displayed on a display part by touch operation in embodiment. 6 is a flowchart illustrating a two-screen shooting deformation process in the embodiment. The top view of the camera for showing arrangement | positioning of a microphone etc. in embodiment. In the embodiment, a subroutine showing the processing of in-frame corresponding voice recording. The block diagram of the circuit applied to a camera equipped with a monaural speaker in the embodiment, in which directivity correction processing is performed during recording. The block diagram of the circuit applied to the camera by which a directivity correction process is performed at the time of reproduction | regeneration, and a camera equipped with a monaural speaker in embodiment. FIG. 5 is a block diagram of a circuit applied to a camera equipped with a stereo speaker in a camera in which directivity correction processing is performed during recording in the embodiment. FIG. 3 is a block diagram of a circuit applied to a camera equipped with a stereo speaker in a camera in which directivity correction processing is performed during reproduction in the embodiment. The figure which shows the reproduction | regeneration screen of a multi image in embodiment. The figure which shows the positional relationship of the whole screen of a to-be-photographed object and two partial images in embodiment. The figure which shows the reproduction | regeneration screen of two partial images in embodiment. The figure explaining the conventional imaging | photography method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Camera, 2 ... Shooting lens, 3 ... Imaging element, 3c ... All imaging area, 3d ... Designated part imaging area,
3e: thinned pixels, 4 ... analog front end (AFE) section,
5 ... selection unit, 6a ... first image processing unit, 6b ... second image processing unit,
7a ... 1st recording part, 7b ... 2nd recording part, 8 ... Image processing unit, 9 ... Display control part 10 ... MPU, 11 ... ROM, 12 ... Display part, 12a ... Area frame, 13 ... Touch panel, 14 ... AF 15, microphone section, 15 R, right microphone, 15 L, left microphone,
DESCRIPTION OF SYMBOLS 16 ... Operation part, 16a, 16b ... Switch, 17 ... Audio | voice recording / reproducing part, 17a ... Directivity correction part 18 ... Speaker part, 18R ... Right speaker, 18L ... Left speaker, 18M ... Monaural speaker,
DESCRIPTION OF SYMBOLS 19 ... Direction recording part, 20 ... Recording medium, 21 ... ADC, 22 ... Time part, 25 ... Subject, Z ... Optical axis

Claims (3)

At the time of multi-display in which two moving images of a whole area image where the entire subject is photographed and a designated part image which is an image of a part of the whole area image are simultaneously displayed as a moving image photographed by one camera, An image reproduction unit for switching whether to display the entire area image larger than the designated area image or to display the designated area image larger than the entire area image;
An audio playback unit for playing back audio signals recorded in at least two channels corresponding to the shooting of the moving image ;
A directivity correction unit that corrects the directivity of the audio signal to be reproduced in accordance with the designated portion image displayed;
The audio reproduction section, when the designation unit image is displayed larger than the entire image by the image reproduction unit reproduces the audio signal corrected by the directional correction unit,
A playback apparatus that plays back an audio signal that has not been corrected by the directivity correction unit when the entire image is displayed larger than the specified image by the image playback unit. A playback device according to claim 1;
A moving image recording unit for simultaneously recording the two moving images of the whole area image and the designated portion image;
A direction recording unit for recording the shooting direction of the designated unit image;
An audio signal input unit for inputting audio signals of at least two channels, wherein the directivity correction unit corrects the directivity according to the shooting direction. An image reproduction unit for displaying a moving image, and audio reproduction unit for reproducing audio signals recorded in at least two channels corresponding to the shooting of the moving image, the directivity of correcting the directivity of the audio signal to be the reproducing In a camera playback control method comprising a correction unit ,
As a moving image captured by one camera, the entire image is displayed at the time of multi-display in which two moving images of a whole area image in which the entire subject is imaged and a designated part image that is a partial area of the whole area image are displayed simultaneously. Controlling the image reproduction unit to switch whether to display larger than the designated part image or to display the designated part image larger than the whole area image ,
In addition, when the image reproduction unit displays the designated part image larger than the whole area image, the audio signal corrected by the directivity correction part is reproduced, and the whole area image is converted to the designated part image by the image reproduction part. The playback control method characterized by controlling the audio reproduction unit to reproduce an audio signal that has not been corrected by the directivity correction unit when displaying larger .

Images