JP5855346B2 - Audio processing apparatus, audio processing method, and program - Google Patents

Description

The present invention relates to an audio processing device, and more particularly, to an audio processing device , an audio processing method, and a program for processing audio data in accordance with image data processing.

  In conventional photography, image processing is performed using an image filter or the like to enhance the emotional atmosphere, and an arbitrary effect (hereinafter referred to as art effect) may be added to the image. In addition, it has been proposed to play music together when playing back captured images. For example, the photographing apparatus disclosed in Patent Document 1 records music data according to image data at the time of photographing. In addition, the photographing apparatus disclosed in Patent Literature 2 determines facial expressions when a person is included in an image, and reproduces music corresponding to the facial expressions together with the image.

JP 2006-261957 A JP 2010-21819 A

  Conventionally, there is a case where an art effect is added and an ambient sound is recorded together. In this case, the ambient sound was recorded so as to be faithful to the original sound. However, if the ambient sound is faithful to the original sound while adding an art effect by image processing, the emotional and entertainment properties during viewing may be reduced. For example, when image processing that makes a retro atmosphere like a toy camera is performed as an art effect, if the ambient sound is beautiful, the art effect is impaired.

The present invention has been made in view of such circumstances, and in recording an image and recording ambient sound, and processing the image, the sound that does not impair the emotion and entertainment during viewing It is an object to provide a processing device , a voice processing method, and a program .

In order to achieve the above object, an audio processing apparatus according to a first invention includes an imaging unit that captures an image, an audio input unit that inputs audio, and a plurality of processings of image data obtained by imaging with the imaging unit. an image processing selecting unit for selecting a processing like processing from the processing, in accordance with the selection result by the image processing selecting unit, and an image processing unit that performs image processing with respect to the image data, the voice The audio processing performed on the audio data input by the input unit is selected from a plurality of audio processes, and the audio processing performed by the audio processing selection unit is selected according to the selection result by the image processing processing selection unit. A sound processing instruction unit for instructing the selection of processing, and sound data output by associating the sound data processed by the sound processing instructed by the sound processing instruction unit with the image data A processing unit; a display unit that displays a live view image captured by the imaging unit; and a display mode selection unit that selects a display mode of the display unit. The display unit is configured to display the display mode in the display mode. When a mode in which a live view image is displayed is selected, a display representing the plurality of selectable processing processes and a display representing the plurality of selectable audio processes are displayed together with the live view image. .

In the voice processing apparatus according to the second invention, in the first invention, the display mode selection unit can further select a recommended mode or a super control panel mode.
Speech processing apparatus according to the third aspect of the invention related to the second invention, when the recommended mode is selected by the display mode selecting unit, in response to the selected processing, audio processing to recommend decide.
In the voice processing apparatus according to the fourth invention, in the second invention, when the super control panel mode is selected by the display mode selection unit, the voice processing that cannot be recommended is not selected.

According to a fifth aspect of the present invention, there is provided a speech processing apparatus according to the first aspect, further comprising a synthesized sound storage unit for storing sound data for synthesis with respect to the input speech data, and the speech processing unit Performs sound processing on the synthesized sound stored in the synthesized sound storage unit according to the instruction.
According to a sixth aspect of the present invention, in the fifth aspect, the voice processing unit includes a filter processing unit that performs a filtering process on the input voice data, and the synthesized sound is converted into the input voice data. Composed of unrelated sound effects.

According to a seventh aspect of the present invention, there is provided a voice processing apparatus according to the first aspect, further comprising a shooting mode selection unit, wherein the voice processing unit performs voice processing according to the shooting mode selected by the shooting mode selection unit. Select.
An audio processing apparatus according to an eighth aspect of the invention is to process an image capturing unit that captures an image, an audio input unit that inputs audio, and image data obtained by capturing with the image capturing unit from among a plurality of processing processes. an image processing selecting unit for selecting a processing, in accordance with the selection result by the image processing selecting unit, and an image processing unit that performs image processing with respect to the image data, voice input by the voice input unit Instructing the selection of audio processing by the audio processing selection unit according to the audio processing selection unit that selects audio processing to be performed on the data from among a plurality of audio processings and the selection result by the image processing processing selection unit Select an audio processing instruction section and a combination of image processing and audio processing to be recorded or reproduced, and image data processed by the selected image processing and audio data processed by audio A control unit that controls to output in association with each other; a display unit that displays a live view image captured by the imaging unit; and a display mode selection unit that selects a display mode of the display unit. When the mode in which the live view image is displayed in the display mode is selected, the unit displays the plurality of selectable processing processes, the display that displays the selectable plurality of audio processes, Are displayed together with the live view image.
According to a ninth aspect of the present invention, there is provided a sound processing method for processing an image capturing step for capturing an image, a sound input step for inputting sound, and image data obtained by capturing the image from the plurality of processing processes. An image processing processing selection step for selecting processing processing, an image processing processing step for performing image processing processing on the image data in accordance with a selection result obtained by the image processing processing selection step, and a voice input in the voice input step Instructing the selection of the audio processing by the audio processing selection step according to the audio processing selection step for selecting the audio processing to be performed on the data from a plurality of audio processing and the selection result by the image processing processing selection step Sound processing instruction step and sound processed by the sound processing instructed in the sound processing instruction step Data, and the audio processing step of outputting in association with the image data, a display step of displaying a live view image captured in the imaging step, a display mode selection step of selecting the display mode in the display step, the And the display step includes a display representing the plurality of selectable processing processes and a plurality of selectable audio processes when the mode in which the live view image is displayed is selected in the display mode. Is displayed together with the live view image.
A program according to a tenth invention includes an imaging step for capturing an image, an audio input step for inputting sound, and a processing process for processing the image data obtained by the imaging step from a plurality of processing processes. An image processing processing selection step for selecting the image data, an image processing processing step for performing image processing processing on the image data in accordance with a selection result obtained by the image processing processing selection step, and voice data input in the voice input step. A voice processing selection step for selecting a voice processing to be performed from a plurality of voice processing, and a voice processing for instructing selection of the voice processing by the voice processing selection step according to a selection result by the image processing processing selection step. Sound processed by the voice processing instructed in the instruction step and the voice processing instruction step Data, and the audio processing step of outputting in association with the image data, a display step of displaying a live view image captured in the imaging step, a display mode selection step of selecting the display mode in the display step, the And the display step includes a display representing the plurality of selectable processing processes and a plurality of selectable audio processes when the mode in which the live view image is displayed is selected in the display mode. And a display for displaying the image together with the live view image.

According to the present invention, it is possible to provide an audio processing device , an audio processing method, and a program that do not impair the emotional and amusement at the time of viewing when an image is recorded and an ambient sound is recorded and the image is processed. can do.

1 is a block diagram mainly showing an electrical configuration of a camera according to an embodiment of the present invention. It is a block diagram which shows the structure mainly related to audio | voice processing of the camera concerning one Embodiment of this invention. FIG. 3 is a block diagram illustrating a flow of data and control during image processing in a camera according to an embodiment of the present invention. FIG. 5 is a block diagram showing a flow of data and control when performing audio processing in addition to image processing in the camera according to the embodiment of the present invention. FIG. 10 is a block diagram showing another example of data and control flow when performing audio processing in addition to image processing in the camera of one embodiment of the present invention. It is a block diagram which shows the detail of a sound process in the camera in one Embodiment of this invention. It is a block diagram which shows the processing content for every sound processing in the camera in one Embodiment of this invention. It is a figure which shows the combination of an art image process and a sound filter in the camera concerning one Embodiment of this invention. 4 is a display screen for performing an interface in a camera according to an embodiment of the present invention. 4 is a display screen for performing an interface in a camera according to an embodiment of the present invention. 4 is a display screen for performing an interface in a camera according to an embodiment of the present invention. 4 is a display screen for performing an interface in a camera according to an embodiment of the present invention. It is a flowchart which shows operation | movement of the camera concerning one Embodiment of this invention. It is a flowchart which shows operation | movement of the camera concerning one Embodiment of this invention.

  Hereinafter, a preferred embodiment will be described using a camera to which the present invention is applied according to the drawings. A camera according to a preferred embodiment of the present invention is a digital camera, includes an imaging unit, converts a subject image into image data by the imaging unit, and converts the subject image into a main body based on the converted image data. Live view is displayed on the display unit placed on the back of the camera. The photographer determines the composition and the photo opportunity by observing the live view display. During the release operation, image data is recorded on the recording medium. The image data recorded on the recording medium can be reproduced and displayed on the display unit when the reproduction mode is selected.

  In addition, the camera according to an embodiment includes a plurality of types of image processing for applying an art effect, and can be appropriately selected by a photographer. In addition, ambient sounds can be recorded at the time of shooting, and voice processing can be performed on the voice input at this time. At this time, sound processing suitable for the art effect selected by the photographer is performed.

  FIG. 1 is a block diagram mainly showing an electrical configuration of a camera according to an embodiment of the present invention. This camera includes a camera body 100 and an interchangeable lens 200 that can be attached to and detached from the camera body 100.

  The interchangeable lens 200 includes a photographing lens 201, a diaphragm 203, a driver 205, a microcomputer 207, and a flash memory 209, and has an interface (hereinafter referred to as I / F) 999 with the camera body 100 described later.

  The photographing lens 201 is an optical lens for forming a subject image, and is configured by a single focus lens or a zoom lens. A diaphragm 203 is disposed behind the optical axis of the photographing lens 201. The diaphragm 203 has a variable aperture, and restricts the subject light flux that has passed through the photographing lens 201. The photographing lens 201 can be moved in the optical axis direction by a driver 205, and the focus position of the photographing lens 201 is controlled based on a control signal from the microcomputer 207. In the case of a zoom lens, the focal length is also controlled. Is done. The driver 205 also controls the aperture of the diaphragm 203.

  The microcomputer 207 connected to the driver 205 is connected to the I / F 999 and the flash memory 209. The microcomputer 207 operates according to a program stored in the flash memory 209, communicates with a microcomputer 121 in the camera body 100 described later, and controls the interchangeable lens 200 based on a control signal from the microcomputer 121. Do.

  The flash memory 209 stores various information such as optical characteristics and adjustment values of the interchangeable lens 200 in addition to the above-described program. The I / F 999 is an interface for performing communication between the microcomputer 207 in the interchangeable lens 200 and the microcomputer 121 in the camera body 100.

  A mechanical shutter 101 is disposed in the camera body 100 on the optical axis of the taking lens 201. The mechanical shutter 101 controls the passage time of the subject luminous flux and employs a known lens shutter or focal plane shutter. An image sensor 103 is disposed behind the mechanical shutter 101 and at a position where a subject image is formed by the photographing lens 201.

  In the image sensor 103 functioning as an image pickup unit, photodiodes constituting each pixel are two-dimensionally arranged in a matrix, and each photodiode generates a photoelectric conversion current corresponding to the amount of received light. Is stored by a capacitor connected to each photodiode. A Bayer color filter is arranged in front of each pixel. The Bayer array has a line in which R pixels and G (Gr) pixels are alternately arranged in a horizontal direction, and a line in which G (Gb) pixels and B pixels are alternately arranged, and the three lines are vertically arranged. It is configured by alternately arranging in the direction.

  The image sensor 103 is connected to an analog processing unit 105. The analog processing unit 105 performs waveform shaping on the photoelectric conversion signal (analog image signal) read from the image sensor 103 while reducing reset noise and the like. Further, the gain is increased so as to obtain a more appropriate luminance. The analog processing unit 105 is connected to an A / D conversion unit 107. The A / D conversion unit 107 performs analog-to-digital conversion on the analog image signal, and sends the digital image signal (hereinafter referred to as image data) to the bus 109. Output.

  The bus 109 is a transfer path for transferring various data read or generated inside the camera body 100 to the camera body 100. In addition to the A / D conversion unit 107, the bus 109 includes an image processing unit 111, an AE processing unit 113, an AF processing unit 115, an image compression / decompression unit 117, a microcomputer 121, an SDRAM (Synchronous Dynamic Random Access Memory) 127, A memory interface (hereinafter referred to as “memory I / F”) 129, a liquid crystal (hereinafter referred to as “LCD”) driver 133, an A / D conversion unit 143, and a sound processing unit 151 are connected.

  An image processing unit 111 connected to the bus 109 includes an optical black subtraction unit (hereinafter referred to as an OB subtraction unit), a white balance correction unit (hereinafter referred to as a WB correction unit), a synchronization processing unit, a gamma conversion unit, and a color reproduction process. A color matrix calculation unit, an edge enhancement processing unit, a noise reduction processing unit (hereinafter referred to as an NR processing unit), and an image processing unit. The image data temporarily stored in the SDRAM 127 is read out. Apply image processing.

  The OB subtraction unit in the image processing unit 111 performs an optical black subtraction process. In this optical black subtraction process, the optical black value resulting from the dark current of the image sensor 103 is subtracted from the pixel value of each pixel constituting the image data.

  A WB correction unit in the image processing unit 111 performs white balance correction on the image data. White balance correction is performed so that white is accurately projected under a light source with various color temperatures. The user sets the light source mode such as clear sky, cloudy, shade, light bulb, and fluorescent light, or selects the auto white balance mode that automatically calculates the white balance correction amount on the camera side. Accordingly, white balance correction is performed on the image data.

  The synchronization processing unit performs synchronization processing on image data acquired under the Bayer array into image data including R, G, and B information per pixel. The gamma conversion unit performs gamma correction processing, and the color reproduction processing unit performs color reproduction processing that changes the color of the image. The color matrix calculation unit corrects the color of the image data by performing linear conversion that multiplies the image data by the color matrix. The edge enhancement processing unit extracts edges from the image data, and performs edge enhancement for enhancing the edges of the image data.

  The NR processing unit performs a process of reducing noise in the image data by using a filter that reduces high frequencies, or by a coring process or the like. Further, the image processing unit 111 has an image processing unit for applying an art effect. The image processing unit performs image processing on the image data output from the A / D conversion unit 107 so that an image effect corresponding to the art effect instructed via the operation unit 123 is generated. The image processing unit 111 selects each of the above-described units and performs each process as necessary, and the image data subjected to the image processing is temporarily stored in the SDRAM 127 via the bus 109.

  The AE processing unit 113 measures the subject brightness and outputs it to the microcomputer 121 via the bus 109. Although a dedicated photometric sensor may be provided for measuring the subject brightness, in this embodiment, the subject brightness is calculated based on image data based on the output of the image sensor 103. The AF processing unit 115 extracts a high-frequency component signal from the image data, acquires a focus evaluation value by integration processing, and outputs the focus evaluation value to the microcomputer 121 via the bus 109. In the present embodiment, the photographing lens 201 is focused by a so-called contrast method.

  The image compression / decompression unit 117 reads the image data from the SDRAM 127 when the image data is recorded on the recording medium 131, compresses the read image data according to various compression methods such as a JPEG compression method, and the compressed image data is read out from the SDRAM 127. Remember once. The microcomputer 121 creates a JPEG file by adding a JPEG header necessary for constructing a JPEG file to the JPEG image data temporarily stored in the SDRAM 127, and stores the created JPEG file in the memory I / F 129. To the recording medium 131.

  The image compression / decompression unit 117 also decompresses JPEG image data for image reproduction display. In decompression, the JPEG file recorded in the recording medium 131 is read out, decompressed by the image compression / decompression unit 117, and the decompressed image data is temporarily stored in the SDRAM 127. In this embodiment, the JPEG compression method is adopted as the image compression method. However, the compression method is not limited to this, and TIFF, MPEG, H.264, and the like. Of course, other compression methods such as H.264 may be used. The recording medium 131 is, for example, a recording medium such as a memory card that is detachably attached to the camera body 100, but is not limited thereto, and may be a hard disk or the like built in the camera body 100.

  The microcomputer 121 functions as a control unit for the entire camera, and comprehensively controls various sequences of the camera. In addition to the I / F 999 described above, an operation unit 123 and a flash memory 125 are connected to the microcomputer 121.

  The operation unit 123 includes operation members such as various input buttons and various input keys such as a power button, a release button, a moving image button, a playback button, a menu button, a cross button, an OK button, a shooting mode dial, and the like. The operation state is detected, and the detection result is output to the macro computer 121. The microcomputer 121 executes various sequences according to the user's operation based on the detection result of the operation member from the operation unit 123. The power button is an operation member for instructing to turn on / off the power of the camera. When the power button is pressed, the digital camera is turned on. When the power button is pressed again, the camera is turned off.

  The release button includes a first release switch that is turned on when the button is half-pressed and a second release switch that is turned on when the button is further depressed after being half-pressed and then fully pressed. When the first release switch is turned on, the microcomputer 121 executes a shooting preparation sequence such as AE processing and AF processing. When the second release switch is turned on, the mechanical shutter 101 and the like are controlled, image data based on the subject image is acquired from the image sensor 103 and the like, and a series of shooting sequences for recording the image data on the recording medium 131 are executed. Take a picture.

  The movie button is a button for starting and ending movie shooting. Since the moving image is not yet shot in the initial state, moving image shooting starts when the moving image button is pressed in this state, and moving image shooting ends when the moving image button is pressed during moving image shooting. Therefore, each time the moving image button is pressed, the start and end of moving image shooting are repeated alternately. The reproduction button is an operation button for setting and canceling the reproduction mode. When the reproduction mode is set, the image data of the photographed image is read from the recording medium 131 and the photographed image is reproduced and displayed on the LCD 135.

  The menu button is an operation button for displaying a menu screen on the LCD 135. On the menu screen, you can set various camera modes.For example, white balance mode such as clear sky, cloudy, shade, light bulb, fluorescent light, and white balance mode such as auto white balance and underwater auto white balance. It can be performed. An operation unit 123 such as a menu button and a cross button is an image processing processing selection unit that selects a processing process in which image data obtained by imaging with the image sensor 103 is processed from a plurality of processing processes by the image processing unit. Function as.

  The flash memory 125 stores a program for executing various sequences of the microcomputer 121. The microcomputer 121 controls the digital camera based on this program. In addition, the flash memory 125 is a table (exposure condition determination table) showing combinations of exposure control values (ISO sensitivity, aperture value, shutter speed) that provide appropriate exposure with respect to the subject brightness, and white balance for white balance correction. Gain and color matrix coefficients are stored.

  The SDRAM 127 is an electrically rewritable volatile memory for temporary storage of image data. The SDRAM 127 temporarily stores the image data output from the A / D conversion unit 107 and the image data processed by the image processing unit 111, the image compression / decompression unit 117, and the like. The SDRAM 127 also temporarily stores audio data collected by the microphone 141 and output from the A / D converter 143.

  The memory I / F 129 is connected to the recording medium 131, and controls writing and reading of image data and data such as a header attached to the image data on the recording medium 131. The recording medium 131 is a memory detachable from the camera body, but may be a memory built in the camera body such as a hard disk.

  The LCD driver 133 is connected to the LCD 135 and displays an image on the LCD 135 based on the image data read from the SDRAM 127 or the recording medium 131 and expanded by the image compression / decompression unit 117. The LCD 135 includes a liquid crystal panel disposed on the back surface of the camera body 100 and performs image display. As the image display, there are a REC view display for displaying the recorded image data for a short time immediately after shooting, a playback display of a still image or a moving image file recorded on the recording medium 131, and a moving image display such as a live view display. included. When displaying compressed image data, the image data is displayed after being decompressed by the image compression / decompression unit 117 as described above. The display unit is not limited to the LCD, and other display panels such as an organic EL may be adopted.

The LCD 135 is arranged not only on the back surface of the camera body 100 but also on the top surface as a control panel. The control panel performs display related to control necessary for photographing. In this control panel, as will be described later, it is possible to determine recommended audio processing according to the selected image processing.

  A microphone 141 that functions as a voice input unit that inputs voice is provided on the exterior of the camera body 100, converts ambient sound into an analog voice signal, and outputs the analog voice signal to the A / D conversion unit 143. The A / D conversion unit 143 performs analog-digital conversion on the analog audio signal and outputs a digital audio signal (hereinafter referred to as audio data) to the bus 109.

  The sound processing unit 151 is connected to the bus 109 and includes a sound processing selection unit 153, a preset sound selection unit 155, a sound filter processing unit 157, and a sound synthesis unit 159. The sound filter processing unit 157 performs various processes such as attenuating or emphasizing a specific frequency region when performing sound processing to be described later. The sound synthesizer 159 synthesizes the sound data output from the A / D converter 143 and the sound for synthesis stored in the flash memory 125. The voice data filtered by the sound filter processing unit 157 and the voice data synthesized by the sound synthesis unit 159 are output to the bus 109. The sound filter processing unit 157 and the sound synthesis unit 159 function as a sound processing unit.

  The sound processing selection unit 153 selects one of the sound processes in the sound filter 157 and the sound synthesis unit 159. The sound processing selection unit 153 functions as an audio processing selection unit that selects audio processing from a plurality of audio processings, using the audio data input by the microphone 141 functioning as an audio input unit, and the sound processing selection unit 153 includes: It also functions as a voice processing instruction unit that instructs selection of voice processing by the above-described sound processing selection unit according to the selection result by the microcomputer 121. The preset sound selection unit 155 selects any of the plurality of synthesis sounds stored in the flash memory 125 for synthesis.

  Next, in the block diagram shown in FIG. 1, details of the configuration related to sound processing and image processing in the present embodiment will be described with reference to FIG. 2. In FIG. 2, the bus 109 is omitted. The output of the microphone 141 functioning as an audio input unit is connected to the A / D conversion unit 143, and the output of the A / D conversion unit 143 is connected to a filter 157A in the sound processing unit 151.

  The image art instruction unit 123A instructs one of a plurality of types of art effects. The instruction of the art effect is displayed on the menu screen by the operation unit 123, and the photographer gives an instruction on the menu screen. Of course, other instruction methods may be used. The art effect instructed by the image art instruction unit 123A is output to the image processing unit 111 and the sound processing selection unit 153A in the sound processing unit 151.

  As described above, the image processing unit 111 includes an image processing unit for applying an art effect and performs image processing according to the art effect instructed by the image art instruction unit 123A. Image data that has undergone image processing such as art effects by the image processing unit 111 is output to the compression unit 117A. The compression unit 117A is a part of the above-described image compression / decompression unit 117, and performs compression processing of image data.

  As shown in FIG. 2, the sound processing unit 151 includes a sound processing selection unit 153A, a filter 157A, an art sound processing selection unit 157B, and a synthesizer 159A. 1 performs the same function as the sound processing unit 151 shown in FIG. 1. The sound processing selection unit 153 in FIG. 1 corresponds to the sound processing selection unit 153A in FIG. 2, and the function of the preset sound selection unit 155 in FIG. In FIG. 2, the sound processing selection unit 153A is also used. The sound filter processing unit 157 in FIG. 1 corresponds to the filter 157A and the art sound processing selection unit 157B in FIG. The sound synthesizer 159 in FIG. 1 corresponds to the synthesizer 159A in FIG. A synthesized sound unit 125A in FIG. 2 is a part of the flash memory 125 in FIG. 1, stores synthesized sound for art effects, and stores sound data for synthesis with input voice data. It has a function as a synthesized sound storage unit.

  The sound processing selection unit 153A that inputs audio data from the A / D conversion unit 143 and inputs the art effect selected from the image art instruction unit 123A determines which sound processing to perform among the plurality of sound processings. select. That is, in the present embodiment, the filter 157A emphasizes the sound data in the specific frequency region and attenuates it in reverse, or any one of the plurality of synthesized sounds 1 to n stored in the synthesized sound unit 125A. Sound processing to be superimposed on audio data is possible. The sound processing selection unit 153A selects sound processing appropriate for the art effect instructed by the image art instruction unit 123A, and instructs the art sound processing unit 157B, the synthesized sound unit 125A, and the display processing unit 133A to select audio processing.

  The art sound processing selection unit 157B performs the art sound processing so as to have a filter characteristic according to the art effect selected by the sound processing selection unit 153A from among the plurality of filter processings when performing an art effect using a filter. select. The synthesized sound unit 125 </ b> A is sound data for synthesis stored in the flash memory 125. As will be described later, as the synthesized sound, various sounds such as a noise sound generated when an old film is screened are stored as synthesized sounds 1 to n. Any one of the synthesized sounds 1 to n selected by the sound processing selection unit 153A is output to the synthesizer 159A.

  The filter 157A performs sound processing corresponding to the filter characteristics selected by the art sound processing selection unit 157B on the sound data output from the A / D conversion unit 143, and outputs the sound data to the combiner 159A. The synthesizer 159A performs sound processing for synthesizing any of the synthesized sounds 1 to n output from the synthesized sound unit 125A on the sound data that has been subjected to sound processing by the filter, and outputs the synthesized sound to the recording processing unit 130A. . The recording processing unit 130A corresponds to the memory I / F 129 and the recording medium 131 in FIG. The recording processing unit 130A records the image data compressed by the compressing unit 117A and the audio data processed by the sound processing unit 151 in association with each other.

  The display processing unit 133A for inputting the output of the sound processing selection unit 153A and the output of the image processing unit 111, and the display unit 135A connected to the display processing unit 133A correspond to the LCD driver 133 and the LCD 135 in FIG. The display processing unit 133A generates image data in which a display representing the processing of the sound selected by the sound processing selection unit 153A is superimposed on the image processed by the image processing unit 111 and displayed on the display unit 135A. The display unit 135A functions as an audio processing display unit that displays the instructed audio processing.

  Next, the flow of image data and audio data and the flow of control in this embodiment will be described with reference to FIGS. The photographer can set the user selection information 123B by operating the operation unit 123 on a menu screen or the like. As user selection information 123B, for example, as shown in FIG. 3, ISO 123Ba for selecting ISO sensitivity, tone control 123Bb for adjusting the brightness of a highlight part and a shadow part, WB 123Bc for setting white balance, exposure There is exposure 124Be for setting a correction value.

  The user selection information 123B includes an art filter 123Bd for designating the type of art effect. As described above, in this embodiment, a plurality of art effects can be selected. Art effects include, for example, pop art that enhances the color colorfully and finishes it like a pop art, a toy photo with a reduced amount of peripheral light as taken with a toy camera, a soft focus with a blurred focus, There are rough monochrome like grainy black and white photographs. Data selected by the user selection information 123B is output to the microcomputer 121.

  3, the filing unit 130B connected to the image processing unit 111, the microcomputer 121, and the sound processing unit 151, and the external recording device 130C connected to the filing unit 130B include the memory I / F 129 in FIG. And the recording medium 131 and the recording processing unit 130A of FIG.

  Image data read from the image sensor 103 and digitized by the A / D converter 107 is subjected to image processing by the image processor 111. Based on user selection information such as ISO sensitivity 123Ba, tone control 123Bb, WB123Bc, and exposure correction 123Be, the microcomputer 121 controls the image processing unit 111 to perform image processing. The image processing unit 111 outputs the image data subjected to the image processing to the file forming unit 130B.

  When the art effect is not selected, the ambient sound is converted into an analog audio signal by the microphone 141, and the A / D conversion unit 143 digitizes the analog audio signal. The digitized sound data is subjected to sound processing by the sound processing unit 151, and the sound data subjected to the sound processing is output to the file forming unit 130B.

  When the image forming unit 130B receives image data from the image processing unit 111 and audio data from the sound processing unit 151, the filing unit 130B generates a recording file in accordance with a control signal from the microcomputer 121, and outputs the file to the external recording device 130C.

  As shown in FIG. 4, when the art filter 123Bd for designating an art effect is selected as the user selection information 123B, information on the selected art effect is output to the microcomputer 121. The microcomputer 121 outputs a control signal for performing image processing according to the selected art effect to the image processing unit 111, and the image processing unit 111 executes image processing according to the control signal.

  The microcomputer 121 also outputs a control signal for performing sound processing according to the selected art effect to the sound processing unit 151, and the sound processing unit 151 executes sound processing according to the control signal. . In the block diagram shown in FIG. 2, the selected art effect is directly transmitted from the image art instruction unit 123A to the sound processing selection unit 153, but actually, the sound processing is performed as a control command from the microcomputer 121. To the sound processing selection unit 153 in the unit 151. The sound processing unit 151 executes sound processing appropriate for the selected art effect. The sound processing suitable for the art effect will be described later with reference to FIGS.

  Further, as shown in FIG. 5, when shooting mode information 123 </ b> Bf is selected as user selection information 123 </ b> B, information on the selected shooting mode is output to the microcomputer 121. The shooting mode is selected by the shooting mode dial or the like in the operation unit 123 described above. An operation unit 123 such as a shooting mode dial has a function as a shooting mode selection unit.

  The microcomputer 121 outputs a control signal for performing image processing according to the selected shooting mode to the image processing unit 111, and the image processing unit 111 executes image processing according to the control signal. The microcomputer 121 also outputs a control signal for performing sound processing according to the selected shooting mode to the sound processing unit 151, and the sound processing unit 151 executes sound processing according to the control signal. . The sound processing unit 151 executes sound processing appropriate for the selected shooting mode.

  Thus, in this embodiment, when information such as ISO sensitivity 123Ba and tone control 123Bb is selected as user selection information, the microcomputer 121 controls the image processing unit 111 based on the selected information. Image processing is performed in accordance with the selected information. When the art effect is selected by the art filter 123Bd, the microcomputer 121 controls the image processing unit 111 and the sound processing unit 151 based on the selected art effect. For this reason, sound processing suitable for the art effect is performed. Similarly, when the shooting mode is selected, the image data corresponding to the shooting mode is processed, and the sound data is also processed according to the shooting mode.

  Next, sound processing will be described with reference to FIGS. FIG. 6A shows the flow of audio data in the sound filter unit 157 when no art effect is added. That is, the ambient sound is converted into an audio signal by the microphone 141 and converted into audio data by the A / D conversion unit 143.

  The level of the audio data is adjusted by an automatic level control (ALC) 157a in the sound filter unit 157. In addition, sound having a frequency such as a high-frequency region or a low-frequency region is transmitted with filter characteristics designated in advance by a low-pass filter / high-pass filter (LPF, HPF) 157b. Further, noise is removed by the notch filter 157c. The sound file from which the noise is removed by the sound filter 157 and the sound in the specific frequency region is cut is generated by the file forming unit 130B.

  When the art effect is added, as shown in FIG. 6B, the audio data from which noise has been removed by the notch filter 157c is input to the art effect correspondence filter 157d. The art effect filter 157d has a filter characteristic for applying an art effect according to the filter characteristic selected by the art sound processing selection unit 157B (see FIG. 2). As the art effect, as shown in FIG. 7A, the art effect is selected from those which give various effects such as a sound filter over the telephone. The sound filter over the telephone is a filter that extracts mid-range sounds and cuts low and high sounds.

  In FIGS. 6B and 7A, the art effect is given by the filter. However, the synthesis sound stored in the synthesized sound unit 125A may be superimposed on the voice data. . In the example shown in FIG. 7B, an old film “sea, crackle” sound is superimposed from among a plurality of synthesized sounds stored in the synthesized sound unit 125A. That is, the voice data from which noise has been removed by the notch filter 157c and the “sea, crackle” sound of the old film stored in the synthesized sound unit 125A are synthesized by the synthesizer 159A, and the synthesized voice data is synthesized by the amplifier 159B. The level is reduced to 1/2. The output of the amplifier 159B is output to the filing unit 130B.

  Moreover, you may make it provide both the filter effect and the effect by a synthetic sound with respect to audio | voice data. In the example shown in FIG. 7C, both the effect of the sound filter over the telephone as the art effect corresponding filter 157d and the effect of superimposing the “sea, crackle” sound of the old film are given to the audio data.

  As described above, the sound processing unit 151 can give various effects to the sound data based on the sound signal acquired by the microphone 141 using the art effect corresponding filter 157d and the synthesized sound 125A.

  As described above, as described above, the sound processing selection unit 153A selects sound processing appropriate for the art effect instructed by the image art instruction unit 123A. An example of this selection is shown in FIG. For example, when pop is selected as the art effect, pop is selected as the sound filter processing by the filter 157A, and bubble sound (sound that bubbles are generated) is selected as the synthesized sound 125A. Also, when glitter is added as an art effect in addition to pop, that is, when adding an effect that adds a streak to a high-luminance part, a glittering sound by a bell is selected as the synthesized sound 125A Is done.

  Next, mode setting on the menu screen in the present embodiment will be described with reference to FIGS. FIG. 9 shows a menu screen selection procedure. FIG. 9A shows a state where the setup menu screen 301 is selected by operating the cross button in the operation unit 123 on the menu screen. Further, when the cursor is moved by operating the cross button and the sound setting 301a is selected, a sound setting screen 303 is displayed as shown in FIG. 9B.

  In the sound setting screen 303, various settings such as operation sound type, operation sound volume, and playback sound volume can be made. When the cursor is moved down, a sound filter mode 303a is set as shown in FIG. You can choose. When the sound filter mode 303a is selected, a sound filter mode screen 305 is displayed as shown in FIG.

  On the sound filter mode screen 305, a recommended mode 305a, a live view (LV) control 305b, and a super control panel 305c are displayed, and these items can be selected by operating the cross button. The recommended mode 305a is a mode in which recommended sound processing is automatically selected when an art effect is selected. The LV control 305b is a mode in which sound processing can be selected during live view display on the LCD 135. The super control panel 303c is not a recommended mode for sound processing when an art effect is selected, but a mode in which a combination of sound processing is determined in advance by the photographer's intention.

  FIG. 10 shows a procedure for selecting sound processing on the live view display screen when the LV control 305b is set. When the LV control is set, a sound process intended by the photographer can be selected from a plurality of prepared sound processes. First, FIG. 10A shows a normal live view display screen. When the OK button in the operation unit 123 is operated on the live view display screen, a setting screen as shown in FIG. 10B is displayed. In this state, the cursors 311a and 311b are moved to select an art effect as shown in FIG.

  When the OK button is operated again in this state, a sound processing menu is displayed as shown in FIG. On this screen, the cursor 311c can be moved by operating the up and down buttons, and the sound processing intended by the photographer can be selected. FIG. 10E shows a state where “over the phone” is selected as the sound processing by the cursor 311c, and when the OK button is operated in this state, “over the phone” is selected as the sound processing (FIG. 10E). )reference).

  Thus, when the LV control 305b is selected, sound processing can be intuitively selected on the live view display screen. In the example shown in FIG. 10, one sound process is selected for the art effect, but it is needless to say that a plurality of sound processes may be selected. FIG. 11 shows an example of selecting a plurality of sound processes in the LV control mode.

  FIG. 11A is a setting screen similar to FIG. When the art effect 2 is selected by operating the up and down buttons on this screen, the screen changes to the screen shown in FIG. When “over the phone” is selected on this screen (see FIG. 11C), the cursor 311c is further moved by operating the up and down buttons, and “old film sound” is selected, as shown in FIG. 11D. Two sound processes are selected: “over phone” and “old film sound”.

  Also, in the recommended mode, the photographer automatically performs sound processing according to the set art effect within a predetermined range of combinations of art effects and sound processing (usually one sound processing for art effects). Can be set to However, there are cases where it is desired to combine sound processing other than a preset combination. In that case, the super control panel 305c may be selected on the screen shown in FIG.

  FIG. 12 shows a control panel display in the super control panel mode. FIG. 12A shows combinations that are predetermined as recommended settings. In this combination, the fantastic focus, daydream, etc. shown on the vertical axis show art effects, and the unprocessed, echo, sound over the telephone, sound of old film, etc. shown on the horizontal axis show sound processing. ing. In the recommended setting shown in FIG. 12A, an echo is automatically set when a fantastic focus is selected as an art effect. Further, when daydream is selected as the art effect, non-processed sound processing is not set as a recommended image.

  FIG. 12B shows the display of the control panel in the super control panel mode in which the photographer changes the combination of settings using the control panel. In this case, the sound processing that can be set corresponding to the art effect can be arbitrarily changed / added / deleted by operating the up / down buttons and the like. In the example shown in FIG. 12B, when the gentle sepia is set, the sound of the old film is automatically set in the recommended setting, but the sound over the telephone is added. When a plurality of sound processes are selected, any one is selected by an operation member such as an up / down button. Note that voice processing that cannot be recommended may be unselected.

  As described above, in the present embodiment, when the recommended mode is set as the sound processing, usually one sound processing corresponding to the art effect is determined, and the recommended sound processing is automatically set. Is done. In addition, when the LV control is set, the photographer can arbitrarily set while viewing the live view display screen. In addition, the recommended setting or the setting can be changed by the super control panel mode.

  Next, the operation in this embodiment will be described with reference to the flowcharts shown in FIGS. These flows are executed by the control unit according to the program stored in the flash memory 125. The flow shown in FIG. 13 and FIG. 14 shows a part related to the recording operation in the main routine. This main routine starts executing when the power button of the operation unit 117 is turned on.

  First, photographing is performed (S1). Here, based on the image signal from the image sensor 103, live view display is performed on the LCD 135, and photometry and distance measurement are performed in response to a half-press of the release button. In addition, a still image is shot in response to a full press of the release button, and moving image shooting is started when the moving image button is operated.

  If a still image or a moving image is shot in step S1, it is next determined whether or not to record after shooting (S3). Since whether to record at the time of shooting or to record after shooting is set on the menu screen, this step determines based on this setting. In the case of moving image shooting, the determination in step S3 is performed prior to shooting prior to step S1. If the result of determination in step S3 is not recording after shooting, that is, if recording is performed during shooting, then recording is started (S5). Here, sound is picked up by the microphone 141, converted into sound data by the A / D conversion unit 143, and the sound data is temporarily stored in the SDRAM 127.

  When recording is started, it is next determined whether or not the recording buffer is full (S7). It is determined whether or not the buffer capacity of the SDRAM 127 for temporarily storing audio data is full. If the result of this determination is that the buffer capacity is not full, recording continues until the buffer capacity is full.

  If the result of determination in step S7 is that the recording buffer is full, it is next determined whether or not a sound filter has been selected (S9). As described above, in this embodiment, when the art effect is selected, the sound processing selection unit 153 (153A) automatically selects the filter 157A of the sound filter processing unit 157. Therefore, in this step, it is determined whether or not a filter is selected by the sound processing selection unit 153.

  If the result of determination in step S9 is that a sound filter has been selected, it is next determined whether or not the selected sound filter is a phone call sound (S11). In this case, as described with reference to FIGS. 7A and 7C, the sound over the telephone is selected as the filter. If the result of this determination is that a phone call sound has been selected, the middle band filter (BPF) is activated (S13). Here, the high-frequency region and the low-frequency region are cut by the filter 157A, and the middle-tone band-pass filter that allows the sound in the middle-tone region to pass is operated.

  If the mid-band filter is activated in step S13 or if the result of determination in step S11 is that no sound over the telephone has been selected, then whether or not floating feeling is selected as the sound filter is determined. Is determined (S15). A sound with a floating feeling is a sound that produces a feeling that bubbles are floating. If the result of this determination is that floating feeling has been selected, the middle bandpass filter (BPF) is operated. The mid-tone bandpass filter has a mid-tone region that is different from the case of the sound over the telephone so that a floating feeling is generated.

  If the middle bandpass filter is operated in step S17 or if the result of determination in step S15 is that no floating sound has been selected, then whether or not a healing sound is selected as the sound filter. Is determined (S19). A healing sound is a sound that produces a feeling of being healed. If the result of this determination is that healing has been selected, the high pass filter (HPF) is activated (S21). Here, the sound in the treble region is passed through the audio data.

  If the high-pass filter is operated in step S21 or if the result of determination in step S19 is that healing has not been selected, it is next determined whether or not echo is selected as the sound filter (S23). Echo is a sound that feels like a repeating sound pattern. If the result of this determination is that echo is selected, the echo is operated (S25).

  If the echo is activated in step S25 or if the result of determination in step S23 is that echo is not selected, it is next determined whether or not a sound effect is selected (S31). Steps S9 to S25 select the characteristics in the filter 157A. Here, as shown in FIGS. 7B and 7C, whether the synthesized sound 125A to be synthesized is selected in the synthesizer 159A. It is a determination of whether or not. Note that the selection of the synthesized sound 125A is performed by the preset sound selection unit 155 (sound processing selection unit 153A in FIG. 2).

  If the result of determination in step S31 is that a sound effect has been selected, it is next determined whether or not the selected sound effect is an old film (S23). If the result of this determination is that the sound effect is an old film, the old film sound is added (S35). In this case, as shown in FIGS. 7B and 7C, a sound such as “sea, crackle” that is emitted when an old film is screened by a projector is superimposed as a synthesized sound. That is, the synthesized sound of the old film is read from the synthesized sound 125A and is superimposed on the audio data by the synthesizer 159A.

  If the sound of the old film is added in step S35, or if the sound effect is not the old film as a result of the determination in step S33, it is next determined whether or not the sound effect is a waterside (S37). If the result of this determination is that the sound effect is waterside, the waterside sound is added (S39). Here, a waterside synthetic sound is read from the synthetic sound 125A, and is superimposed on the audio data by the synthesizer 159A.

  If the waterside sound is added in step S39 or if the result of the determination in step S37 is that the sound effect is not a waterside, it is next determined whether or not the sound effect is a firework (S41). If the result of this determination is that the sound effect is fireworks, firework sounds are added (S43). Here, the synthesized sound of fireworks is read from the synthesized sound 125A, and is superimposed on the audio data by the synthesizer 159A.

  If the sound of fireworks is added in step S43, or if the sound effect is not fireworks as a result of the determination in step S41, it is next determined whether or not the sound effect is a road (S45). If the result of this determination is that the sound effect is a road, footsteps and car sounds are added (S47). Here, footsteps and car synthesized sounds are read from synthesized sound 125A and superimposed on audio data by synthesizer 159A.

  If footsteps and car sounds are added in step S47, or if the result of determination in step S45 is that the sound effect is not a road, or if the result of determination in step S3 is to record after shooting, then: A file is created (S49). Since the audio data temporarily stored in the SDRAM 127 is subjected to sound processing in steps S9 to S47, in this step, the audio data subjected to the sound processing and the captured still image or moving image File the data. This filing is performed by the filing unit 130B.

  Once the file is created, this flow is ended when the moving image shooting or still image shooting has been completed. If the moving image shooting or the still image shooting has not ended, the process returns to step S1 to continue shooting.

  As described above, in the flow according to the present embodiment, when recording is performed during shooting, various sound processing such as changing sound with a filter or adding synthesized sound to sound data is performed. Also, the sound processing at this time is performed by automatically selecting a process suitable for the set art effect. In addition, the addition of the sound filter in steps S11 to S25 and the synthesized sound in steps S33 to S47 are examples. In addition to those shown in this flow, various sound processing such as the example shown in FIG. 8 may be performed.

  As described above, in one embodiment of the present invention, image processing is performed when processing data to be processed is selected from a plurality of processing processes (art effects) for image data obtained by imaging by an imaging unit. According to the processing selection result, selection of audio processing is instructed. For this reason, when the image is recorded and the ambient sound is recorded and the image is processed, the emotion and entertainment at the time of viewing are not impaired.

  In the embodiment of the present invention, a digital camera is used as an apparatus for photographing. However, the camera may be a digital single-lens reflex camera or a compact digital camera, such as a video camera or a movie camera. It may be a camera for moving images, or may be a camera built in a mobile phone, a personal digital assistant (PDA), a game device, or the like.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 100 ... Camera body, 101 ... Mechanical shutter, 103 ... Image sensor, 105 ... Analog processing part, 107 ... A / D conversion part, 109 ... Bus, 111 ... Image processing 113 ... AE processing unit, 115 ... AF processing unit, 117 ... image compression / decompression unit, 117A ... compression unit, 121 ... microcomputer, 123 ... operation unit, 123A. ..Image art instruction section, 123B ... user selection information, 123Ba ... ISO, 123Bb ... tone control, 123Bc ... white balance (WB), 123Bd ... art filter, 123Be ... exposure Correction, 123Bf ... Shooting mode, 125 ... Flash memory, 125A ... Synthetic sound part, 127 ... SDRAM, 129 ... Memory / F, 130A... Recording processing unit, 130B... Filing unit, 130C... External recording device, 131... Recording medium, 133... LCD driver, 133A. ... LCD, 135A ... display unit, 141 ... microphone, 143 ... A / D conversion unit, 151 ... sound processing unit, 153 ... sound processing selection unit, 155 ... preset Sound selection unit, 157 ... Sound filter processing unit, 157A ... Filter, 157B ... Art sound processing selection unit, 157a ... Automatic level adjustment (ALC), 157b ... LPF / HPF, 157c, .... Notch filter, 157d ... Art effect filter, 159 ... Sound synthesizer, 159A ... Synthesizer, 159B ... Amplifier, 200 ... Interchangeable lens, 201 ... Photography Lens, 203 ... Aperture, 205 ... Driver, 207 ... Microcomputer, 209 ... Flash memory, 301 ... Setup menu screen, 301a ... Sound settings 301a, 303 ... Sound settings Screen, 303a ... Sound filter mode, 305 ... Sound filter mode screen, 305a ... Recommended mode, 305b ... LV control, 305c ... Super control panel, 999 ... I / F

Claims (10)

An imaging unit that captures an image;
A voice input unit for inputting voice;
An image processing processing selection unit for selecting a processing process desired to be processed from a plurality of processing processes on the image data obtained by imaging by the imaging unit;
An image processing unit that performs image processing on the image data in accordance with a selection result by the image processing processing selection unit;
A voice processing selection unit that selects voice processing to be performed on voice data input by the voice input unit from a plurality of voice processing;
A voice processing instruction unit for instructing selection of voice processing by the voice processing selection unit according to a selection result by the image processing processing selection unit;
An audio processing unit that outputs audio data processed by audio processing instructed by the audio processing instruction unit in association with the image data;
A display unit for displaying a live view image captured by the imaging unit;
A display mode selection unit for selecting a display mode of the display unit;
Comprising
The display unit displays a display representing the plurality of selectable processing processes and a display representing the plurality of selectable audio processes when a mode in which the live view image is displayed in the display mode is selected. Is displayed together with the live view image.   The speech processing apparatus according to claim 1, wherein the display mode selection unit is further capable of selecting a recommended mode or a super control panel mode. Above when the display mode the recommended mode by the selection unit is selected, in accordance with the selected processing, speech processing apparatus according to claim 2, wherein the determining the audio processing recommended. The voice processing apparatus according to claim 2, wherein when the super control panel mode is selected by the display mode selection unit, voice processing that cannot be recommended is not selected.   Furthermore, it has a synthesized sound storage unit that stores sound data to be synthesized with respect to the input voice data, and the voice processing unit uses the synthesized sound stored in the synthesized sound storage unit as the instruction. The voice processing apparatus according to claim 1, wherein voice processing is performed in response.   6. The voice according to claim 5, wherein the voice processing unit includes a filter processing unit that performs a filtering process on input voice data, and the synthesized sound includes a sound effect that is not related to the input voice data. Processing equipment.   The audio processing apparatus according to claim 1, further comprising a shooting mode selection unit, wherein the audio processing unit selects audio processing according to the shooting mode selected by the shooting mode selection unit. An imaging unit that captures an image;
A voice input unit for inputting voice;
An image processing processing selection unit for selecting a processing process desired to be processed from a plurality of processing processes on the image data obtained by imaging by the imaging unit;
An image processing unit that performs image processing on the image data in accordance with a selection result by the image processing processing selection unit;
A voice processing selection unit that selects voice processing to be performed on voice data input by the voice input unit from a plurality of voice processing;
A voice processing instruction unit for instructing selection of voice processing by the voice processing selection unit according to a selection result by the image processing processing selection unit;
A control unit that selects a combination of image processing and audio processing to be recorded or reproduced, and controls to output the image data processed by the selected image processing and the audio data that has been audio-processed in association with each other; and
A display unit for displaying a live view image captured by the imaging unit;
A display mode selection unit for selecting a display mode of the display unit;
Comprising
The display unit displays a display representing the plurality of selectable processing processes and a display representing the plurality of selectable audio processes when a mode in which the live view image is displayed in the display mode is selected. Is displayed together with the live view image. An imaging step for imaging an image;
A voice input step for inputting voice;
An image processing process selection step for selecting a processing process desired to be processed from among a plurality of processing processes on the image data obtained by imaging in the imaging step;
An image processing step for performing image processing on the image data in accordance with the selection result of the image processing processing selection step;
A voice processing selection step of selecting voice processing to be performed on the voice data input in the voice input step from a plurality of voice processing;
An audio processing instruction step for instructing selection of audio processing by the audio processing selection step according to a selection result by the image processing processing selection step;
An audio processing step for outputting the audio data processed by the audio processing instructed in the audio processing instruction step in association with the image data;
A display step for displaying the live view image captured in the imaging step;
A display mode selection step of selecting a display mode in the display step;
Comprising
In the display step, when a mode in which the live view image is displayed is selected in the display mode, a display representing the plurality of selectable processing processes and a display representing the plurality of selectable audio processes Is displayed together with the live view image. An imaging step for imaging an image;
A voice input step for inputting voice;
An image processing process selection step for selecting a processing process desired to be processed from among a plurality of processing processes on the image data obtained by imaging in the imaging step;
An image processing step for performing image processing on the image data in accordance with the selection result of the image processing processing selection step;
A voice processing selection step of selecting voice processing to be performed on the voice data input in the voice input step from a plurality of voice processing;
An audio processing instruction step for instructing selection of audio processing by the audio processing selection step according to a selection result by the image processing processing selection step;
An audio processing step for outputting the audio data processed by the audio processing instructed in the audio processing instruction step in association with the image data;
A display step for displaying the live view image captured in the imaging step;
A display mode selection step of selecting a display mode in the display step;
Comprising
In the display step, when a mode in which the live view image is displayed is selected in the display mode, a display representing the plurality of selectable processing processes and a display representing the plurality of selectable audio processes And causing the computer to execute the display together with the live view image.