JP2013211715A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To more efficiently execute processing across a plurality of image processing circuits.SOLUTION: An imaging device comprises an imaging unit, a first image processing circuit and a second image processing circuit, a first memory connected to the first image processing circuit, a second memory connected to the second image processing circuit, and a bus connecting the first image processing circuit to the second image processing circuit. The first image processing circuit and the second image processing circuit have address spaces respectively, and the first memory and the second memory are mapped in any of the respective address spaces.

Description

  The present invention relates to an imaging apparatus.

  In recent years, in an imaging apparatus such as a digital camera, the processing capability of an image processing circuit tends to be insufficient as the number of pixels of an image sensor increases. For this reason, an imaging apparatus that realizes high-speed image processing by parallel processing using a plurality of image processing circuits has been proposed (see, for example, Patent Document 1).

JP 2008-219319 A

  In the conventional imaging apparatus, when image data is processed across a plurality of image processing circuits, the image data is written to the memory every time the image data is transferred between the image processing circuits.

  Further, in the conventional imaging apparatus, when processing image data across a plurality of image processing circuits, the processing target image data recorded in the memory of the transfer source image processing circuit is transferred to the transfer destination image processing circuit. The process was executed after copying (or moving) to the memory. Therefore, in the conventional imaging device, the time until the process is started becomes long (latency becomes large).

  An imaging apparatus according to one aspect includes an imaging unit that outputs image data obtained by capturing an image of a subject, a first image processing circuit and a second image processing circuit that perform image processing on each image data, and first image processing A first memory connected to the circuit; a second memory connected to the second image processing circuit; and a bus connecting the first image processing circuit and the second image processing circuit. Each of the first image processing circuit and the second image processing circuit has an address space, and the first memory and the second memory are mapped to each address space.

  In the one aspect, the first image processing circuit may display and output the image stored in the first memory on the display device.

  In the one aspect described above, the imaging unit may output moving image data. The first image processing circuit may execute a recording process for recording a moving image file obtained by compressing moving image data on a nonvolatile recording medium. In addition, the second memory may store a reference frame used for inter-frame predictive coding at the time of compression, and read the reference frame from the second memory. Further, the second image processing circuit may execute a moving image encoding process in compression.

  In addition, the current frame to be compressed may be stored in the second memory. Alternatively, the current frame to be compressed may be stored in the first memory.

  The imaging apparatus may further include an audio signal processing unit that acquires audio data when capturing a moving image. The second image processing circuit may associate the audio data with the moving image.

  Each image processing circuit can directly access image data on a memory connected to another image processing circuit, and can efficiently execute processing across a plurality of image processing circuits.

The figure which shows the structural example of the imaging device in one embodiment. The block diagram which shows the structural example of a 1st image processing circuit and a 2nd image processing circuit A diagram schematically showing an example of a memory map Diagram showing an example of frame distribution when continuous shooting is performed in still image shooting mode Outline diagram when recording the JPEG image file on the slave side to the recording medium Schematic diagram when displaying the slave freeze image on the display Schematic diagram of operation example 1 of the electronic camera in the moving image shooting mode Schematic of operation example 2 of electronic camera in moving image shooting mode Schematic diagram of operation example 3 of electronic camera in moving image shooting mode Schematic diagram of operation example 4 of electronic camera in moving image shooting mode Schematic diagram of operation example 5 of the electronic camera in the moving image shooting mode Schematic of operation example 6 of electronic camera in moving image shooting mode

<Description of Configuration Example of Imaging Device>
FIG. 1 is a diagram illustrating a configuration example of an imaging apparatus according to an embodiment. In this embodiment, an example of an electronic camera capable of selecting a still image shooting mode and a moving image shooting mode will be described as an example of an imaging apparatus.

  The electronic camera includes a photographing optical system 11, an imaging unit 12, a data distributor 13, a first image processing circuit 14 and a second image processing circuit 15, a serial bus 16, a first memory 17 and a second memory 18. A first flash memory 19, a second flash memory 20, a display unit 21, an external I / F connector 22, a media connector 23, a release button 24, an operation button 25, and an audio signal processing unit 27. doing. The imaging unit 12 and the audio signal processing unit 27 are connected to the data distributor 13.

  The photographing optical system 11 is composed of a plurality of lenses including, for example, a zoom lens and a focus lens. For simplicity, FIG. 1 shows the photographing optical system 11 with a single lens.

  The imaging unit 12 includes an imaging device 31, an analog front end (AFE) unit 32, and a timing generator (TG) unit (not shown). The imaging element 31 is a device that captures an image of a subject by a light beam that has passed through the photographing optical system 11. The image signal output of the image sensor 31 is connected to the AFE unit 32. The image sensor 31 of the present embodiment may be a solid-state image sensor (for example, a CCD) that reads an image signal by a sequential scanning method (or an interlace method), and a solid-state image sensor (for example, a CMOS) that reads an image signal by an XY address method. It may be.

  A plurality of light receiving elements are arranged in a matrix on the light receiving surface of the image sensor 31. A color filter is disposed on each light receiving element of the image sensor 31. For example, in this embodiment, red (R), green (G), and blue (B) color filters are arranged in each light receiving element according to a known Bayer arrangement. For this reason, the light receiving element of the image pickup element 31 outputs an image signal corresponding to each color by color separation in the color filter. Thereby, the image pick-up element 31 can acquire a color image at the time of imaging | photography.

  Here, the basic operation of the image pickup device 31 is roughly divided into a charge accumulation operation for accumulating charges in each pixel and a charge readout operation for moving the accumulated charges from the pixels. In addition, there are two types of charges accumulated in each pixel: unwanted charges (noise) generated by unintentional light reception and dark current, and signal charges generated from the luminous flux of the subject image. In order to prevent the unnecessary charges from being mixed into the signal charges, the imaging element 31 may be shifted to the charge accumulation operation in a state where the luminous flux of the subject image is received after the unnecessary charges are discharged during imaging by the imaging element 31. As a result, only signal charges can be accumulated in each pixel. For this reason, imaging of the image sensor 31 is performed in a sequence of “discharge of unnecessary charges → accumulation of signal charges (also referred to as exposure operation) → reading of signal charges (output of image signals)”. Among these, the signal charge accumulation period is the so-called exposure time (shutter seconds). The above-described imaging of the imaging device 31 is performed by control from a TG unit (not shown).

  In the imaging operation of the imaging unit 12 as a whole, the imaging device 31 performs the first mode (also referred to as a one-shot mode) in which the imaging device 31 performs the exposure operation and the signal charge readout operation in a single shot. There is a second mode (also referred to as a continuous mode) in which these two operations are automatically repeated at a predetermined time period (frame period). Switching between the first mode and the second mode is generally performed by rewriting a register in the TG unit via an unillustrated SIO (serial I / O).

  The first mode described above is used when taking a still image with an electronic camera. In the first mode, since the exposure operation and the signal charge reading operation are generally performed by operating the TG unit from outside, the timing of the imaging operation including the exposure time is determined by the timing of performing these operations. This corresponds to the fact that shooting of a still image occurs asynchronously by a user operation. The operation on the TG unit includes an operation using an external signal using a GPIO unit 49 described later in addition to the serial communication described above. In addition, the image signal output in the first mode is generally of the full resolution of the image sensor 31. In taking a still image with an electronic camera, an image generated from this image signal is recorded.

  On the other hand, in the above-described second mode, the image pickup device 31 is picked up by parameters set in advance in the register of the TG unit. In the second mode, the exposure timing including the exposure time and the output timing of the image signal are also fixed according to the setting, and the exposure time is limited to the above-described frame period or less. Instead, in the second mode, the image signal is output continuously and automatically at a constant cycle, which is suitable for live view (LiveView) before moving image shooting with an electronic camera or still image shooting. In the case of the second mode, there are generally a plurality of combinations of resolutions and frame periods of output image signals, and such combinations are changed by rewriting the register of the TG unit.

  The resolution of the image signal in the second mode is selected according to the purpose of use. For example, when live view is performed on a VGA (640 × 480 pixels) size LCD panel, an image signal having a resolution slightly larger than the VGA size is set to be output from the imaging unit 12. In this case, power consumption during live view of the electronic camera can be suppressed by not outputting an image signal larger than necessary from the imaging unit 12. On the other hand, when shooting a movie of full HD (1920 × 1080 pixels) size, the image pickup unit 12 is set to output an image signal having a resolution slightly larger than the full HD size.

  The frame period in the second mode is also selected according to the purpose of use. For example, when shooting a moving image of 60 fps, the image signal is set to be output from the imaging unit 12 at 60 fps. When shooting a 30 fps moving image, setting is made so that an image signal is output from the imaging unit 12 at 30 fps. For example, when focusing at a high speed by AF, it is more advantageous to output an image signal at 60 fps than at 30 fps.

  In the still image shooting mode of the electronic camera, the operation of the electronic camera changes from “live view → still image shooting → live view”. Correspondingly, it is necessary to switch the imaging operation of the imaging unit 12 as “second mode → first mode → second mode”. In the case of single shooting, the imaging operation of the imaging unit 12 returns to the imaging operation in the second mode after the imaging operation in the first mode is performed for one frame. In the case of continuous shooting, the imaging operation of the imaging unit 12 returns to the imaging operation in the second mode after the imaging operation in the first mode is continuously performed for the number of continuous shooting frames.

  On the other hand, in the moving image shooting mode of the electronic camera, there is no switching of the image capturing operation of the image capturing unit 12 as described above, and the image capturing unit 12 performs the image capturing operation in the second mode with a combination of the resolution and the frame period suitable for the moving image specifications. continue.

  Returning to FIG. 1, the AFE unit 32 is a circuit that performs analog signal processing on an image signal output from the image sensor 31. The AFE unit 32 performs correlated double sampling, image signal gain adjustment, and A / D conversion of the image signal. The digital image data after the signal processing by the AFE unit 32 is sent to the first image processing circuit 14 or the second image processing circuit 15 via the data distributor 13. The image data output from the AFE unit 32 is RGB image data (hereinafter referred to as Bayer data) having a Bayer array structure before color interpolation. When the image sensor 31 is a CMOS, the image sensor 31 may include an AFE unit 32 or a TG unit (not shown).

  The data distributor 13 is a selector with one input and two outputs, and the two outputs are connected to the first image processing circuit 14 and the second image processing circuit 15, respectively. The data distributor 13 outputs the image data output from the AFE unit 32 to the first image processing circuit 14 and the second image processing circuit 15. Further, the data distributor 13 outputs the audio data output from the audio signal processing unit 27 to the first image processing circuit 14 and the second image processing circuit 15.

  The first image processing circuit 14 and the second image processing circuit 15 are image processing processors for performing predetermined image processing on image data, and are often configured by one LSI. As shown in FIG. 2, the first image processing circuit 14 and the second image processing circuit 15 of the present embodiment have the same circuit configuration and are connected in parallel to the data distributor 13.

  The first image processing circuit 14 and the second image processing circuit 15 are connected to each other by an external serial bus 16. A first memory 17, a first flash memory 19, a display unit 21, an external I / F connector 22, a media connector 23, a release button 24 and an operation button 25 are connected to the first image processing circuit 14. A second memory 18 and a second flash memory 20 are respectively connected to the second image processing circuit 15. The serial bus 16 of the present embodiment transmits data from the first image processing circuit 14 to the second image processing circuit 15 and transmits data from the second image processing circuit 15 to the first image processing circuit 14. This is a bidirectional bus consisting of two buses.

  The first image processing circuit 14 operates as a master image processing circuit and is always in an operating state in the photographing mode. On the other hand, the second image processing circuit 15 operates as a slave image processing circuit. When the image processing load of the electronic camera is relatively low (for example, when performing single shooting in the still image shooting mode or in the playback mode), the second image processing circuit 15 is in a power-off state or low power consumption. The operation has stopped due to the power state. When the load of image processing of the electronic camera is relatively high (for example, when continuous shooting is performed in the still image shooting mode or in the moving image shooting mode), the second image processing circuit 15 is in an operating state, Photographing processing is performed in parallel by the first image processing circuit 14 and the second image processing circuit 15. Note that on / off control of the second image processing circuit 15 is performed by the first image processing circuit 14.

  The first memory 17 and the second memory 18 are large-capacity volatile memories, and usually DRAMs are used. These memories temporarily store image data (generally a plurality of frames), store variables used in the program, and are used as a stack when executing the program. The first flash memory 19 and the second flash memory 20 are non-volatile memories, and programs such as a real-time operating system (RTOS) kernel, device driver, middleware, application, etc., which will be described later. Used to store setting information and the like.

  The display unit 21 is a display device such as a liquid crystal panel or an organic EL panel, and is disposed, for example, on the rear surface of the casing of the electronic camera. The external I / F connector 22 is used to connect an external device (such as a personal computer) having an interface (for example, USB) of a predetermined communication standard. The media connector 23 is used for connecting a non-volatile recording medium 34. Note that the recording medium 34 connected to the media connector 23 is, for example, a hard disk or a memory card incorporating a semiconductor memory.

  The release button 24 is a button used in the still image shooting mode and the moving image shooting mode, and a half-press signal for instructing the start of a shooting preparation operation such as automatic focusing (AF) and the start of shooting. Two of the full-press signals for instructing are generated. The operation buttons 25 collectively represent various buttons operated by the user, and include, for example, buttons for switching the shooting mode.

  The audio signal processing unit 27 includes a microphone that collects external audio from the electronic camera, an amplifier that amplifies the audio signal collected by the microphone, an auto level control unit that maintains the level of the audio signal constant, and an A / D converter. And a serial data output unit for outputting digitized audio data.

  Next, a configuration example of the first image processing circuit 14 and the second image processing circuit 15 will be described in detail with reference to FIG. In the following description, “n_1” is used to indicate the element of the symbol n of the first image processing circuit 14, and “n_2” is used to indicate the element of the symbol n of the second image processing circuit 15. .

  First, the configuration of the first image processing circuit 14 will be described. The first image processing circuit 14 includes an image processing unit 41_1, a still image compression unit 42_1, a moving image compression unit 43_1, a DMAC (Direct Memory Access Controller) 44_1, a CPU 45_1, an interrupt controller 46_1, and a built-in memory 47_1. , Display controller 48_1, GPIO (General Purpose Input / Output) section 49_1, memory controller 50_1, flash memory controller 51_1, external I / F controller 52_1, media controller 53_1, audio I / F 54_1, internal bus 55_1 and a bus bridge 56_1. Each unit of the first image processing circuit 14 is connected to the internal bus 55_1. The internal bus 55_1 is normally a parallel bus, and may be, for example, a known on-chip bus (AMBA AXI, OCP (Open Core Protocol, etc.)).

  The image processing unit 41_1 includes a pre-processing unit and a post-processing unit (both not shown) that are independent circuits, and performs image processing (pre-processing and post-processing) on input image data. Here, the preprocess is a process that mainly performs image correction, and performs signal processing such as defective pixel correction, shading correction, and OB clamp processing on input image data. In addition to executing the preprocess, the image processing unit 41_1 uses the corrected image data to calculate an auto white balance (AWB) evaluation value, an AF evaluation value, and an automatic exposure (Auto Exposure: AE) evaluation value, respectively. Generate.

  In post-processing, image processing after pre-processing is performed mainly on color processing (for example, color interpolation processing, gradation conversion processing, white balance adjustment processing, noise removal processing, contour enhancement processing, color Conversion processing). The RGB Bayer data is converted into YCbCr format image data (hereinafter referred to as YCbCr data) in which desired color reproduction is performed by the image processing in the image processing unit 41_1 described above. The YCbCr data is used when the image data is subjected to compression processing or when the image data is displayed on the display unit 21.

  Here, at the time of shooting a still image, the image processing unit 41_1 generates a main image, a freeze image, and a thumbnail from the captured still image data. These are all images of YCbCr data. The main image is an image (usually high resolution) recorded as main data. The freeze image is an image displayed on the display unit 21 immediately after shooting for confirmation of the shooting result. The thumbnail is a reduced image recorded in an image file together with the main image in order to improve the searchability of the image. The through image and the freeze image have a size of 640 × 480 pixels, for example, and the thumbnail has a size of 160 × 120 pixels, for example. The image processing unit 41_1 generates these small resolution images using an internal resolution conversion circuit (not shown).

  In the still image shooting mode, the still image compression unit 42_1 performs, for example, JPEG compression processing on the YCbCr data of the captured still image.

  The moving image compression unit 43_1 performs compression processing using inter-frame prediction on YCbCr data of a moving image for recording in the moving image shooting mode. For example, the moving image compressing unit 43_1 may be H.264. H.264 (MPEG-4 AVC (Part 10)) moving image compression processing is performed (the encoding of moving images in H.264 will be described later).

  The DMAC 44_1 is a circuit that performs data transfer in hardware, and realizes high-speed data transfer compared to the case where data is transferred by the program of the CPU 45_1. The DMAC 44_1 transfers data while shifting the source address and the destination address for each unit transfer data amount. Note that the DMAC 44_1 may be incorporated in the above-described image processing unit 41_1, still image compression unit 42_1, moving image compression unit 43_1, or some circuit blocks described later.

  The CPU 45_1 executes an application program for realizing the function of the electronic camera, and controls the operation of each part in the electronic camera based on the executed application program. For example, the CPU 45_1 performs AF using a known contrast detection method using the AF evaluation value and AE using the AE evaluation value. Further, the CPU 45_1 executes an image display process on the display unit 21, an image file recording process on the recording medium 34, and the like. These processes such as AF and AE are implemented as the above-described application programs executed by the CPU 45_1. The application program can be divided into processing units called tasks corresponding to the respective processes, and a plurality of tasks can be executed in parallel under the management of RTOS (kernel). The execution order of a plurality of tasks depends on the priority order, and the RTOS executes the task with the highest priority among the plurality of tasks in an executable state. As an example, when the task B having a higher priority is in an executable state during the execution of the task A, the task A being executed is interrupted and the task B is executed. Thereafter, when the processing of task B is completed or waits, the RTOS resumes the processing of task A that has been temporarily suspended.

  The interrupt controller 46_1 receives an interrupt processing request from each unit, and plays a role of traffic control to transmit an interrupt signal to the CPU 45_1 according to the priority. By the operation of the interrupt controller 46_1, the CPU 45_1 can cope with a plurality of interrupt factors with one interrupt input terminal.

  The built-in memory 47_1 is a high-speed memory for storing data when the CPU 45_1 executes a program. For example, an SRAM is used. The built-in memory 47_1 can be accessed at a higher speed than the first memory 17 which is the main memory. Note that the CPU 45_1 may make part of the application program resident in the built-in memory 47_1 and perform these processes at high speed.

  The display unit 21 is connected to the display controller 48_1. The display controller 48_1 takes in the data of the above-described through image and freeze image and displays them on the display unit 21.

  For example, the GPIO unit 49_1 inputs signals from the operation buttons 25 and the release button 24 and outputs control signals for the input signals to peripheral devices in the electronic camera. Here, the above-mentioned peripheral devices are, for example, a lens, a mechanical shutter in the photographing optical system 11, a motor driver IC for driving a diaphragm, a charging IC and a light emission control IGBT included in a flash device, and an electronic camera. These are LEDs for displaying the status (note that illustration of peripheral devices is omitted).

  The first memory 17 is connected to the memory controller 50_1. The memory controller 50_1 reads and writes data from and to the first memory 17.

  The first flash memory 19 is connected to the flash memory controller 51_1. The flash memory controller 51_1 reads and writes data from and to the first flash memory 19.

  The external I / F connector 22 is connected to the external I / F controller 52_1. The external I / F controller 52_1 performs data communication with an external device (such as a personal computer) connected to the external I / F connector 22 using an interface (for example, USB) of a predetermined communication standard.

  The media connector 23 is connected to the media controller 53_1. The media controller 53_1 reads and writes data from and to the recording medium 34 connected to the media connector 23.

The audio I / F 54_1 receives the audio data sent from the data distributor 13. Note that a standard interface such as I ² S is used for the audio I / F 54_1.

  The bus bridge 56_1 is connected to the internal bus 55_1 and the serial bus 16. The bus bridge 56_1 performs protocol conversion between the internal bus and the external serial bus. Accordingly, an access (READ / WRITE) request from the internal bus 55_1 of the first image processing circuit 14 is transmitted to the serial bus 16 as a packet (command, address, WRITE data). Conversely, a packet (response, READ data) sent from the serial bus 16 is transmitted to the internal bus 55_1 of the first image processing circuit 14. Therefore, the CPU 45_1 of the first image processing circuit 14 can access the resources of the first image processing circuit 14 and the second image processing circuit 15 with the same protocol by the operation of the bus bridge 56_1. Access to the resource of the first image processing circuit 14 and access to the resource of the second image processing circuit 15 are different in access time, and access to the resource of the second image processing circuit 15 takes more time. The access from the DMAC 44_1 of the first image processing circuit 14 is the same as described above. The access from the CPU 45_2 and the DMAC 44_2 of the second image processing circuit 15 to the above-mentioned resources is the same. In this case, the access to the resources of the first image processing circuit 14 takes longer.

  Next, the configuration of the second image processing circuit 15 will be described. The second image processing circuit 15 will be described only with respect to differences from the first image processing circuit 14, and the description of the overlapping parts will be omitted. In the second image processing circuit 15 of the present embodiment, the second memory 18 is connected to the memory controller 50_2, and the second flash memory 20 is connected to the flash memory controller 51_2. The display controller 48_2, the GIPO unit 49_2, the external I / F controller 52_2, and the media controller 53_2 are set so as not to operate with low power consumption because there is no external device to be connected.

  Here, the resource of the first image processing circuit 14 and the resource of the second image processing circuit 15 are mapped in the address space of the first image processing circuit 14. Similarly, the resource of the first image processing circuit 14 and the resource of the second image processing circuit 15 are mapped to the address space of the second image processing circuit 15, respectively. In addition to the internal resources of each image processing circuit, these resources include the resources of the first memory 17 and the second memory 18 connected to the outside, and the resources of the first flash memory 19 and the second flash memory 20. included.

  FIG. 3 schematically shows an example of a memory map of the first image processing circuit 14 and the second image processing circuit 15. The resources of the first image processing circuit 14 include the first memory 17, the first flash memory 19, the built-in memory 47_1, Peripheral_1, and the like. The resources of the second image processing circuit 15 include the second memory 18, the second flash memory 20, the built-in memory 47_2, Peripheral_2, and the like. The “Peripheral” described above is a general term for internal resources excluding the internal memory 47 in each image processing circuit. For example, in the address area described as “Peripheral”, the registers such as the image processing unit 41 and the still image compression unit 42, the buffer memory built in the moving image compression unit 43, the display controller 48, the media controller 53, and the like. Each FIFO is mapped. In the example of FIG. 3, the resources of the second image processing circuit 15 are mapped above the address space of the first image processing circuit 14, and the first image processing is positioned above the address space of the second image processing circuit 15. The resources of the circuit 14 are mapped.

  The first image processing circuit 14 and the second image processing circuit 15 use the functions of the bus bridge 56 and the serial bus 16 to transmit packets such as commands, addresses, data, and responses to the connection destination image processing circuit, respectively. be able to. Each image processing circuit can directly access not only its own resources but also the resources of the connected image processing circuit according to each memory map. For example, the first image processing circuit 14 can directly access image data on the second memory 18 connected to the second image processing circuit 15 according to the memory map of the first image processing circuit 14. Conversely, the second image processing circuit 15 can directly access image data on the first memory 17 connected to the first image processing circuit 14 according to the memory map of the second image processing circuit 15.

In other words, in the present embodiment, the image data stored in the memory connected to the transfer source image processing circuit can be directly read and processed by the transfer destination image processing circuit. For example, the DMAC 44_1 of the first image processing circuit 14 directly reads the freeze image data stored on the second memory 18 connected to the second image processing circuit 15 and displays it in the first image processing circuit 14. The data can be transferred to the controller 48_1 for display, and the data of a JPEG image file (described later) that is also stored in the second memory 18 is directly read out to the media controller 53_1 in the first image processing circuit 14. It can be transferred and recorded on the recording medium 34. That is, the operation that is performed in the conventional electronic camera, such as temporarily moving (or copying) the data to the first memory 17 connected to the first image processing circuit 14 that is the transfer destination, becomes unnecessary. For this reason, in the present embodiment, even when image data is transferred between the first image processing circuit 14 and the second image processing circuit 15, extra writing to the memory connected to the destination image processing circuit is performed. There is an advantage that it does not occur.
Accordingly, the bandwidth of the memory connected to the image processing circuit at the transfer destination is not consumed excessively, and the time required for moving (or copying) the image data between the memories does not occur, so that the processing time (or Latency) is not generated. Is shortened.

  In addition, since the first image processing circuit 14 and the second image processing circuit 15 execute processing in parallel, when communicating with each other, data can be transmitted and received using a well-known serial port. Communication may be performed using the serial bus. The data transmitted from the communication source may be written in the built-in memory 47 in the communication destination image processing circuit so that the CPU 45 of the communication destination image processing circuit can read at high speed. In order to notify the image processing circuit of the communication destination that the data has been transmitted, an interrupt can be performed using an interrupt signal and a parallel port assigned to an external terminal of the image processing circuit. You may use the bus. In this case, the communication source image processing circuit (CPU 45) directly writes to the register of the interrupt controller 46 in the communication destination image processing circuit through the serial bus, and issues an interrupt to the communication destination CPU 45.

  In addition, the image processing circuit cannot access some resources from another image processing circuit so that a malfunction does not occur due to another image processing circuit inadvertently accessing the resource of the connection destination image processing circuit. It is good to do so. For example, an address range that can be accessed from the outside is restricted, and when an address in the restricted access range is accessed from another image processing circuit, an error response is returned to the accessing image processing circuit. To. The address range restriction function described above is implemented in the bus bridge 56, for example.

<Description of Operation Example of Imaging Device>
Subsequently, the operation of the electronic camera according to the present embodiment will be specifically described with reference to three cases of single shooting and continuous shooting in the still image shooting mode and moving image shooting in the moving image shooting mode.

(Single-shot shooting operation in still image shooting mode)
When single shooting is performed in the still image shooting mode, the first image processing circuit 14 is in an operating state to perform single shooting processing, while the second image processing circuit 15 is in a power-off state or has low power consumption. It has stopped in a state. Then, the CPU 45_1 of the first image processing circuit 14 controls the overall operation of single shooting. Note that the processes described below are all performed under the control of a program executed by the CPU 45_1.

  The shooting mode of the electronic camera starts with the live view operation, not limited to single shooting. In the live view, the imaging unit 12 is caused to perform a through image imaging operation. The imaging unit 12 repeatedly captures a through image at a predetermined cycle (frame cycle), and outputs the image data to the outside of the imaging unit 12. The through image data periodically output from the imaging unit 12 is sequentially input to the first image processing circuit 14 via the data distributor 13.

  The image processing unit 41_1 performs image processing (pre-process and post-process) on the input data to generate YCbCr data of a through image. The generated through-image YCbCr data is temporarily stored in a data storage area provided on the first memory 17. The CPU 45_1 sets the start address of the data storage area where YCbCr data is stored as the source address of the DMAC 44_1, the address of the display controller 48_1 (internal FIFO) as the destination address, and the number of bytes of YCbCr data as the number of data transfers. Start the transfer. When the data transfer is started, the YCbCr data of the through image is read out and input to the display controller 48_1, and then sent to the display unit 21 and displayed on the screen. As a result, a live image of the subject can be viewed on the screen of the display unit 21 (live view display). The user determines the composition of shooting by observing the through image displayed on the display unit 21 and determines the shutter chance (timing of shooting instruction by full pressing operation of the release button 24).

  Further, during the live view operation, the image processing unit 41_1 generates an AF evaluation value, an AE evaluation value, and an AWB evaluation value from the pre-processed through image data. The CPU 45_1 performs photometry using the generated AE evaluation value, and performs exposure control of the through image based on the photometry result. As a result, the brightness of the through image is adjusted to an appropriate level. The CPU 45_1 executes the AWB algorithm using the AWB evaluation value described above, and adjusts the white balance of the through image based on the result. As a result, the color of the through image is also adjusted to an appropriate state. Further, when the release button 24 is half-pressed, the CPU 45_1 performs AF by a known contrast detection method using the above-described AF evaluation value, and focuses the photographing optical system 11 on the target subject. When the release button 24 is pressed halfway, exposure control parameters (ISO sensitivity value, aperture value, shutter speed, etc.) at the time of still image shooting are also determined.

  Then, when the release button 24 is fully pressed, the CPU 45_1 switches the imaging unit 12 from the through-image imaging operation to the still-image imaging operation to capture a still image of the subject. At this time, the CPU 45_1 performs exposure according to the exposure control parameter for still image shooting determined according to the half-press operation. When the exposure ends, the CPU 45_1 shifts the imaging unit 12 to a still image reading operation and outputs image data of the captured still image. The still image data output from the imaging unit 12 is input to the first image processing circuit 14 via the data distributor 13.

  The still image data (Bayer data) input to the first image processing circuit 14 is first preprocessed only by the image processing unit 41_1, and then output from the image processing unit 41_1 and provided on the first memory 17. Temporarily stored in the Bayer data storage area. Subsequently, the CPU 45_1 uses the start address of the Bayer data storage area where the Bayer data is stored as the source address of the DMAC 44_1, the address of the image processing unit 41_1 (internal FIFO) as the destination address, and the number of bytes of Bayer data as the number of data transfers. To start data transfer. When the data transfer is started, the preprocessed Bayer data is read from the above-described Bayer data storage area and enters the image processing unit 41_1, and the image processing unit 41_1 performs the post process and takes a still image. A main image (YCbCr data) of the image is generated. At this time, the image processing unit 41_1 uses the internal resolution conversion circuit (not shown) to generate the freeze image YCbCr data and the thumbnail YCbCr data together. The YCbCr data of the main image, the freeze image, and the thumbnail are provided in the first memory 17 as a YCbCr data storage area (including three storage areas: a main image storage area, a freeze image storage area, and a thumbnail storage area). Temporarily stored. The CPU 45_1 uses the start address of the freeze image storage area in which the freeze image YCbCr data is stored as the source address of the DMAC 44_1, the address of the display controller 48_1 (internal FIFO) as the destination address, and the freeze image YCbCr data as the number of data transfers. Then, the transfer of the YCbCr data of the freeze image is started immediately. When the data transfer is started, the YCbCr data of the freeze image is read from the YCbCr data storage area described above, enters the display controller 48_1, and is then sent to the display unit 21 and displayed on the screen. Thereby, the user can confirm the state of the captured image immediately after shooting.

  Subsequently, the above-described main image, freeze image, and thumbnail YCbCr data are sequentially subjected to JPEG compression using the still image compression unit 42_1. In order to transfer the three image data one by one to the still image compression unit 42_1, the CPU 45_1 each time the source address of the DMAC 44_1 (the start address of each YCbCr data in the YCbCr data storage area), the destination address (the still image compression unit 42_1). (Internal FIFO) address) and the number of data transfers (number of bytes of each YCbCr data) are set, and data transfer is performed three times in total. Three pieces of YCbCr data are read one by one from the YCbCr data storage area described above, enter the still image compression unit 42_1, and are subjected to JPEG compression individually by the still image compression unit 42_1. (Hereinafter referred to as JPEG data) is temporarily stored in a JPEG data storage area provided on the first memory 17. The CPU 45_1 combines the JPEG data of the main image, the freeze image, and the thumbnail together with various metadata on the JPEG data storage area according to a predetermined file format, and combines one JPEG image file (memory image of the file). Generate. As the file format, an Exif format, a multi-picture format (MPF), or the like is used. The JPEG data of the freeze image is used for high-speed display during image reproduction, while the JPEG data of the thumbnail is used for searching for JPEG image files. Finally, the CPU 45_1 records the JPEG image file (memory image) stored in the JPEG data storage area on the recording medium 34. The CPU 45_1 sets the start address of the JPEG image file as the source address of the DMAC 44_1, the address of the media controller 53_1 (internal FIFO) as the destination address, and the number of bytes of the JPEG image file as the data transfer number, and starts data transfer. When the data transfer is started, the data of the JPEG image file (memory image) is read from the above-described JPEG data storage area and enters the media controller 53_1, and then sent to the recording medium 34 to be recorded in the recording medium 34. To be recorded.

  When the recording operation of the JPEG image file to the recording medium 34 is completed, the single shooting (one frame still image shooting) processing is completed, and the CPU 45_1 returns the electronic camera to the live view operation again, and the next single shooting is completed. Prepare for photo shoots. The above is the basic operation of single shooting in the still image shooting mode.

(Operation example for continuous shooting in still image shooting mode)
Next, a case where continuous shooting is performed in the still image shooting mode will be described. In the case of continuous shooting, shooting processing is performed in parallel by the first image processing circuit 14 (master side) and the second image processing circuit 15 (slave side). The first image processing circuit 14 starts the live view operation as in the case of single shooting, while the second image processing circuit 15 completes various settings necessary for still image shooting and then performs continuous shooting. Wait until shooting starts. At this time, it is preferable that the second image processing circuit 15 stops supplying clocks to circuit blocks other than the necessary circuit blocks and stands by in a low power consumption state. The first image processing circuit 14 and the second image processing circuit 15 need to be in a state where data can be transmitted and received with each other through the external serial bus 16.

  The live view operation by the first image processing circuit 14 in continuous shooting is almost the same as in the case of single shooting including the processing when the release button 24 is half-pressed. Description of is omitted. When the release button 24 is fully pressed subsequent to the half-press operation, the continuous shooting of the still image is started, but the subsequent processing is different from the case of the single shooting.

  While the release button 24 is fully pressed (during continuous shooting), still image shooting is repeated. During the continuous shooting, still image shooting is continued with the focus state fixed using the exposure parameter for still image shooting determined at the time of half-pressing operation. In addition, during continuous shooting, a live view is not displayed, and the live view operation is not performed.

  In the case of continuous shooting, when the release button 24 is fully pressed, the CPU 45_1 supplies the clock to the operating state (all circuit blocks to be used) from the low power consumption state of the second image processing circuit 15. State). In addition, the CPU 45_1 switches the imaging unit 12 from the through image imaging operation to the still image imaging operation to capture a still image of the subject. The CPU 45_1 performs exposure according to the above-described exposure control parameter. When the exposure ends, the CPU 45_1 shifts the imaging unit 12 to a still image reading operation and outputs image data of the captured still image. Note that the CPU 45_1 cannot output image data when there is no available Bayer data storage area provided on the memory, as will be described later. Therefore, the CPU 45_1 obtains one available Bayer data storage area in advance and then stores it in the imaging unit 12. Start exposure.

  In the case of continuous shooting, the still image data of the odd frames (1, 3, 5, 7,...) Including the first captured frame is processed by the first image processing circuit 14 and the even frames (2, 4, 4, etc.) are processed. 6, 8,...) Are processed by the second image processing circuit 15. Therefore, the data distributor 13 outputs the odd frame still image data to the first image processing circuit 14 and outputs the even frame still image data to the second image processing circuit 15 (see FIG. 4).

  The still frames of the even and odd frames described above are taken by the same imaging unit 12, and the imaging operation of the imaging unit 12 is controlled by the CPU 45_1 on the master side (first image processing circuit 14). Therefore, the CPU 45_1 can grasp all the shooting order of each still image shot by continuous shooting. Therefore, the CPU 45_1 assigns a frame number (serial number) indicating the shooting order to each still image during continuous shooting. Then, the shooting order of still image data is managed by this frame number.

  Here, as in the case of single-shot shooting, the basic processing performed in continuous shooting is “still image shooting operation” by the imaging unit 12, “pre-processing” by the image processing unit 41, and the image processing unit 41. Are “post-process”, “JPEG compression processing” by the still image compression unit 42, and “JPEG image file recording” by the media controller 53. Since these five processes are basically the same as in the case of single shooting, the description of the overlapping parts is omitted. However, “recording of a JPEG image file” in the second image processing circuit 15 will be described later because there is a difference from the case of single shooting.

  In the case of the above-described single shooting, the next frame is shot after the five processes described above are executed sequentially. On the other hand, in the case of continuous shooting, the above-described five processes are performed in parallel, and while a certain process is being performed on still image data of a certain frame, The difference is that separate processing is performed on the still image data. That is, in the case of continuous shooting, shooting of the next frame can be started before the five processes in shooting of one frame have not yet been completed. If attention is paid to individual frames that are processed in parallel during continuous shooting, the above-described five processes are executed sequentially.

  The reason why parallel processing as described above is possible is that the internal resources (hardware) of the image processing circuits (14, 15) used in each processing are different. For example, since the pre-process unit (not shown) and the host process unit in the image processing unit 41 are independent circuits, pre-process and post-process are performed in parallel for still image data of two consecutive frames. Can be applied. Similarly, since the still image compression unit 42 and the media controller 53 are also independent circuits, if these four circuits are used, still image data of four consecutive frames can be processed in parallel. Of the five processes described above, the output of image data included in the “still image capturing operation” and the “preprocess” are the same when the image capturing unit 12 and the preprocess unit are continuously operated. The frame is always executed integrally. Therefore, when considering parallel processing in continuous shooting, the above-described “still image capturing operation” and “preprocess” processing are collectively treated as one processing.

  Specific operations in the parallel processing described above are as follows. In the following description of the parallel processing, four consecutive frames processed by the same image processing circuit are referred to as a first frame to a fourth frame.

  First, “still image capturing operation” and “preprocess” processing are executed independently for the first frame of continuous shooting. Subsequently, the first frame shifts to a “post process” process, and in parallel with this, a “still image capturing operation” and a “pre process” process are executed for the second frame. Subsequently, the first frame shifts to “JPEG compression processing”, the second frame shifts to “post process” processing, and in parallel with this, “still image capturing operation” is performed on the third frame. A “preprocess” process is executed. Subsequently, the first frame proceeds to “JPEG image file recording” processing, the second frame proceeds to “JPEG compression processing” processing, and the third frame proceeds to the “post processing” processing described above. In parallel, the “still image capturing operation” and “preprocess” processes are executed for the fourth frame. Thereafter, the photographing process is completed in order from the first frame, and the photographing process is started for a new frame.

  When the above-described continuous shooting operation reaches a steady state, the shooting interval (frame interval) is determined by the most time-consuming processing among the parallel processing in continuous shooting and becomes equal to the processing time. That is, shooting of one frame of a still image is completed for each processing time described above (pipeline processing).

  Further, when performing the parallel processing described above, it is necessary to secure a plurality of data storage areas (Bayer data storage area, YCbCr data storage area, and JPEG data storage area) described in the single-shot shooting on the memory. In the following description, the case of the first frame and the second frame will be specifically described, but the situation is the same for the subsequent frames.

  For example, the first frame still image data (Bayer data) output from the imaging unit 12 is first preprocessed by the image processing unit 41 and then output from the image processing unit 41 to store the Bayer data on the memory. It is temporarily stored in the storage area. The Bayer data of the first frame after the preprocessing is post-processed by the image processing unit 41 to generate the main image (YCbCr data) of the first frame. At that time, the image processing unit 41 uses the internal resolution conversion circuit (not shown) to generate the freeze image YCbCr data and the thumbnail YCbCr data together. The YCbCr data of the first frame is temporarily stored in the YCbCr data storage area on the memory. On the other hand, during the post process for the first frame, the Bayer data of the second frame is output from the imaging unit 12, and only the preprocessing is performed by the image processing unit 41 and temporarily stored in the Bayer data storage area. It will be. At this time, if there is only one Bayer data storage area, the Bayer data of the first frame is overwritten with the Bayer data of the second frame, and normal processing cannot be performed. Therefore, it can be seen that at least two Bayer data storage areas are required to avoid overwriting the data described above.

  Further, for example, the above-described YCbCr data of the first frame is subjected to JPEG compression processing by the still image compression unit 42 to generate JPEG data of the first frame. The JPEG data of the first frame is temporarily stored in a JPEG data storage area on the memory. On the other hand, during the execution of the JPEG compression process for the first frame, the post-process is performed on the Bayer data of the second frame by the image processing unit 41, and the YCbCr data of the second frame is generated. The YCbCr data of the second frame is temporarily stored in the YCbCr data storage area on the memory. At this time, if there is only one YCbCr data storage area, the YCbCr data of the first frame is overwritten with the YCbCr data of the second frame, and normal processing cannot be performed. Therefore, it can be seen that at least two YCbCr data storage areas are necessary to avoid overwriting the data described above.

  Further, for example, the JPEG data of the first frame described above is added with various metadata by the CPU 45 and combined into one on the JPEG data storage area to generate a memory image of one JPEG image file. . Then, the memory image of the JPEG image file of the first frame is read and enters the media controller 53_1, and then sent to the recording medium 34 and recorded in the recording area inside the recording medium 34. On the other hand, while the JPEG image file of the first frame is being recorded, the JPEG compression processing is performed on the YCbCr data of the second frame by the still image compression unit 42, and the JPEG data of the second frame is generated. . The JPEG data of the second frame is temporarily stored in the JPEG data storage area on the memory. At this time, if there is only one JPEG data storage area, the JPEG data of the first frame is overwritten with the JPEG data of the second frame, and normal processing cannot be performed. Therefore, it can be seen that at least two JPEG data storage areas are required to avoid overwriting the data described above.

  For the reasons described above, two or more Bayer data storage areas, YCbCr data storage areas, and JPEG data storage areas are secured in the memories (17, 18) of the image processing circuit (see FIG. 4). The Bayer data storage area is shared by the preprocess and the postprocess. The YCbCr data storage area is shared by the post process and the JPEG compression process. The JPEG data storage area is shared by JPEG compression processing and JPEG image file recording processing. In order to avoid overwriting data in each data storage area, data is written to one of the two shared data storage areas in the former process, and data is read from the other of the two shared data storage areas in the latter process.

  When performing the parallel processing described above, tasks corresponding to “still image capturing operation”, “pre-process”, “post-process”, “JPEG compression processing”, and “record JPEG image file” are managed by RTOS. Runs in parallel below. Each task synchronizes between tasks by communicating between tasks, and performs exclusive control of writing and reading of data in each data storage area.

  For example, when the release button is fully pressed, the imaging processing task corresponding to the “still image imaging operation” transmits a message to the “preprocessing” task to convey information on the frame number and “preprocessing”. Request preparation for processing. When the above-described message is received, the “preprocess” task acquires an empty Bayer data storage area (acquires a semaphore), and returns a response indicating completion of preparation to the imaging processing task after acquisition of the Bayer data storage area. When the above-described response is received, the imaging processing task causes the imaging unit 12 to start a still image imaging operation. When exposure is completed and still image data is output from the imaging unit 12, the data is input to a predetermined image processing circuit (14, 15), and preprocessing is performed in the preprocessing unit. The DMAC 44 that writes the pre-processed Bayer data uses the pre-process unit (internal FIFO) as the source address, and uses the pre-acquired start address of the Bayer data storage area as the destination address, to the Bayer data storage area by DMA transfer. Write the Bayer data. When the writing of the Bayer data for one frame is completed, the “preprocess” task returns the Bayer data storage area (semaphore return).

  Subsequently, the “pre-process” task sends a message to the “post-process” task, and requests that the above-described Bayer data be post-processed. At this time, the “preprocess” task includes the frame number information and the start address of the Bayer data storage area in the message to the “postprocess” task. The “post process” task that has received the above message acquires a Bayer data storage area of the received address value and a free YCbCr data storage area (acquisition of a semaphore). After acquiring the YCbCr data storage area, the “post process” task causes the post process section to start the post process. The DMAC 44 that reads out the Bayer data from the Bayer data storage area uses the previously acquired Bayer data storage area start address as the source address, the post process part (internal FIFO) as the destination address, and the DMA data transfer to transfer the Bayer data to the post process part. To enter. As a result, the Bayer data is input to the post-processing unit and subjected to post-processing. Further, the DMAC 44 that writes the post-process YCbCr data stores the YCbCr data by DMA transfer using the post-process unit (internal FIFO) as the source address and the head address of the YCbCr data storage area acquired in advance as the destination address. Write YCbCr data (main image, freeze image, thumbnail) in the area. When the reading of the Bayer data for one frame is completed, the “post process” task returns the Bayer data storage area (semaphore return). When the writing of YCbCr data for one frame is completed, the “post process” task returns the YCbCr data storage area (semaphore return).

  Subsequently, the “post process” task sends a message to the “JPEG compression process” task and requests that the above-described YCbCr data be subjected to the JPEG compression process. At this time, the “post process” task includes the frame number information and the start address of the YCbCr data storage area in the message to the “JPEG compression process” task. The “JPEG compression process” task that has received the above-mentioned message acquires a YCbCr data storage area of the received address value and a free JPEG data storage area (acquisition of semaphore). After acquiring the JPEG data storage area, the task “JPEG compression processing” causes the still image compression unit 42 to start JPEG compression processing. The DMAC 44 that reads YCbCr data from the YCbCr data storage area uses the YCbCr data storage area acquired in advance as the source address, the still image compression unit 42 (internal FIFO) as the destination address, and the YCbCr data is statically transferred by DMA transfer. Input to the image compression unit 42. Thus, the YCbCr data is input to the still image compression unit 42 and subjected to JPEG compression processing. The DMAC 44 that writes JPEG data after JPEG compression processing uses DMA transfer with the still image compression unit 42 (internal FIFO) as the source address and the pre-acquired start address of the JPEG data storage area as the destination address. Write JPEG data in the JPEG data storage area. In parallel with this, the task of “JPEG compression processing” collects metadata such as shooting information and writes the metadata to another address in the JPEG data storage area. The “JPEG compression processing” task combines one JPEG data and metadata output from the still image compression unit 42 into one JPEG image file (memory image) starting from the top of the JPEG data storage area. ). When the reading of YCbCr data for one frame is completed, the “JPEG compression processing” task returns the YCbCr data storage area (semaphore return). When the above-described JPEG image file is created, the “JPEG compression process” task returns the JPEG data storage area (semaphore return).

  Subsequently, the “JPEG compression process” task sends a message to the “record JPEG image file” task and requests the above-mentioned JPEG image file to be recorded on the recording medium 34. At this time, the “JPEG compression processing” task includes the frame number information, the start address of the JPEG data storage area, and the size of the JPEG image file in the message to the “record JPEG image file” task. . Since JPEG compression is variable length compression, the size (code amount) of JPEG data also changes with each compression process, so the size of the JPEG image file also changes. For this reason, the size of the JPEG image file is included in the message in the above case. The “JPEG image file recording” task that has received the above message acquires the JPEG data storage area of the received address value (acquisition of a semaphore). Then, the task of “record JPEG image file” sets the media controller 53_1 to the operation state (Active). Also, the “record JPEG image file” task issues a WRITE command and a write address to the recording medium 34 via the media controller 53_1, and starts a JPEG image file recording operation. The DMAC 44 that reads a JPEG image file from the JPEG data storage area uses the JPEG data storage area acquired in advance as a source address and the media controller 53_1 (internal FIFO) as a destination address. Is input to the media controller 53_1. The media controller 53_1 transmits the data of the input JPEG image file to the recording medium 34, and records it in the recording area specified by the write address described above. When the recording of the JPEG image file is completed, the “record JPEG image file” task returns the JPEG data storage area (semaphore return).

  As described above, by using the RTOS function (semaphore), the CPU 45 can reliably perform exclusive control of writing and reading of data in each data storage area. Further, since frame number information is transmitted by inter-task communication, the RTOS executed by the CPU 45 can process still image data according to the shooting order by referring to the frame number information.

  In the electronic camera of the present embodiment, the parallel processing described above is executed by the first image processing circuit 14 for the still image data of odd frames, and the second image processing circuit 15 for the still image data of even frames. The parallel processing described above is executed. When two image processing circuits are used, two frames can be processed during the same processing time, so that the processing time is relatively halved. That is, since the frame interval of the entire electronic camera is halved, continuous shooting can be performed twice as fast as when processing is performed by one image processing circuit.

  By the way, in the present embodiment, the media connector 23 is connected only to the first image processing circuit 14. Therefore, the JPEG image file generated by the first image processing circuit 14 and the JPEG image file generated by the second image processing circuit 15 are alternately placed on the recording medium 34 in the order of frame numbers under the control of the CPU 45_1 on the master side. Will be recorded.

  The recording process of the JPEG image file on the slave side is basically the same as that on the master side, but the data of the JPEG image file is recorded in the JPEG data storage area of the second memory 18 and the data is This is different from the case of the master side in that it is transferred to the block of the first image processing circuit 14 via the serial bus 16. However, the CPU 45_1 and the DMAC 44_1 of the first image processing circuit 14 can access the data on the second memory 18 similarly to the data on the first memory 17 by the operation of the bus bridge 56. Therefore, it is easy to transfer the JPEG image file generated by the second image processing circuit 15 to the block of the first image processing circuit 14.

  For example, the electronic camera may record the slave-side JPEG image file on the recording medium 34 according to the following procedure. FIG. 5 is a schematic diagram when a slave-side JPEG image file is recorded on the recording medium 34. In FIG. 5, the data flow of the JPEG image file is indicated by a dashed line arrow.

  First, each time the JPEG image file data is generated, the slave CPU 45_2 sends the address of the JPEG data storage area in which the target JPEG image file is stored to the master CPU 45_1 and the JPEG image. Tell the file frame number. When the CPU 45_1 records the JPEG image file of the frame number stored in the second memory 18 on the recording medium 34, the CPU 45_1 uses the start address of the JPEG data storage area acquired by the above-described communication to select the target JPEG image file. What is necessary is just to transfer to the recording medium 34. Note that the specific processing relating to the recording of the JPEG image file on the recording medium 34 is the same as that already described above, and therefore, redundant description is omitted.

  Also, consider a case where a freeze image is displayed during continuous shooting. In the present embodiment, the display unit 21 is connected only to the first image processing circuit 14. In the case where an odd-numbered frame freeze image is displayed on the display unit 21, the master side may perform the same control as in the case of single shooting. When displaying the freeze image of the even number frame on the display unit 21, it is basically the same as the case of the master side, but the YCbCr data of the freeze image is recorded in the YCbCr data storage area of the second memory 18, This is different from the case of the master in that the data is transferred to the block of the first image processing circuit 14 via the serial bus 16. However, the CPU 45_1 and the DMAC 44_1 of the first image processing circuit 14 can access the data on the second memory 18 similarly to the data on the first memory 17 by the operation of the bus bridge 56. Therefore, it is easy to transfer the freeze image data generated by the second image processing circuit 15 to the block of the first image processing circuit 14.

  For example, the electronic camera may display a freeze image according to the following procedure. FIG. 6 is a schematic diagram when a freeze image on the slave side is displayed on the display unit 21. In FIG. 6, the data flow of the freeze image is indicated by a one-dot chain line arrow.

  At the time of continuous shooting, the CPU 45_1 on the master side obtains a frame number for displaying a freeze image. If the frame number is an even number, the freeze image data to be displayed is stored in the YCbCr data storage area of the second memory 18. When displaying the freeze image, the CPU 45_1 transmits a display flag indicating the display of the freeze image together with the frame number to the CPU 45_2 in advance when shooting the frame.

  Here, when the Bayer data of the even frame is input to the second image processing circuit 15, the CPU 45_2, the address of the Bayer data storage area of the second memory 18 in which the Bayer data is stored, the frame number, the display flag, Is transmitted to the image processing unit 41_2. Then, the image processing unit 41_2 performs post-processing on the above Bayer data to generate main image, freeze image, and thumbnail YCbCr data. When the display flag is set, the image processing unit 41_2, when the YCbCr data of the freeze image is generated, the start address of the YCbCr data storage area in which the freeze image to be displayed is stored and its frame The number is transmitted to the CPU 45_1 on the master side.

  Then, the master CPU 45_1 DMA-transfers the freeze image data from the second memory 18 to the display controller 48_1. At this time, the CPU 45_1 sets the start address of the YCbCr data storage area where the freeze image to be displayed is recorded as the source address of the DMAC 44_1, and sets the FIFO address of the display controller 48_1 as the destination address of the DMAC 44_1. The CPU 45_1 may set the number of bytes of the freeze image to be displayed as the transfer count of the DMAC 44_1.

  Thereafter, when the CPU 45_1 calls the display driver, the freeze image data is transferred to the display controller 48_1, and the freeze image is displayed on the screen of the display unit 21. When the same freeze image is continuously displayed on the display unit 21 for a certain period, the DMAC 44_1 may be set to repeat the above-described freeze image data transfer. Therefore, the DMAC 44_1 may have a reload function for performing DMA transfer with the same address as the previous time.

  The above is the operation when continuous shooting is performed in the still image shooting mode.

(Operation example 1 in movie shooting mode)
Next, an operation example of the electronic camera in the moving image shooting mode will be described. First, H. The encoding of moving images in the H.264 standard will be briefly described.

  H. In the encoding of a moving image by H.264, one frame of moving image data is divided into a large number of blocks called macroblocks, and encoding processing is performed in units of macroblocks. There are three modes of encoding, called I mode, P mode, and B mode.

  The I mode is an intraframe predictive coding mode. In the I mode, intra-frame prediction is performed in which each macroblock is encoded using only pixels in the current frame to be encoded.

  On the other hand, the P mode and the B mode are inter-frame prediction encoding modes. In general, in the P mode, a current frame is predicted from one temporally past frame. In the B mode, the current frame is predicted from one frame in the past and the future in terms of time. Here, a frame that is compared with the current frame in inter-frame prediction in the P mode and the B mode is referred to as a reference frame. As this reference frame, a frame that has been reproduced (decoded) from an already encoded frame is used. H. The H.264 encoder includes a local decoder. The encoded macroblock of the current frame is immediately decoded by the local decoder and stored in the reference frame storage area on the memory.

  In inter-frame prediction, the value of the prediction error is reduced by performing motion compensation by detecting the motion of the subject. That is, in motion detection, a search area larger than the macroblock is read from the reference frame based on the position of the macroblock in the current frame. Then, the position of the block that minimizes the above-described prediction error in the search area and the prediction error (minimum value) at that time are obtained. A difference (offset) between the position of the block obtained by the motion detection and the reference position is referred to as a motion vector. In inter-frame prediction, a prediction error and a motion vector are encoded together to generate compressed macroblock data including header information. When interframe prediction is performed, the compressed data of the macroblock of the current frame that has been encoded is immediately decoded by the local decoder, and a new reference frame (decodes the current frame) from the decoded prediction error and motion vector. Is generated).

  H. In the H.264 standard, one frame is divided by a group of macroblocks called slices, and the coding mode can be changed in units of slices.

  H. The H.264 standard can use a larger number of reference frames than the conventional MPEG standard. In the case of a P macroblock, one frame is selected from one reference frame group and P mode encoding is performed. In the case of a B macroblock, one frame is selected from two reference frame groups, or two frames are selected from one reference frame group, and B-mode encoding is performed.

  By the way, when selecting the above frame, it is highly possible to select a frame having a value closer to the macroblock (current macroblock) of the current frame after evaluating a plurality of reference frames. In addition, since the reference frame changes when the slice changes, the above-described evaluation of the reference frame is performed each time. Therefore, H.H. In the H.264 standard, the number of access times of the reference frame is larger than that in the conventional MPEG, and the memory bandwidth can be greatly used.

  Based on the above description, the operation of the electronic camera in the moving image shooting mode in this embodiment will be described.

  FIG. 7 is a schematic diagram of an operation example 1 of the electronic camera in the moving image shooting mode. In FIG. 7, the data flow of the current frame (pointing to moving image data for recording), the reference frame, and the moving image file are indicated by alternate long and short dashed arrows.

  In the operation example 1 illustrated in FIG. 7, the moving image compression unit 43_1 on the master side encodes a moving image. The first memory 17 stores YCbCr data of the current frame and a moving image file. On the other hand, the YCbCr data of the reference frame is stored in the second memory 18. In operation example 1, through image data and moving image data are input only to the first image processing circuit 14 via the data distributor 13.

  At the stage of accepting the activation instruction for the moving image shooting mode, the first image processing circuit 14 starts the live view operation as in the case of single shooting, while the second image processing circuit 15 is required for moving image shooting. After completing the various settings, wait until movie shooting starts. At this time, it is preferable that the second image processing circuit 15 stops supplying clocks to circuit blocks other than the necessary circuit blocks and stands by in a low power consumption state. The first image processing circuit 14 and the second image processing circuit 15 need to be in a state where data can be transmitted and received with each other through the external serial bus 16.

  Since the live view operation by the first image processing circuit 14 in moving image shooting is almost the same as that in the case of single-shot shooting of a still image, description of overlapping portions is omitted.

  Further, during the live view operation, the image processing unit 41_1 generates an AF evaluation value, an AE evaluation value, and an AWB evaluation value from the pre-processed through image data. The CPU 45_1 performs photometry using the generated AE evaluation value, and performs exposure control of the through image based on the photometry result. The CPU 45_1 executes the AWB algorithm using the AWB evaluation value described above, and adjusts the white balance of the through image based on the result. Further, the CPU 45_1 performs AF by a known contrast detection method using the AF evaluation value at a predetermined timing, and focuses the photographing optical system 11 on the target subject. Note that the CPU 45_1 performs the above-described live view display, exposure control, white balance adjustment, and AF even during moving image shooting (data flow related to live view display is not shown in FIG. 7).

  And CPU45_1 will start the recording of a moving image, if a moving image recording instruction | indication is received. When the moving image recording instruction described above is performed, the CPU 45_1 shifts the second image processing circuit 15 from the low power consumption state to the operation state (a state where clocks are supplied to all circuit blocks to be used). In addition, the CPU 45_1 instructs the imaging unit 12 to output moving image data with a combination of resolution and frame period that matches the moving image specifications. At the time of moving image shooting, the CPU 45_1 may generate YCbCr data of a through image by performing resolution conversion on preprocessed moving image data and perform the above-described live view display or the like.

  Here, the basic processing performed in moving image shooting is “moving image imaging operation” by the imaging unit 12, “pre-processing” by the image processing unit 41_1, “post-processing” by the image processing unit 41_1, and the moving image compression unit 43_1. Are “moving image compression processing” and “recording of moving image file” by the media controller 53_1. These processes are processed in parallel by the first image processing circuit 14 as in the case of the continuous shooting described above. Note that the operation from the moving image capturing operation to the post process operation is substantially the same as the still image capturing operation from the still image capturing operation in the case of continuous shooting, and thus redundant description is omitted.

  At the time of moving image shooting, under the control of the CPU 45_1, the YCbCr data of the current frame after the post process is recorded in the YCbCr data storage area of the first memory 17. When the YCbCr data of the current frame is generated, the CPU 45_1 instructs the moving image compression unit 43_1 to start moving image compression processing.

  In response to the above instruction, the moving image compression unit 43_1 requests data of the current macroblock. The DMAC 44_1 partially reads the macroblock from the current frame (YCbCr data) in the first memory 17 via the memory controller 50_1, and transfers it to the moving image compression unit 43_1. At this time, the CPU 45_1 sets the start address of the YCbCr data storage area of the first memory 17 as the source address of the DMAC 44_1, the address of the moving picture compression unit 43_1 (internal FIFO), and the number of data transfers as the destination address, respectively. Just do.

  On the other hand, the DMAC 44_1 transfers the prediction data to the moving picture compression unit 43_1 using another channel. If the current macroblock is an I macroblock, the moving picture compression unit 43_1 may perform intraframe prediction from data encoded earlier in the same slice.

  Here, when the current frame is the first frame of the moving image, there is no past frame, so the moving image compression unit 43_1 performs encoding in the I mode described above. The moving image compression unit 43_1 sequentially generates decoded data of the current macroblock in the above-described encoding process. The CPU 45_1 directly writes the decoded data (YCbCr data) of the current macroblock into the reference frame storage area secured in the second memory 18 by the DMAC 44_1. At this time, the CPU 45_1 sets the address of the moving picture compression unit 43_1 (internal FIFO) as the source address of the DMAC 44_1, sets the start address of the reference frame storage area of the second memory 18 as the destination address, and the number of data transfers. Can be done. The decoded data recorded in the reference frame storage area described above becomes a new reference frame.

  When the current frame is the second frame or later, the moving picture compression unit 43_1 can use the reference frame stored in the reference frame storage area of the second memory 18, and can include the P slice and the B slice. In the case of the P macro block and the B macro block described above, the moving image compression unit 43_1 acquires prediction data (YCbCr data in the search area of the reference frame) from the reference frame stored in the second memory 18. At this time, the CPU 45_1 sets the address of the reference frame storage area of the second memory 18 as the source address of the DMAC 44_1, the address of the moving picture compression unit 43_1 (internal FIFO) as the destination address, and the number of data transfers. Can be done. Note that in the compression of the P macroblock and the B macroblock, the moving picture compression unit 43_1 may read prediction data from a plurality of reference frames and evaluate the size of the prediction error with the current macroblock. Then, the moving picture compression unit 43_1 may select a reference frame having a smaller prediction error and use it for encoding the current macroblock.

  By the way, when compressing a moving image, the source address of the DMAC 44 is frequently changed in order to read the data of the reference frame. Therefore, the DMAC may be incorporated in the moving image compression unit 43 from the viewpoint of efficiently performing DMA transfer during moving image compression. At this time, the moving image compression unit 43 may be provided with a function of changing the source address and the destination address and a function of determining a frame to be encoded by incorporating a microcomputer in the moving image compression unit 43. According to such a configuration, each time a current frame is generated, the moving image compression unit 43 can autonomously encode a moving image simply by the CPU 45 communicating the address of the current frame to the moving image compression unit 43. It becomes.

  The moving image compression unit 43_1 converts the encoded data (moving image file) generated by intra-frame predictive encoding (I mode) or inter-frame predictive encoding (P mode, B mode) into a standard format packet including a header. Divide and output. The above-described packet is written into the moving image file storage area of the first memory 17 by the DMAC 44_1. At this time, the CPU 45_1 sets the address of the moving picture compression unit 43_1 (internal FIFO) as the source address of the DMAC 44_1, the address of the moving picture file storage area of the first memory 17 as the destination address, and the number of data transfers. Can be done. Note that the CPU 45_1 switches the recording destination to another moving image file storage area each time a certain amount of the packet data is accumulated in the moving image file storage area.

  Also, during the moving image file recording process, the moving image file data is read from the moving image file storage area in which the writing of the moving image file data has been completed and written to the recording medium 34. At this time, the CPU 45_1 sets the start address of the moving image file storage area as the source address of the DMAC 44_1, the address of the media controller 53_1 (internal FIFO) as the destination address, and the number of bytes of the moving image file packet as the data transfer number. Start data transfer. When data transfer is started, moving image file data is read from the above-described moving image file storage area, enters the media controller 53_1, and is then sent to the recording medium 34 to be recorded in the recording area inside the recording medium 34. .

  As described above, each time a certain amount of packet data is accumulated, the recording destination moving image file storage area is switched to write another packet data storage area while writing packet data to a certain moving image file storage area. The data can be read out and written into the recording medium 34. Therefore, it can be understood that the above-described five processes can be performed in parallel in the case of moving image shooting as well as in the case of continuous shooting. Above, description of the operation example 1 is complete | finished.

  In the case of playing a moving image file, since there is no motion detection, the memory bandwidth is not consumed significantly. Therefore, in the case of reproducing a moving image file, only the first image processing circuit 14 is set in an operating state. Then, the moving image compression unit 43 of the first image processing circuit 14 may decode each frame of the moving image file.

  Hereinafter, the operation and effect in the operation example 1 of the moving image shooting mode shown in FIG.

  In the operation example 1 described above, the current frame is stored in the first memory 17 and the reference frame is stored in the second memory 18. In addition, the moving image compression unit 43_1 directly accesses the reference frame of the second memory 18 and performs encoding by inter-frame prediction of the current frame.

  Therefore, in the operation example 1 described above, the bandwidth of the first memory 17 is used for pre-process / post-process processing of the current frame, display of a through image, and recording of a moving image file on the recording medium 34. . The bandwidth of the second memory 18 is used for writing / reading reference frame data with a relatively large load. Therefore, in the operation example 1 described above, access to each memory is distributed, so that efficient processing can be performed by the two image processing circuits.

  In the operation example 1 described above, the image data stored in the memory connected to the transfer source image processing circuit can be directly read and processed by the transfer destination image processing circuit. Therefore, the operation of temporarily moving (or copying) the data to the memory connected to the transfer destination image processing circuit becomes unnecessary. Further, the bandwidth of the memory connected to the image processing circuit of the transfer destination is not consumed excessively, and the time required for moving (or copying) the image data between the memories does not occur, so the processing time (or Latency) is not generated. Is shortened.

  In the description of each operation example in the moving image shooting described below, any overlapping description of the parts common to the above operation example 1 is omitted.

(Operation example 2 in movie shooting mode)
FIG. 8 is a schematic diagram of Operation Example 2 of the electronic camera in the moving image shooting mode. This operation example 2 is a modification of the operation example 1 shown in FIG. 7, and is different from the operation example 1 in that the moving image compression unit 43 of the second image processing circuit 15 encodes a moving image. In FIG. 8, the data flow of the current frame, the reference frame, and the moving image file are indicated by alternate long and short dashed arrows.

  Even in the case of moving image shooting in the operation example 2, the YCbCr data of the current frame after the post process is recorded in the YCbCr data storage area of the first memory 17 under the control of the CPU 45_1. When the YCbCr data of the current frame is generated, the CPU 45_1 instructs the moving image compression unit 43_2 to start moving image compression processing.

  In response to the above instruction, the moving picture compression unit 43_2 requests data of the current macroblock. The DMAC 44_2 partially reads the macroblock from the current frame (YCbCr data) in the first memory 17 via the memory controller 50_1, and transfers the macroblock to the moving image compression unit 43_2. At this time, the CPU 45_2 sets the start address of the YCbCr data storage area of the first memory 17 as the source address of the DMAC 44_2, the address of the moving picture compression unit 43_2 (internal FIFO) as the destination address, and the number of data transfers. Can be done.

  On the other hand, the DMAC 44_2 transfers the prediction data (reference frame data in the reference frame storage area) stored in the second memory 18 to the moving picture compression unit 43_2 using another channel. Then, the moving image compression unit 43_2 performs encoding in any of the above-described I mode, P mode, and B mode. Note that the operation of the moving image compression unit 43_2 in each of the above-described modes is the same as the operation of the moving image compression unit 43_1 in Operation Example 1, and thus description thereof is omitted.

  The moving image compression unit 43_2 converts the encoded data (moving image file) generated by intra-frame predictive encoding (I mode) or inter-frame predictive encoding (P mode, B mode) into a standard format packet including a header. Divide and output. The above-described packet is written into the moving image file storage area of the first memory 17 by the DMAC 44_2. At this time, the CPU 45_2 sets the address of the moving picture compression unit 43_2 (internal FIFO) as the source address of the DMAC 44_2, the address of the moving picture file storage area of the first memory 17 as the destination address, and the number of data transfers. Can be done. Note that the CPU 45_2 switches the recording destination to another moving image file storage area each time a certain amount of the packet data is accumulated in the moving image file storage area. The moving image file recording process is executed in the same manner as in the first operation example under the control of the CPU 45_1. Above, description of the operation example 2 is complete | finished.

  Hereinafter, the operation and effect in the operation example 2 of the moving image shooting mode shown in FIG.

  The operation example 2 shown in FIG. 8 obtains the same effect as the operation example 1 described above in that access to each memory is distributed and the bandwidth of the memory is not consumed in the transfer destination image processing circuit. Can do.

  In the operation example 2 described above, the moving image compression unit 43_2 on the second image processing circuit 15 side performs the encoding process using the reference frame stored in the second memory 18. Therefore, the data passing through the serial bus 16 in the operation example 2 is YCbCr data of the current frame and encoded data. Of the data passing through the serial bus 16, the encoded data is compressed, so the amount of data is small. Further, in the operation example 2, since the reference frame data does not pass through the serial bus 16, there is a margin in the bandwidth of the serial bus 16.

  Moreover, in the operation example 2, since the moving image compression unit 43_2 performs the encoding process, a difference in calculation load between the image processing circuits is reduced. Therefore, it is possible to reduce the bias in power consumption in the two image processing circuits.

(Operation example 3 in movie shooting mode)
FIGS. 9A and 9B are schematic views of Operation Example 3 of the electronic camera in the moving image shooting mode. In the operation example 3, the operation until the moving image recording instruction is received is the same as that in the operation example 1 described above.

  When the moving image recording instruction is received in the operation example 3, the CPU 45_1 shifts the second image processing circuit 15 from the low power consumption state to the operation state (a state where clocks are supplied to all circuit blocks to be used). In addition, the CPU 45_1 instructs the imaging unit 12 to output moving image data with a combination of resolution and frame period that matches the moving image specifications. Further, the CPU 45_1 switches the operation of the data distributor 13, and causes both the first image processing circuit 14 and the second image processing circuit 15 to input Bayer data of each frame of the moving image. In the operation example 3, the first image processing circuit 14 bears live view display processing, and the second image processing circuit 15 bears pre-processing, post-processing, and moving image compression processing for recording moving images. .

  FIG. 9A shows an outline of processing by the first image processing circuit 14 at the time of moving image recording in the operation example 3. The image processing unit 41_1 on the master side sequentially performs preprocessing and postprocessing on the Bayer data input from the data distributor 13. At this time, the image processing unit 41_1 generates a through image YCbCr data by performing resolution conversion on the preprocessed moving image data using a resolution conversion circuit (not shown). The CPU 45_1 records the through image YCbCr data in the YcbCr data storage area of the first memory 17. Then, the display unit 21 performs live view display of the through image under the control of the CPU 45_1. The details of the live view display operation are the same as those in the above-described operation example, and thus the description of the overlapping parts is omitted. Further, in FIG. 9A, the flow of live view data relating to live view display is indicated by an alternate long and short dash line arrow.

  On the other hand, FIG. 9B shows an outline of processing by the second image processing circuit 15 at the time of moving image recording in the operation example 3. The image processing unit 41_2 on the slave side sequentially performs pre-processing and post-processing on the Bayer data input from the data distributor 13, and generates YCbCr data for the current frame. The CPU 45_2 records the YCbCr data of the current frame in the YcbCr data storage area of the second memory 18. When the YCbCr data of the current frame is generated, the CPU 45_2 instructs the moving image compression unit 43_2 to start moving image compression processing.

  In response to the above instruction, the moving picture compression unit 43_2 requests data of the current macroblock. The DMAC 44_2 partially reads the macroblock from the current frame (YCbCr data) in the second memory 18 via the memory controller 50_2, and transfers the macroblock to the moving image compression unit 43_2. At this time, the CPU 45_2 sets the start address of the YCbCr data storage area of the second memory 18 as the source address of the DMAC 44_2, the address of the moving picture compression unit 43_2 (internal FIFO), and the number of data transfers as the destination address, respectively. Can be done.

  On the other hand, the DMAC 44_2 transfers the prediction data (reference frame data in the reference frame storage area) stored in the second memory 18 to the moving picture compression unit 43_2 using another channel. Then, the moving image compression unit 43_2 performs encoding in any of the above-described I mode, P mode, and B mode. Note that the operation of the moving image compression unit 43_2 in each of the above-described modes is the same as the operation of the moving image compression unit 43_1 in Operation Example 1, and thus description thereof is omitted.

  The moving image compression unit 43_2 converts the encoded data (moving image file) generated by intra-frame predictive encoding (I mode) or inter-frame predictive encoding (P mode, B mode) into a standard format packet including a header. Divide and output. The above-described packet is written into the moving image file storage area of the second memory 18 by the DMAC 44_2. At this time, the CPU 45_2 sets the address of the moving image compression unit 43_2 (internal FIFO) as the source address of the DMAC 44_2, the address of the moving image file storage area of the second memory 18 as the destination address, and the number of data transfers. Can be done. Note that the CPU 45_2 switches the recording destination to another moving image file storage area each time a certain amount of the packet data is accumulated in the moving image file storage area. Then, the CPU 45_2 notifies the CPU 45_1 of completion of encoding of the current frame.

  When the CPU 45_1 receives the above-described encoding completion notification, the CPU 45_1 executes a moving image file recording process on the recording medium 34. At this time, the CPU 45_1 uses the DMAC 44_1 source address as the start address of the moving image file storage area of the second memory 18, the destination address as the media controller 53_1 (internal FIFO) address, and the number of data transfer packet bytes as the number of data transfers. Is set to start data transfer. When data transfer is started, moving image file data is read from the above-described moving image file storage area, enters the media controller 53_1, and is then sent to the recording medium 34 to be recorded in the recording area inside the recording medium 34. . In FIG. 9B, the data flow of the current frame, the reference frame, and the moving image file are indicated by alternate long and short dashed arrows.

  In the operation example 3 shown in FIG. 9, the same effect as the operation example 1 described above is obtained in that the access to each memory is distributed and the memory bandwidth is not consumed in the transfer destination image processing circuit. Can do.

  In the third operation example, the YCbCr data of the current frame generated by the second image processing circuit 15 is stored in the YCbCr data storage area of the second memory 18. In the operation example 3 described above, the moving image compression unit 43_2 performs the encoding process using the reference frame stored in the second memory 18. In other words, since only the encoded data passes through the serial bus 16 in the operation example 3, even if the bandwidth of the serial bus 16 connecting the two image processing circuits is relatively small, the moving image data is parallelized. Processing can be performed easily.

  Further, in the above operation example 3, all of the moving image compression processing is performed by the second image processing circuit 15, and the first image processing circuit 14 bears overall control at the time of moving image shooting by the electronic camera. Therefore, in the operation example 3, the influence of the image processing of the moving image on the entire system can be made relatively small.

(Operation example 4 in movie shooting mode)
FIG. 10 is a schematic diagram of an operation example 4 of the electronic camera in the moving image shooting mode. This operation example 4 is a modification of the operation example 3 shown in FIG. 9, in which the DMAC 44_2 writes the encoded data output from the moving image compression unit 43_2 into the moving image file storage area of the first memory 17. 3 and different. In FIG. 10, the data flow of the moving image file is indicated by a dashed line arrow.

  According to this operation example 4, access is distributed from the second memory 18 to the first memory 17 by the amount of moving image file data stored in the first memory 17. Therefore, the access amount to the second memory 18 can be reduced as compared with the case of the above operation example 3.

(Operation example 5 in movie shooting mode)
FIG. 11 is a schematic diagram of an operation example 5 of the electronic camera in the moving image shooting mode. This operation example 5 is an example in which a moving image file is generated by associating audio data with a moving image. In the operation example 5, the operation until the moving image recording instruction is received is the same as that in the operation example 1 described above.

  When the moving image recording instruction is received in the operation example 5, the CPU 45_1 shifts the second image processing circuit 15 from the low power consumption state to the operation state (a state where clocks are supplied to all circuit blocks to be used). In addition, the CPU 45_1 instructs the imaging unit 12 to output moving image data with a combination of resolution and frame period that matches the moving image specifications. Further, the CPU 45_1 switches the operation of the data distributor 13, and causes both the first image processing circuit 14 and the second image processing circuit 15 to input Bayer data of each frame of the moving image. Then, the first image processing circuit 14 performs live view display on the display unit 21 by the same processing as in the above-described operation example 3. Further, the CPU 45_1 starts the recording process by the audio signal processing unit 27.

  The image processing unit 41_2 on the slave side sequentially performs pre-processing and post-processing on the Bayer data input from the data distributor 13, and generates YCbCr data for the current frame. The CPU 45_2 records the YCbCr data of the current frame in the YcbCr data storage area of the second memory 18. When the YCbCr data of the current frame is generated, the CPU 45_2 instructs the moving image compression unit 43_2 to start moving image compression processing. Note that the operation of the moving image compression processing by the moving image compression unit 43_2 is substantially the same as that of the above-described operation example 3, and thus description of the overlapping portions is omitted.

  On the other hand, the audio data output from the audio signal processing unit 27 by the recording process is input to the second image processing circuit 15 via the data distributor 13. The audio data may be uncompressed data (linear PCM) or data compressed by an audio compression unit (not shown). The CPU 45_2 temporarily stores the input audio data in the audio data storage area of the second memory 18. In FIG. 11, the flow of audio data is indicated by a dashed line arrow.

  Then, the CPU 45_2 generates one compressed AV data (AV code stream) by multiplexing the image data compressed by the moving image compression process and the audio data described above. At this time, the CPU 45_2 stores the compressed AV data described above in the moving image file recording area of the first memory 17. The compressed AV data described above is written into the moving image file storage area of the first memory 17 by the DMAC 44_2. At this time, the CPU 45_2 sets the address of the moving picture compression unit 43_2 (internal FIFO) as the source address of the DMAC 44_2, the address of the moving picture file storage area of the first memory 17 as the destination address, and the number of data transfers. Can be done. Note that the CPU 45_2 switches the recording destination to another moving image file storage area each time a certain amount of the packet data is accumulated in the moving image file storage area. The moving image file recording process is executed in the same manner as in the first operation example under the control of the CPU 45_1. Above, description of the operation example 5 is complete | finished.

  According to this operation example 5, in addition to the same effect as the operation example 4 described above, a moving image to which audio data is added can be recorded on the recording medium 34.

(Operation example 6 in movie shooting mode)
FIG. 12 is a schematic diagram of an operation example 6 of the electronic camera in the moving image shooting mode. This operation example 6 is a modification of the above operation example 5 and is different from the operation example 5 in that audio data is input from the first image processing circuit 14 to the second image processing circuit 15 via the serial bus 16. To do. In FIG. 12, the flow of audio data is indicated by a dashed-dotted arrow.

  In the configuration of this operation example 6, the audio signal processing unit 27 only needs to be connected to the first image processing circuit 14, and the configuration for inputting audio data to the second image processing circuit 15 becomes unnecessary. Therefore, according to the configuration of the operation example 6, the hardware configuration of the electronic camera can be further simplified. Note that in the data communication of the serial bus 16, the bit rate of the audio data is significantly lower than the bit rate of the image data, so that the audio data does not consume much bandwidth of the serial bus 16. Therefore, in the above operation example 6, it is considered that the possibility that transmission of audio data adversely affects communication between image processing circuits is extremely low.

<Supplementary items of the embodiment>
(1) In the above-described embodiment, an example of a so-called compact electronic camera in which AF and AE are performed by the image sensor 31 that captures an image for recording has been described. However, the imaging apparatus of the present invention is a single-lens reflex electronic camera. Is also applicable naturally.

  (2) In the above embodiment, an example has been described in which processing is performed using two image processing circuits at the time of continuous shooting and still image shooting of still images. However, when the continuous shooting speed of the still images is low or the resolution of the still images is low, the second image processing circuit 15 may be turned off or in a low power consumption state even during continuous shooting. Similarly, when the frame rate of the moving image is low or the resolution is low, the second image processing circuit 15 may be turned off or in a low power consumption state even during moving image shooting.

  (3) In the above embodiment, the moving image compression unit 43 is the H.264 standard. The moving image may be encoded by a moving picture compression standard (such as MPEG2) other than H.264.

  (4) In the operation examples 5 and 6 in the moving image shooting mode, the electronic camera may temporarily store the compressed AV data in the moving image file storage area of the second memory 18.

  From the above detailed description, features and advantages of the embodiments will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

DESCRIPTION OF SYMBOLS 11 ... Imaging optical system, 12 ... Imaging part, 13 ... Data distributor, 14 ... 1st image processing circuit, 15 ... 2nd image processing circuit, 16 ... Serial bus, 17 ... 1st memory, 18 ... 2nd memory, DESCRIPTION OF SYMBOLS 19 ... 1st flash memory, 20 ... 2nd flash memory, 21 ... Display part, 22 ... External I / F connector, 23 ... Media connector, 24 ... Release button, 25 ... Operation button, 27 ... Audio | voice signal processing part, 31 Image sensor 32 ... AFE unit 34 ... Recording medium 41 ... Image processing unit 42 ... Still image compression unit 43 ... Movie compression unit 44 ... DMAC, 45 ... CPU 46 ... Interrupt controller 47 ... Built-in Memory 48... Display controller 49. GPIO section 50. Memory controller 51. Flash memory controller 52. External I / F controller 53 53 Media controller 54 ... audio I / F, 55 ... internal bus, 56 ... Bus bridge

Claims (7)

An imaging unit that outputs image data obtained by capturing an image of a subject;
A first image processing circuit for performing image processing on the image data;
A first memory connected to the first image processing circuit;
A second image processing circuit for performing image processing on the image data;
A second memory connected to the second image processing circuit;
A bus connecting the first image processing circuit and the second image processing circuit,
The first image processing circuit and the second image processing circuit each have an address space, and the first memory and the second memory are mapped to each of the address spaces. The imaging device according to claim 1,
The first image processing circuit is an imaging device that displays and outputs the image stored in the first memory on a display device. In the imaging device according to claim 1 or 2,
The imaging unit outputs moving image data,
The first image processing circuit executes a recording process for recording a moving image file obtained by compressing the moving image data on a nonvolatile recording medium;
The imaging device, wherein the second memory stores a reference frame used in inter-frame predictive encoding at the time of the compression, and reads out the reference frame from the second memory. The imaging device according to claim 3.
The second image processing circuit is an imaging apparatus that executes a moving image encoding process in the compression. In the imaging device according to claim 3 or 4,
An imaging device in which the current frame to be compressed is stored in the second memory. In the imaging device according to claim 3 or 4,
An imaging apparatus in which a current frame to be compressed is stored in the first memory. The imaging apparatus according to any one of claims 3 to 6,
An audio signal processing unit that acquires audio data when the moving image is captured;
The second image processing circuit is an imaging device that associates the audio data with the moving image.

Images