JP2015053557A - Imaging apparatus, method for controlling the same, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently perform access to a memory, when reading image data from the memory and correcting distortion generated in an optical system in the image data.SOLUTION: An imaging apparatus includes an imaging element photoelectrically converting an object image formed through the optical system, signal processing means for processing an image signal read from the imaging element, to generate the image data, the memory storing the signal-processed image data, distortion correction means for reading the image data from the memory, to correct the distortion generated in the optical system in the image data, and control means for controlling the writing and reading of the image data into and from the memory. The control means controls the writing of the image data into the memory, so that the pixel of an optical center in the image data is stored in the top address of an access unit with respect to the memory.

Description

  The present invention relates to an imaging apparatus that corrects distortion generated in an optical system.

  In recent years, with the miniaturization of semiconductor processes, system LSIs equipped with a CPU for system control and an image processing circuit have been developed, and are also used in imaging devices such as digital still cameras and video cameras. In a camera using such a system LSI, the amount of image data to be handled has increased in recent years, and the memory bandwidth for image processing has also increased. For this reason, various techniques for improving the transfer efficiency of data to the memory have been proposed (see, for example, Patent Document 1). In addition, a technique for correcting image distortion caused by an optical system using a system LSI has also been proposed.

JP 2008-005084 A

  When an image distortion caused by an optical system is to be corrected by the system LSI, it is necessary to efficiently access the memory.

  An object of the present invention is to make it possible to efficiently access a memory when image data is read from the memory and distortion generated in an optical system in the image data is corrected.

  In order to solve the above problems and achieve the object, an imaging apparatus of the present invention processes an image sensor that photoelectrically converts a subject image formed through an optical system, and an image signal read from the image sensor. A signal processing unit for generating image data, a memory for storing the signal processed image data, and a distortion correction unit for reading out the image data from the memory and correcting distortion generated in the optical system in the image data And control means for controlling writing and reading of the image data to and from the memory, wherein the control means stores the optical center pixel in the image data at the head address of the access unit to the memory. The writing of image data to the memory is controlled.

  According to the present invention, when image data is read from the memory and distortion occurring in the optical system in the image data is corrected, the memory can be accessed efficiently.

FIG. 2 is a block diagram illustrating a device configuration according to the first embodiment. The figure which shows the relationship between the position of the optical center of an image, and the image read from memory. FIG. 9 is a diagram for explaining a camera shake correction method according to the second embodiment. FIG. 3 is a block diagram showing a device configuration of a second embodiment. The figure explaining the change of the imaging position by the influence of distortion.

  Hereinafter, embodiments for carrying out the present invention will be described in detail. The embodiment described below is an example for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment. Moreover, you may comprise combining suitably one part of each embodiment mentioned later.

  [Embodiment 1] An embodiment in which the imaging apparatus of the present invention is applied to, for example, a digital still camera or a digital video camera (hereinafter referred to as a camera) will be described below.

  <Overview> The camera of this embodiment is configured as a system LSI in which a development processing unit 106, a memory controller 107, a CPU 108, a distortion correction unit 110, and an encoding unit 111, which will be described later, are incorporated in one semiconductor chip. Yes. In this case, since there are a large number of accesses to the memory 109 from each circuit of the system LSI, an access conflict occurs. Therefore, the memory buses of the memory controller 107 and the memory 109 are widened with respect to the local bus between each circuit in the system LSI and the memory controller 107, and the internal processing is not interrupted even if access conflicts. The However, when the memory bus has a wide bandwidth, the minimum access unit becomes larger than that of the local bus, so that the bandwidth may be wasted depending on how the memory 109 is accessed.

  On the other hand, there is a method of correcting distortion generated in the optical system by signal processing. Hereinafter, with reference to FIG. 5, the change in the imaging position due to the influence of distortion will be described.

  In FIG. 5, a dotted-line rectangle 510 indicates an actual subject image, and a solid-line rectangle 520 indicates a subject image formed on the image sensor due to the influence of distortion. Pixels at the four corners of the subject image indicated by the dotted rectangle 510 are shown as solid lines ● for the optical center pixel 500, and dots 511, 512, and 513 in the clockwise direction as lower left, upper left, and lower right pixels. Pixels of the subject image affected by the distortion indicated by the solid line rectangle 520 are indicated by the solid line circles 521, 522, and 523 at the lower left, upper left, and lower right pixels. Here, the optical center pixel 500 is not affected by the distortion, and thus is at the same position. Since the influence of the distortion is proportional to the distance from the optical center, the dotted line ◯ 512 pixel that is far away is most affected. Further, the pixels 511 and 513 on the X axis and Y axis with respect to the optical center are imaged on the X axis and Y axis even if they are affected by distortion.

  On the other hand, when processing an image affected by distortion as described above, there is a method of correcting distortion by dividing an image because the access efficiency at the time of reading varies depending on the address of the pixel at the optical center. Hereinafter, a method for correcting distortion by dividing an image will be described with reference to FIG.

  FIG. 2 shows the relationship between the position of the optical center of the image on which distortion correction is performed and the image read from the memory 109. The pixel position of the optical center is indicated by ● and the boundary of the access unit of the memory 109 is indicated by a dotted line. FIG. 2A shows an actual subject image, and a lattice pattern is given for convenience of explanation. FIG. 2B shows a subject image formed on the image sensor under the influence of distortion. Since the influence of distortion is proportional to the distance from the optical center, the four corners of the subject image are most affected. Here, as shown in FIG. 2A, an image is read out from the memory 109 in a case where an image is divided into a plurality of rectangles in a grid pattern, and the image before correction is corrected for each rectangle. A description will be given using four rectangles 201 to 204 with the center 200 as the center. FIG. 2C and FIG. 2D respectively show storage positions (addresses) of subject image data stored in the memory 109. FIG. 2C is an explanatory diagram in a case where the optical center data is stored at an address different from the address serving as the boundary of the access unit of the memory 109. The image data held in the memory 109 is read in an access unit having a data amount (number of bits) determined by the memory controller 107. When performing distortion correction, it is necessary to read out all rectangular areas necessary for correction. That is, in order to read out the image data of the rectangles 201 and 203, it is necessary to access the area from the address x1 to x3, and in order to read out the image data of the rectangles 202 and 204, from the address x2 to x4 Need to access the area.

  Next, with reference to FIG. 2D, a case where optical center data is stored at an address that is a boundary between access units in the memory 109 will be described. In FIG. 2D, in order to read the image data of the rectangles 201 and 203, the area from the address x1 to x2 is accessed, and in order to read the image data of the rectangles 202 and 204, the address x2 to x3 is read. What is necessary is just to access an area. As described above, the access efficiency at the time of reading image data from the memory 109 in order to perform distortion correction varies depending on the storage address of the image data in the memory 109.

  In this embodiment, when the image data is written in the memory 109, the address at which the image data is written is controlled in accordance with the access unit of the memory 109, thereby improving the access efficiency at the time of reading.

  <Apparatus Configuration> With reference to FIG. 1, the configuration and functions of the camera of this embodiment will be described.

  In FIG. 1, an optical image of a subject incident through the fixed lens 101 and the diaphragm 102 is focused on the imaging surface of the image sensor 104 with the focal length adjusted by the focus lens 103. The diaphragm 102 adjusts the amount of incident light of the optical image of the subject. The image sensor 104 is an image sensor such as a CCD or CMOS, and photoelectrically converts a subject image formed by the focus lens 103.

  An AFE (analog front end) 105 includes a CDS (correlated double sampling) circuit for extracting a signal level from an analog pixel signal output from the image sensor 104, and an A / D converter that converts the analog pixel signal into a digital signal. Is an analog signal processing circuit. The AFE 105 corrects lens aberrations, corrects defective pixel signals of the image sensor 104, and electrically amplifies video signals.

  The digital image signal processed by the AFE 105 is output to the development processing unit 106 and subjected to development processing such as aperture correction, gamma correction, and white balance correction, and the generated image data is output to the memory controller 107.

  The CPU 108 controls the operation of the entire apparatus according to a firmware program or the like. In particular, the system control CPU 108 outputs distortion correction parameters and optical center position information to the memory controller 107 and the distortion correction unit 110 in accordance with lens distortion information registered in advance.

  The memory 109 has a semiconductor memory such as SDRAM, and temporarily holds image data generated by the development processing unit 106 and other data. Access to the memory 109 is controlled by the memory controller 107. In order to perform distortion correction by the distortion correction unit 110, writing and reading of image data is controlled by the memory controller 107.

  The memory controller 107 writes and reads data to and from the memory 109 in units of a predetermined amount of data (number of bits). As described with reference to FIG. 2D, the memory controller 107 stores the optical center pixel 200 of the image data read from the memory 109 to perform distortion correction at the head address of the minimum access unit of the memory 109. Thus, the write address to the memory 109 is controlled.

  Further, the memory controller 107 controls the write address of the memory 109 so that the data of each pixel of the image data of one screen is stored in the memory 109 at an address corresponding to the position of each pixel in the screen.

  The distortion correction unit 110 divides the image into a plurality of rectangles as illustrated in FIG. 2A, and calculates an image area necessary for correction according to the corrected position information and the distance from the optical center for each rectangle. . Then, the distortion correction unit 110 specifies the address of the memory 109 in which the calculated image area data is stored, and reads out the image data before distortion correction subjected to the development processing by the memory controller 107.

  At this time, as shown in FIG. 2D, the distortion correction unit 110 includes the calculated start address of the access unit including the calculated image area based on the access unit of the memory 109 and the data amount (number of bits) to be read. Is sent to the memory controller.

  For example, as shown in FIG. 2D, when image data is stored in the memory 109 and the image data in the image area 201 is read, the distortion correction unit 110 uses the address x2 that is the head of the access unit as the column address. Designate and designate the number of bits from x2 to x3 as the amount of data to be read.

  The distortion correction unit 110 performs distortion correction processing on the image data read from the memory 109 using a predetermined distortion correction parameter, and writes it back to the memory 109 via the memory controller 107. The image data after distortion correction is encoded by the encoding unit 111 and written to the recording medium 112.

  According to the present embodiment, with respect to an image affected by the distortion of the optical system, the pixel on the vertical line at the optical center where the read address of the image data for performing distortion correction does not change matches the boundary of the memory access unit. By controlling the write address in this way, reading from unnecessary areas can be reduced and access efficiency can be improved.

  In FIG. 2D, one rectangle is stored in the range of one access unit, but one rectangle may be stored over a plurality of access units. In such a case, the distortion correction unit 110 specifies the head address of the access unit including the necessary image area, and specifies the data amount corresponding to one rectangular image data as the data amount to be read.

  Further, when the configuration is such that the number of pixels of one screen of the image data to be captured can be changed, the CPU 108 sets the number of pixels of the image data output from the development processing unit 106 according to the number of pixels set by the user. change. The position (number of pixels) from the screen edge of the optical center in the image data output from the development processing unit 106 varies depending on the number of pixels of the image data. Therefore, the CPU 108 determines the position of the pixel serving as the optical center according to the number of pixels of the image data output from the development processing unit 106, and stores the pixel at the optical center at the start address of the access unit in the memory 109. The memory controller 107 is instructed.

  [Embodiment 2] Next, referring to FIG. 3 and FIG. 4, as Embodiment 2, camera shake is detected, and the read address of image data from the image sensor is controlled according to the detected amount of camera shake. A method for correcting camera shake will be described.

  FIG. 3 is a schematic explanatory diagram of camera shake correction according to the present embodiment, and the horizontal and vertical center positions are indicated by dotted lines.

  In FIG. 3A, reference numeral 301 denotes an entire imaging region of the imaging element, and 302 denotes a cutout frame that is actually read out of the imaging region. Reference numeral 303 denotes a subject photographed by the photographer.

  FIG. 3B shows the direction of camera movement when the photographer shakes the camera in the direction indicated by the black arrow 304. In this case, the subject 303 is imaged on the image sensor while moving in the white arrow 305 in the direction opposite to the black arrow 304. Here, by moving the cutout frame 302 by the same amount as the movement of the white arrow 305, the movement is canceled between the areas that are actually read. However, when the cutout position from the image sensor changes, the position of the optical center of the image signal input to the development processing unit 106 changes.

  Therefore, in the present embodiment, the position of the optical center is calculated from the cutout position of the image signal from the image sensor 104, and control is performed so that the optical center pixel of the image data for which distortion correction is performed matches the boundary of the memory access unit. .

  FIG. 4 shows an apparatus configuration of the present embodiment, and the same elements as those of the first embodiment of FIG. 1 are denoted by the same reference numerals.

  The gyro sensor 113 detects the movement of the camera and outputs it to the CPU 108 and the readout range control unit 114. The readout range control unit 114 sets the cutout position in the opposite direction to the camera movement, and controls the readout range of the image signal from the image sensor 104. The CPU 108 calculates the position of the optical center of the image signal input from the image sensor 104 based on the movement of the camera, and outputs it to the memory controller 107. The memory controller 107 controls the write address so that the pixel at the optical center of the image data for which distortion correction is performed matches the boundary of the minimum access unit of the memory 109.

  According to the present embodiment, when camera shake is corrected by controlling the position where the image signal is cut out from the image sensor 104, the position of the optical center is corrected according to the change in the position where the image signal is cut out. Access efficiency can be improved.

  [Other Embodiments] The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program code. It is processing to do. In this case, the program and the storage medium storing the program constitute the present invention.

Claims (8)

An image sensor that photoelectrically converts a subject image formed through an optical system;
Signal processing means for processing image signals read from the image sensor to generate image data;
A memory for storing the signal processed image data;
Distortion correction means for reading image data from the memory and correcting distortion generated in the optical system in the image data;
Control means for controlling writing and reading of image data to and from the memory,
The image pickup apparatus, wherein the control unit controls writing of image data into the memory so that a pixel at an optical center in the image data is stored at a head address of an access unit for the memory.   The control means controls writing of the image data to the memory so that the optical center pixel in the image data is stored at the head address of the minimum access unit accessible to the memory. The imaging apparatus according to claim 1. Detecting means for detecting movement of the imaging device;
Read range control means for controlling a range in which an image signal is read from the image sensor in accordance with the movement detected by the detection means;
3. The image pickup apparatus according to claim 1, wherein the control unit detects a position of a pixel at the optical center in image data in the readout range based on a readout range of the image signal. The detecting means is a gyro sensor;
The imaging apparatus according to claim 3, wherein the reading range control unit sets a range in which an image signal is read from the imaging element in a direction opposite to the movement of the imaging apparatus. The signal processing means, the distortion correction means, and the control means are configured as a system LSI in which circuits having respective functions are incorporated in one semiconductor chip,
The imaging apparatus according to claim 1, wherein the control unit controls access from the system LSI to the memory. An image sensor that photoelectrically converts a subject image formed through an optical system, a signal processing unit that processes image signals read from the image sensor to generate image data, and stores the signal processed image data A memory for reading out image data from the memory and correcting distortion generated in the optical system in the image data, and control means for controlling writing and reading of image data to and from the memory. A method for controlling an imaging apparatus,
The image pickup apparatus comprising: a step of controlling the writing of the image data to the memory so that the control means stores the optical center pixel in the image data at a head address of an access unit to the memory. Control method.   The program for functioning a computer as each means of the imaging device described in any one of Claims 1 thru | or 5.   A storage medium storing a program for causing a computer to function as each unit of the imaging apparatus according to any one of claims 1 to 5.

Images