JP2012090212A - Image processing device, personal digital assistance, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To combine improvement in the image quality of a photographic image with reduction in power consumption by providing image quality correction corresponding to the characteristic of an imaging device and an electronic zoom function.SOLUTION: A line delay processing part 302 outputs image data obtained by delaying RGB-format image data by each line. A horizontal delay processing part 303 outputs image data obtained by delaying RGB-format image data by each sampling frequency. Horizontal filters 304 to 307 perform arithmetic processing by four pixels each time in a horizontal direction. Vertical filters 308 to 311 perform arithmetic processing by four pixels each time in a vertical direction. An arithmetic processing part 313 selectively performs phase correction or enlargement processing using pixels from the horizontal filters 304 to 307 and the vertical filters 308 to 311.

Description

  The present invention relates to an image processing device, a portable information terminal device, an image processing method, and a program for processing image data from an image sensor that requires phase correction.

  Conventionally, in an electronic zoom in an imaging device such as a digital camera, a part of an image captured by an image sensor such as a CCD is cut out, and a subject image having a size corresponding to the zoom magnification is obtained by enlarging or reducing the extracted image. Getting was done. In this case, a clipping process, an enlargement process, or a reduction process is indispensable, and a signal processing load during imaging is increased.

  Therefore, for example, when reading out an image pickup signal from the image pickup device, “pixel addition read” for reading out the signal charges of a plurality of pixels of the same filter color, or “thinning out reading” for reading out the signal charge by thinning out pixels from the image pickup device. ”Is performed to reduce the number of pixels of the image read out as the imaging signal, thereby reducing the processing load of the signal required for the clipping process at the time of imaging.

  However, even if these pixel addition readout and thinning readout are performed, it is possible to reduce the number of pixels of the image, but it is indispensable to cut out a predetermined area in accordance with the electronic zoom magnification. The image pickup signal includes a signal of a pixel other than the predetermined region cut out during the cut-out process, and the signal processing load in the image pickup apparatus is insufficiently reduced.

  Thus, conventionally, a technique for significantly reducing the signal processing burden when performing electronic zoom has been proposed (see, for example, Patent Document 1). In Patent Document 1, during electronic zoom processing, the image sensor is selected to output an imaging signal obtained by adding signal charges corresponding to a predetermined number of pixels in the vertical direction and / or horizontal direction in a predetermined selection area of the imaging area. Drive in region addition mode.

  At this time, the image size of the selected area is set according to the size of the imaging area and the electronic zoom magnification. At the same time, the number of added pixels of the signal charge in the vertical and horizontal directions is controlled so that the image size after the addition is equal to or greater than the required image size such as the size of the through image, and the larger the electronic zoom magnification becomes, In addition, the switching is performed step by step so as to decrease as the magnification increases.

  Accordingly, when the display angle of view is wide at a low magnification, the data amount of the image pickup signal is reduced by the addition process, and the image pickup sensitivity is increased by the pixel addition effect to compensate for the deterioration of the image quality. In addition, when the angle of view is narrow at a high magnification, priority is given to high resolution by reducing the number of additions, and the amount of data is suppressed to a predetermined value or less by narrowing the angle of view (= selected area).

JP 2008-17090 A

  By the way, in a portable information terminal device such as a cellular phone terminal in recent years, a digital still camera and a digital camcorder (hereinafter, both will be described as digital cameras) that record a subject image as electronic image data on a recording medium. Models with functions are prevalent. The number of image sensors for imaging with the digital camera is increasing year by year, and the burden of image processing is increasing.

  FIG. 6 is a conceptual diagram illustrating an example of one-pixel addition reading using a reading method from the image sensor. Pixel addition reading is performed by adding two pixels in which color filters of the same color are arranged every other pixel in the vertical direction as shown in (b) of FIG. The center of gravity of the added and read signal is located in the middle of the two added pixels.

  Therefore, when focusing on the position of the center of gravity, the pixels are not evenly arranged in the vertical direction, and are arranged close to each other by two lines. Since this signal is output at equal intervals as shown in (c) in the figure, when a color difference / luminance signal is generated from the signal obtained by pixel addition reading, a false signal is generated due to the bias of the signal arrangement, and the edge is not stable. In addition, there is a problem that an adverse effect such as image distortion occurs.

  In area selection readout, only pixels in a necessary area are cut out from the image sensor and then scaled up to a desired display area. Therefore, there is no image distortion caused by pixel addition readout, and there is an advantage that image quality is improved. However, in the area selective reading, since the number of pixels read from the image sensor increases, it is necessary to increase the reading speed of the image sensor, and the power consumption increases. Also, in shooting with a high zoom magnification, only a pixel in a necessary area in all the image sensors is read, so wide-angle shooting cannot be performed. In addition, since the amount of light received per image sensor is one pixel, there is a problem in that sensitivity is lowered and photographing in a dark place is severely difficult.

  For functions mounted on a portable device, it is desired to fit in a small casing and to have low power consumption. Since portable devices are battery-powered, it is possible to use a large-capacity battery so that it can be used continuously for a long time, but it can not be housed in a small housing, and furthermore the inside of the housing generates heat, Realization with low power consumption is required.

  However, when converting the reading method of the image sensor according to the zoom magnification shown in Patent Document 1, the influence of the rattling of the edge due to the center of gravity shift (phase shift) at the time of pixel addition reading and the distortion of the image is corrected. Since there is no description about the means, there is a problem that the image quality is affected.

  The present invention has been made in view of such circumstances, and the object of the present invention is to provide image quality correction that matches the characteristics of the image sensor and an electronic zoom function, and to improve image quality of captured images. An object of the present invention is to provide an image processing device, a portable information terminal device, an image processing method, and a program that can simultaneously reduce power consumption.

  In order to solve the above-described problem, the present invention inputs an image to which image processing is performed by inputting one of first image data having a phase information shift and second image data having no phase information shift. A processing device, which holds the first image data or the second image data, and outputs a pixel of each image data in a predetermined order; and input data is the first image If it is data, phase correction is performed using pixels of the first image data output in a predetermined order from the buffer means, and if the input data is the second image data, a predetermined value is output from the buffer means. And an arithmetic processing means for performing an enlargement process using the pixels of the second image data output in the above order.

  In order to solve the above-described problems, the present invention is a portable information terminal device including an imaging device, and an imaging unit including a plurality of imaging devices that photoelectrically convert an optical image of a subject and output as a charge signal. A pixel addition readout method for outputting a readout signal from the imaging means, and outputting an imaging signal obtained by adding charge signals from a plurality of imaging elements selected at predetermined intervals in the vertical direction and / or horizontal direction from the imaging means; Select a predetermined area in the vertical direction and / or horizontal direction from all the imaging areas of the imaging means, and selectively switch the area selection readout system to output a charge signal for the number of pixels as an imaging signal, and the pixel addition readout system Or an image sensor drive control means for outputting image data from the image pickup means in either of the region selection readout method, and the image sensor drive control means from the image pickup device drive control means. Buffer means for holding image data output by either the prime addition reading method or the region selection reading method, and outputting the pixels of each image data in a predetermined order, and the pixel addition reading method are selected. If the region selection read method is selected, the phase correction is performed using the pixels of the image data output from the buffer means in a predetermined order. A portable information terminal device comprising: arithmetic processing means for performing enlargement processing using the pixels of the image data to be output.

  In order to solve the above-described problem, the present invention inputs either one of the first image data having a phase information shift and the second image data having no phase information shift to perform image processing. An image processing method to be applied, the step of holding the first image data or the second image data and outputting the pixels of the respective image data in a predetermined order, and the input data being the first data If it is image data, phase correction is performed using pixels of the first image data output in the predetermined order, and if the input data is the second image data, it is output in the predetermined order. And an arithmetic processing step for performing an enlargement process using the pixels of the second image data.

  In order to solve the above-described problem, the present invention inputs either one of the first image data having a phase information shift and the second image data having no phase information shift to perform image processing. The computer of the image processing apparatus to be applied holds the first image data or the second image data, and outputs a pixel function of each pixel of the image data in a predetermined order. If it is image data, phase correction is performed using pixels of the first image data output in the predetermined order, and if the input data is the second image data, it is output in the predetermined order. A program for executing an arithmetic processing function for performing an enlargement process using a pixel of the second image data.

  According to the present invention, it is possible to provide an image quality correction and an electronic zoom function that match the characteristics of the image sensor, and it is possible to achieve both improvement in image quality of captured images and reduction in power consumption.

It is a block diagram which shows the structure of the image processing apparatus by embodiment of this invention. It is a conceptual diagram which shows the operation | movement of the phase distortion correction of the image data by the image processing apparatus 3 of the 1st embodiment. It is a conceptual diagram for demonstrating the operation | movement of the expansion process of the image data using the image processing apparatus 3 by this 1st Embodiment. It is a block diagram which shows the structure of the image processing apparatus by this 2nd Embodiment. It is a block diagram which shows the structure of the portable information terminal with an image pick-up element by this 3rd Embodiment. It is a conceptual diagram which shows an example of one pixel addition reading of the reading system from an image sensor.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
A. First Embodiment First, a first embodiment of the present invention will be described.
FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. In the figure, RGB format image data is input to the input terminal 301. At this time, as image data in the RGB format, a pixel addition readout method (outputs an imaging signal obtained by adding charge signals from a plurality of imaging elements selected at predetermined intervals in the vertical direction and / or horizontal direction from the imaging unit), Alternatively, read out by any one of the area selection readout method (selecting a predetermined area in the vertical direction and / or the horizontal direction from the entire imaging area of the imaging unit and outputting a charge signal for the number of pixels as an imaging signal) Image data. The line delay processing unit 302 includes, for example, a memory, and outputs image data obtained by delaying image data in RGB format in units of lines. The horizontal delay processing unit 303 includes, for example, a flip-flop, and outputs image data obtained by delaying RGB format image data in units of sampling frequencies.

  For example, the horizontal filters 304, 305, 306, and 307 are configured to perform arithmetic processing in units of four pixels in the horizontal direction. In addition, pixel data delayed in line units from the line delay processing unit 302 is input to the horizontal filters 304, 305, 306, and 307. As a specific example, image data of the first line is input to the horizontal filter 304, the second line to the horizontal filter 305, the third line to the horizontal filter 306, and the fourth line to the horizontal filter 307 in units of sampling frequency. Is done.

  For example, the vertical filters 308, 309, 310, and 311 are configured to perform arithmetic processing in units of four pixels in the vertical direction. The vertical filters 308, 309, 310, and 311 receive four pixels that are delayed from the horizontal delay processing unit 303 in the vertical direction in units of sampling frequency. The arithmetic processing unit 313 calculates the image data processed in the horizontal direction and the vertical direction. The calculation method (pixel addition reading method or region selection reading method) of the calculation processing unit 313 is configured to be switchable according to an input from the input terminal 312. The calculation result of the calculation processing unit 313 is output from the output terminal 314.

FIGS. 2A to 2C are conceptual diagrams illustrating the operation of correcting the phase distortion of the image data by the image processing apparatus 3 according to the first embodiment. (A) is a figure which shows an example of the image data of the Bayer format which added the pixel input into the input terminal 301. FIG. The horizontal direction between the pixel G ₁₂ and the pixel R ₁₅ in FIG. (A), before the pixel addition has 2 pixel data has become a thinned state. Further, although there was pixel data for two lines in the vertical direction between the pixel G ₂₁ and the pixel R ₅₁ , this is also thinned out. If this pixel data is R ₁₁ → G ₁₂ → R ₁₅ → G ₁₆ . . . . , G ₂₁ → B ₂₂ → G ₂₅ → B ₂₆ . . . . Is entered.

The line delay processing unit 302 stores pixel data for each line,
The horizontal filter 304 includes R ₁₁ → G ₁₂ → R ₁₅ → G ₁₆ . . . . ,
The horizontal filter 305 includes G ₂₁ → B ₂₂ → G ₂₅ → B ₂₆ . . . . ,
The horizontal filter 306 includes R ₅₁ → G ₅₂ → R ₅₅ → G ₅₆ . . . . ,
The horizontal filter 307 includes G ₆₁ → B ₆₂ → G ₆₅ → B ₆₆ . . . . ,
Is supplied.

The output of the line delay processing unit 302 is supplied to the horizontal delay processing unit 303,
The vertical filter 308 includes R ₁₁ , G ₂₁ , R ₅₁ , G ₆₁ ,
The vertical filter 309 includes G ₁₂ , B ₂₂ , G ₅₂ , B ₆₂ ,
The vertical filter 310 includes R ₁₅ , G ₂₅ , R ₅₅ , G ₆₅ ,
The vertical filter 311 includes G ₁₆ , B ₂₆ , G ₅₆ , B ₆₆ ,
Is supplied.

By this filter processing, calculation between lines and between pixels is enabled as shown in FIG. For example, G ₀₁ , B ₃₁ , G ₄₁ , B ₇₁ are generated by the vertical filter 308, R ₀₂ , G ₃₂ , R ₄₂ , G ₇₂ are generated by the vertical filter 309, and G ₀₅ , B ₃₅ , G ₄₅ and B ₇₅ are generated, and the vertical filter 309 generates R ₀₆ , G ₃₆ , R ₄₆ , and G ₇₆ .

In the horizontal direction, the horizontal filter 304 generates G ₁₀ , B ₁₃ , G ₁₄ , and B ₁₇ , the horizontal filter 305 generates R ₂₀ , G ₂₃ , R ₂₄ , and G ₂₇ , and the horizontal filter 306 generates G ₅₀ , B ₅₃ , G ₅₄ , and B ₅₇ are generated, and the horizontal filter 307 generates R ₆₀ , G ₆₃ , R ₆₄ , and G ₆₇ .

In the arithmetic processing unit 313, R ₀₀ , G ₀₃ , R ₀₄ , G ₀₇ , G ₃₀ , B ₃₃ , G ₃₄ , B ₃₇ , R ₄₀ , G ₄₃ , R ₄₄ , G ₄₇ , G ₇₀ , B ₇₃ , G ₇₄ and B ₇₇ are calculated. The calculation at this time is interpolation correction between pixels, but may be extrapolation correction. For example, each of RGB may be processed after being replaced with a luminance component.

(C) in the figure shows an example in which the phase correction processing is performed on the peripheral pixel data obtained as a result of the calculation. From combinations that can maintain the phase interval from the surrounding pixels (in the case of R ′, R ₀₀ , R ₀₂ , R ₂₀ , R ₂₂ , in the case of G ′, G ₀₃ , G ₁₂ , G ₁₄ , G ₂₃ ), R ′, G ′, and B ′ can be newly obtained to correct the phase correctly.

  FIGS. 3A and 3B are conceptual diagrams for explaining the operation of image data enlargement processing using the image processing apparatus 3 according to the first embodiment. In FIG. 5A, only the R component is shown as an example of RGB format image data input to the input terminal 301, and the description is omitted because the same applies to the G component and the B component. .

Pixel data is not thinned out in the horizontal and vertical directions,
R ₀₀ → R ₀₁ → R ₀₂ → R ₀₃ . . . . ,
R ₁₀ → R ₁₁ → R ₁₂ → R ₁₃ . . . . ,
Are input to the input terminal 301 in this order.

The line delay processing unit 302 stores pixel data for each line. The line delay processing unit 302 stores pixel data for each line,
The horizontal filter 304 includes R ₀₀ → R ₀₁ → R ₀₂ → R ₀₃ . . . . ,
The horizontal filter 305 includes R ₁₀ → R ₁₁ → R ₁₂ → R ₁₃ . . . . ,
Is supplied.

The output of the line delay processing unit 302 is supplied to the horizontal delay processing unit 303, R ₀₀ and R ₁₀ are supplied to the vertical filter 308, and R ₀₁ and R ₁₁ are supplied to the vertical filter 309. This pixel data is subjected to filter processing to perform inter-line and inter-pixel calculations as shown in FIG. For example, to generate the _{R 0.5_0} vertical filter 308, a vertical filter 309 to generate the _{R 0.5_1.} In the horizontal direction, the horizontal filter 304 generates R 0 — _0.5 , and the horizontal filter 305 generates R 1 — _0.5 . The arithmetic processing unit 313 calculates R _{0.5 — 0.5} from R _{0.5 — 0} , R _{0.5 — 1} , R _{0 — 0.5} , and R 1 — _0.5 that are outputs of the respective filters.

  As a result, double pixel data is generated in the horizontal direction and the vertical direction by the enlargement process. The calculation at this time is interpolation correction between pixels, but may be extrapolation correction. For example, each of RGB may be processed after being replaced with a luminance component.

  As described above, the image processing unit 3 can perform the phase correction process and the enlargement process with substantially the same circuit configuration, and thus can be configured in a small scale and with reduced power consumption.

  Moreover, since it can be used properly according to the application, for example, it is configured to be easy to use when incorporated in an LSI or FPGA.

B. Second Embodiment Next, a second embodiment of the present invention will be described.
FIG. 4 is a block diagram showing the configuration of the image processing apparatus according to the second embodiment. In FIG. 4, RGB format image data is input to the input terminal 201. The line delay processing unit 202 includes, for example, a memory, and outputs image data obtained by delaying image data in RGB format in units of lines.

  The phase correction unit 203 performs phase correction on image data in RGB format. The enlargement processing unit 204 enlarges the image data in the RGB format twice in the horizontal direction and twice in the vertical direction. The selection unit 205 selects one of the RGB signals of the enlargement processing unit 204 or the phase correction processing unit 203 based on the signal of the input terminal 206, and outputs it from the output terminal 207.

  According to the configuration of the second embodiment, only the line delay processing unit 202 is shared from the image processing apparatus described in the first embodiment, and the circuit can be similarly reduced in size.

C. Third Embodiment Next, a third embodiment of the present invention will be described.
FIG. 5 is a block diagram showing a configuration of a portable information terminal with an image sensor according to the third embodiment. In the third embodiment, the image processing device 3 according to the first embodiment described above is applied to the portable information terminal 100 with an image sensor. In FIG. 5, the portable information terminal 100 with an image sensor can record image data from the optical lens unit 1 onto the memory card 9 or reproduce the image data recorded on the memory card 9 and display it on the display unit 7. it can.

  The optical lens unit 1 adjusts the aperture for adjusting the focal length of the subject and the amount of light passing through the lens. The imaging device 2 is composed of, for example, a sensor such as a CMOS or a CCD, converts an optical image of a subject from the optical lens unit 1 into an electrical signal, and converts Bayer-format image data into RGB-format image data. Output.

  The image processing unit 3 performs phase correction processing or enlargement processing on the pixel data from the image sensor 2. The DSP 4 performs image processing such as format conversion from RGB to YUV or noise removal on the pixel data from the image processing unit 3. The output from the DSP 4 is composed of, for example, a 10-bit luminance signal with a synchronization signal, a 10-bit color difference signal, and a 1-bit transmission clock, and is supplied to the back-end processing unit 5.

  The back-end processing unit 5 performs a resizing process for matching the image data from the DSP 4 with the number of pixels of the display unit 7 and outputs the processed data to the LCD controller 6. The LCD controller 6 has a built-in memory for storing an image of about 1 screen or 2 screens, and either intermittently or partially sent from the CPU 13 or the back-end processing unit 5. Either one or both are combined to generate a frame image, which is continuously read out at a frequency of about 60 Hz and output to the display unit 7.

  The display unit 7 can use a striped configuration in which three pixels of RGB constitute one pixel, such as QVGA (320 × 240 × RGB), VGA (640 × 480 × RGB), (wide VGA (800 × 480 × RGB), full-wide VGA (854 × 480 × RGB), etc. Devices having various numbers of pixels can be used.

  Here, an example in which the liquid crystal display is used will be described. However, a self-luminous organic EL display may be used. In addition, the back end processing unit 15, the memory card 9, image data from the DSP 4, for example, MPEG2 (ISO / IEC 13813-2, International Standard), which is a moving image coding technology, H. High compression processing such as H.264 (ITE / ISO 14496-10 / H.264AVC) / AVC (Advanced Video Coding) is performed, and recording is performed on the memory card 9. Further, the image data is recorded as still image data subjected to compression processing such as JPEG (Joint Photographic Experts Group).

  The back-end processing unit 5 reads the compressed data recorded on the memory card 9 and outputs the decoded image data to the LCD controller 6. The audio is decoded and then output from the speaker 11 via the DAC 10. Further, the back-end processing unit 5 performs the same processing on the video / audio stream acquired from the memory card 9.

  In addition to the above-described video stream in which video and audio are multiplexed, the memory card 9 can hold still images, animation image files, audio-only files, and the like. In addition, the back-end processing unit 5 and the CPU 13 can exchange control commands and video / audio data for devices connected to the back-end processing unit 5 side.

  The memory 18 is connected to the CPU 13 via the CPU bus 8 and stores various control programs, databases, telephone books, address books, incoming melody, music data, moving images, image data such as still images, and the like. The DAC 10 converts a ring tone output from the CPU 13 and a digital voice signal such as a call voice into an analog signal and supplies the analog signal to the speaker 11. The speaker 11 outputs a ring tone, a call voice, and the like based on the analog signal supplied from the DAC 10.

  The CPU 13 performs processing related to detection processing of the operation content of the key 12, voice call processing, music and image reproduction processing, and the portable information terminal 100 with an image sensor in general. For example, the CPU 13 acquires a program from the memory 18 via the CPU bus 8 and controls the encoding / decoding processing circuit 21 and the radio circuit 20 to perform processing related to waiting for an incoming call. On the other hand, when receiving a call, the CPU 13 reads the name of the caller, the incoming melody, the incoming call image, etc. from the telephone directory in the memory 18. The CPU 13 outputs the audio data to the DAC 10, and outputs the other party's telephone number, name, and image data to the LCD controller 6 via the back-end processing unit 5.

  The power supply control unit 15 charges the battery by supplying a charging current supplied from the power supply 14 connected to the charging input terminal to a battery (not shown) connected to the power supply control unit 15. Moreover, the power supply control part 15 supplies the electric power output from a battery to each part of the portable information terminal 100 with an image pick-up element.

  The communication antenna 19 receives a radio wave transmitted through the air, converts it into a high-frequency electric signal, and inputs it to the radio circuit 20. The communication antenna 19 converts the high-frequency electrical signal output from the radio circuit 20 into a radio wave and transmits the radio wave. The radio circuit 20 demodulates the high-frequency electrical signal received by the communication antenna 19 based on an instruction from the CPU 13 and inputs the demodulated signal to the encoding / decoding processing circuit 21. Further, the radio circuit 20 performs modulation processing on the output signal of the encoding / decoding processing circuit 21, converts it into a high-frequency electric signal, and outputs it to the communication antenna 19.

  The encoding / decoding processing circuit 21 performs a decoding process on the output signal of the radio circuit 20 according to the control of the CPU 13, outputs a call voice signal to the receiver 23, and outputs character data, image data, and the like to the CPU 13. The encoding / decoding processing circuit 21 encodes the voice of the user or the like input via the microphone 22, the character edited by the user operating the key 12, the image data read from the memory 18, or the like. And outputs a signal obtained as a result to the radio circuit 20.

  With this configuration, during wide-angle shooting, the image sensor 2 is set to output a signal obtained by pixel addition. At the same time, the CPU 13 controls the input terminal 206 so that the selection unit 205 of the image processing unit 3 selects a signal from the phase correction processing unit 203. Thereby, the phase shift of the pixel data generated at the time of pixel addition can be corrected, and the influence of image distortion due to pixel addition can be eliminated.

  Further, at the time of shooting at a high magnification, a setting is made such that a part of the image sensor 2 is cut out and all pixels in the cut out range are read out without adding pixels. At the same time, the CPU 13 controls the input terminal 206 so that the selection unit 205 of the image processing unit 3 selects a signal from the enlargement processing unit 204. In this case, since the image sensor 2 is not subjected to pixel addition, it is not necessary to perform phase correction, so the line processing unit 202 can be used for the enlargement processing 204.

  Therefore, the enlargement processing unit 204 can perform enlargement processing in the horizontal and vertical directions. In addition, since pixel data from the image sensor 2 is not pixel-added, it is possible to perform enlargement processing with good image quality.

Further, the control of the input terminal 206 of the image processing unit 3 may be switched depending on the use application of the portable information terminal 100 with an image sensor. For example, during low-illuminance shooting, the readout of the image sensor 2 is a pixel-added signal, and the image processing unit 3 is a phase correction process, which increases the sensitivity of adding pixels and increases noise. Can be reduced.
The present invention can be used for a mobile phone with a video camera function and an electronic zoom function.

DESCRIPTION OF SYMBOLS 1 Optical lens part 2 Image pick-up element part 3 Image processing part 4 DSP
5 Back-end processing unit 6 LSD controller unit 7 Display unit 8 CPU bus 9 Memory card 10, 16 DAC
11, 17, 23 Speaker 12 Key input unit 13 CPU
DESCRIPTION OF SYMBOLS 14 Power supply part 15 Power supply control part 18 Memory 19 Communication antenna 20 Radio | wireless circuit 21 Coding / decoding processing circuit 22 Microphone 100 Portable information terminal with an image sensor 201, 206, 301, 312 Input terminal 202, 302 Line delay processing part 203 Phase processing part 204 Enlargement processing unit 205 Selection unit 207, 314 Output terminal 206 Selection unit 303 Horizontal delay processing unit 304, 305, 306, 307 Horizontal filter 308, 309, 310, 311 Vertical filter 313 Operation processing unit

Claims (8)

An image processing apparatus that inputs either one of first image data having a phase information shift and second image data having no phase information shift, and performs image processing.
Buffer means for holding the first image data or the second image data and outputting pixels of the respective image data in a predetermined order;
If the input data is the first image data, phase correction is performed using pixels of the first image data output in a predetermined order from the buffer means, and the input data is the second image data. An image processing apparatus comprising: an arithmetic processing unit that performs an enlargement process using pixels of the second image data output from the buffer unit in a predetermined order. The buffer means includes
Line delay processing means for delaying and outputting the first image data or the second image data in units of lines;
Horizontal delay processing means for outputting the image data delayed in line units from the line delay processing means with a delay in sampling frequency units, and
The arithmetic processing means includes:
Using the pixels of the image data delayed in line units from the line delay processing means and the pixels of the image data delayed in sampling frequency units from the horizontal delay processing means, an interline and interpixel calculation is performed. Filter means;
If the input data is the first image data, the phase correction is performed on the pixels of the image data on which the interline and interpixel calculation has been performed by the filter means, and the input data is the second image data. If so, a calculation means for performing enlargement processing,
The image processing apparatus according to claim 1. The buffer means includes
Line delay processing means for delaying and outputting the first image data or the second image data in units of lines;
The arithmetic processing means includes:
If the input data is the first image data, phase correction means for performing phase correction using pixels of the image data delayed in line units from the line delay processing means;
If the input data is the second image data, an enlargement processing unit that performs an enlargement process using pixels of the image data delayed in line units from the line delay processing unit,
The image processing apparatus according to claim 1. A portable information terminal device including an image sensor,
Imaging means comprising a plurality of imaging elements that photoelectrically convert an optical image of a subject and output as a charge signal;
A readout method from the imaging unit includes a pixel addition readout method for outputting an imaging signal obtained by adding charge signals from a plurality of imaging elements selected at predetermined intervals in the vertical direction and / or the horizontal direction from the imaging unit, and the imaging Selecting a predetermined area in the vertical direction and / or the horizontal direction from the entire imaging area of the means, and selectively switching the area selection readout system for outputting the charge signal for the number of pixels as the imaging signal, the pixel addition readout system, Or an image sensor drive control means for outputting image data from the imaging means in any one of the region selective readout methods;
Buffer means for holding image data output from the image sensor drive control means by either the pixel addition reading method or the region selection reading method, and outputting the pixels of the respective image data in a predetermined order;
When the pixel addition reading method is selected, phase correction is performed using the pixels of the image data output in a predetermined order from the buffer means, and when the region selection reading method is selected, A portable information terminal device comprising: arithmetic processing means for performing enlargement processing using pixels of the image data output in a predetermined order from the buffer means. The buffer means includes
Line delay processing means for outputting the image data output from the image sensor drive control means by either the pixel addition reading method or the area selection reading method with a delay in line units;
Horizontal delay processing means for outputting the image data delayed in line units from the line delay processing means with a delay in sampling frequency units, and
The arithmetic processing means includes:
Using the pixels of the image data delayed in line units from the line delay processing means and the pixels of the image data delayed in sampling frequency units from the horizontal delay processing means, an interline and interpixel calculation is performed. Filter means;
When the pixel addition readout method is selected for the pixels of the image data for which the inter-line and inter-pixel calculation has been performed by the filter unit, the phase correction is performed and the region selection readout method is selected. A calculation means for performing enlargement processing, and
The portable information terminal device according to claim 4, wherein: The buffer means includes
Line delay processing means for outputting the image data output from the image sensor drive control means by either the pixel addition reading method or the area selection reading method with a delay in line units,
The arithmetic processing means includes:
When the pixel addition readout method is selected, a phase correction unit that performs phase correction using pixels of image data delayed in line units from the line delay processing unit;
An enlargement processing unit that performs an enlargement process using pixels of the image data delayed in line units from the line delay processing unit when the area selection readout method is selected;
The portable information terminal device according to claim 4, wherein: An image processing method for performing image processing by inputting one of first image data having a phase information shift and second image data having no phase information shift,
Holding the first image data or the second image data, and outputting the pixels of the respective image data in a predetermined order;
If the input data is the first image data, phase correction is performed using the pixels of the first image data output in the predetermined order. If the input data is the second image data, An image processing method comprising: an arithmetic processing step of performing an enlargement process using pixels of the second image data output in the predetermined order. Either one of the first image data in which the phase information is shifted and the second image data in which the phase information is not shifted is input to the computer of the image processing apparatus that performs image processing.
A buffer function for holding the first image data or the second image data and outputting the pixels of the respective image data in a predetermined order;
If the input data is the first image data, phase correction is performed using the pixels of the first image data output in the predetermined order. If the input data is the second image data, A program for executing an arithmetic processing function for performing an enlargement process using pixels of the second image data output in the predetermined order.

Images