US20050008230A1 - Image processing apparatus - Google Patents

Image processing apparatus Download PDF

Info

Publication number
US20050008230A1
US20050008230A1 US10/811,840 US81184004A US2005008230A1 US 20050008230 A1 US20050008230 A1 US 20050008230A1 US 81184004 A US81184004 A US 81184004A US 2005008230 A1 US2005008230 A1 US 2005008230A1
Authority
US
United States
Prior art keywords
data
image
image processing
buffer memory
processing unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/811,840
Other languages
English (en)
Inventor
Gen Sasaki
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
MegaChips Corp
Original Assignee
MegaChips Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by MegaChips Corp filed Critical MegaChips Corp
Assigned to MEGA CHIPS COPORATION reassignment MEGA CHIPS COPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SASAKI, GEN
Assigned to MEGA CHIPS CORPORATION reassignment MEGA CHIPS CORPORATION RECORD TO CORRECT THE ASSIGNEE'S NAME AND ADDRESS ON AN ASSIGNMENT PREVIOUSLY RECORDED ON REEL/FRAME 015161/0450 Assignors: SASAKI, GEN
Publication of US20050008230A1 publication Critical patent/US20050008230A1/en
Priority to US13/230,068 priority Critical patent/US8531546B2/en
Priority to US13/944,238 priority patent/US9082206B2/en
Priority to US13/944,173 priority patent/US9082205B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/21—Intermediate information storage
    • H04N1/2104—Intermediate information storage for one or a few pictures
    • H04N1/2112—Intermediate information storage for one or a few pictures using still video cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/21—Intermediate information storage
    • H04N1/2104—Intermediate information storage for one or a few pictures
    • H04N1/2112—Intermediate information storage for one or a few pictures using still video cameras
    • H04N1/2137—Intermediate information storage for one or a few pictures using still video cameras with temporary storage before final recording, e.g. in a frame buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N1/32358—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device using picture signal storage, e.g. at transmitter
    • H04N1/32491—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device using picture signal storage, e.g. at transmitter alternate storage in and retrieval from two parallel memories, e.g. using ping-pong buffers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H04N5/772—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera the recording apparatus and the television camera being placed in the same enclosure
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2101/00—Still video cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
    • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N2201/3285—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device using picture signal storage, e.g. at transmitter
    • H04N2201/329—Storage of less than a complete document page or image frame
    • H04N2201/3292—Storage of less than a complete document page or image frame of one or two complete lines

Definitions

  • the present invention relates to an image processing apparatus.
  • FIGS. 7 and 8 each schematically show a flow of image data captured by a conventional digital camera.
  • An interlace sensor 2 is used as a CCD sensor for image capturing in FIG. 7
  • a progressive sensor 18 is used in FIG. 8 .
  • image processing operation of a conventional digital camera will be discussed below.
  • the interlace sensor 2 first converts captured information as light about a subject into a digital signal which is then outputted as image data.
  • the outputted digital signal is subjected to noise control by correlated dual sampling, A/D conversion into a digital signal and the like at a signal processing unit 3 , and is thereafter sent to a main processor 4 .
  • the main processor 4 comprises a sensor processing unit (hereinafter referred to as “SPU”) 6 for performing signal processing such as white balance control on the received image data.
  • SPU sensor processing unit
  • the processed image data is stored in a main memory 5 .
  • the interlace sensor 2 outputs the captured image of one frame in two fields, namely, odd and even fields.
  • image processing using the SPU 6 and later units requires peripheral pixel information, and hence, information only about the odd field or about the even field is not responsive to the image processing.
  • subsequent process steps are put on hold until the interlace sensor 2 outputs the image data of two fields to reproduce the image data of one frame as original image data 16 on the main memory 5 .
  • a real-time processing unit (hereinafter referred to as “RPU”) 7 performs signal processing such as interpolation, color space conversion and pseudo color suppression on the image data written to a line memory 8 from the main memory 5 .
  • the image data processed by the RPU 7 is thereafter sent to an image compression and expansion unit 13 through line buffers A 24 and B 25 on the main memory 5 .
  • the line buffers A and B each normally have a capacity of 8 lines.
  • the RPU 7 continues to process subsequent image data read from the original image data 16 , which processed data is then sent to the line buffer B 25 .
  • the image data sent in advance to the line buffer A 24 is concurrently sent to the image compression and expansion unit 13 and is subjected to processing therein.
  • the RPU 7 continues to be operative to send subsequent image data to the empty line buffer A 24 , concurrently with which the image data sent in advance to the line buffer B 25 is sent to the image compression and expansion unit 13 . That is, the two line buffers A 24 and B 25 become alternately operative.
  • the image data read from the line buffers A 24 and B 25 are subjected to compression into a predetermined format such as JPEG at the image compression and expansion unit 13 , and are thereafter sent as compressed image data 17 to the main memory 5 .
  • the compressed image data 17 is subjected to processing by an image display unit 19 and a signal conversion unit 20 , whereby an image is displayed on a display device 21 , or the compressed image data 17 is stored in a memory card through a memory card interface.
  • the image display unit 19 uses a display buffer 29 as a working buffer memory.
  • the progressive sensor 18 as a CCD sensor for image capturing outputs the captured image of one frame at a time, and hence, generation of the original image data 16 on the main memory 5 is not required. Except for this, image processing using the progressive sensor 18 follows the same flow as the foregoing image processing using the interlace sensor 2 .
  • the CCD sensors 2 and 18 each have a growing number of pixels per horizontal line, reaching such a length that exceeds the horizontal line length of the line memory 8 of the RPU 7 in some cases. On the occurrence of such excess, regardless of whether the CCD sensor 2 or 18 is used, the image data outputted from the CCD sensor 2 or 18 cannot be directly processed by the RPU 7 . In response, even in the use of the progressive sensor 18 , the original image data 16 is generated once on the main memory 5 , and thereafter the subsequent process steps are followed. More specifically, with reference to FIG.
  • the original image data 16 is divided into left and right regions to avoid excess of the pixels per horizontal line of the original image data 16 over the horizontal line length of the line memory 8 of the RPU 7 .
  • the left region including regional data P 1 through P (n/2) is processed first to generate a region 27 including regional data Q 1 through Q (n/2) .
  • the right region including regional data P (n/2)+1 through P n is processed to generate a region 28 including regional data Q (n/2)+1 through Q n .
  • two regional data R 1 and R 2 on the left and right are synthesized to form intermediate data 26 of one frame on the main memory 5 which is then processed by the image compression and expansion unit 13 .
  • image data transmission should be repeated between the main processor 4 and the main memory 5 through a bus 14 . More specifically, a digital camera using the interlace sensor 2 requires five transmissions through the bus 14 as shown in FIG. 7 . Even a digital camera using the progressive sensor 18 requires three transmissions.
  • the CCD sensors 2 and 18 have a horizontal line of pixels reaching a length that exceeds the horizontal line length of the line memory 8 of the RPU 7 , in which case the regional data R 1 and R 2 on the left and right separately processed are synthesized into the intermediate data 26 which is then held in the main memory 5 .
  • the main memory 5 should be larger in capacity than required in the case in which the CCD sensors 2 and 18 contain pixels per horizontal line which do not reach the horizontal line length of the line memory 8 of the RPU 7 .
  • the foregoing data transmissions are handled by a DMA control unit 15 which realizes high-speed DMA transmission through the bus 14 with no intervention of a CPU.
  • the CCD sensors 2 and 18 are a 5-megapixel-class CCD sensor with 2560 pixels high by 1920 pixels wide each storing 16 bits of information, for example, the image data outputted from the CCD sensor 2 or 18 contains information as much as about 10 megabytes.
  • repetition of transmission of such high-volume data causes heavily places a burden on a bus band, thus leading to increase in power consumption, lower processing speed, and the like.
  • the present invention is intended for an image processing apparatus in which an image processing unit writes image data to a buffer memory, and a compression unit reads the image data from the buffer memory to perform compression thereon.
  • the image processing apparatus comprises: an image processing part for performing image processing on captured data of an image; and a storage unit provided outside the image processing part.
  • the image processing part includes: an image processing unit for performing a predetermined process on the captured data to obtain image data, and outputting the image data; a buffer memory for storing the image data sent from the image processing unit; and a compression unit for compressing the image data read from the buffer memory.
  • the buffer memory includes two buffer memories.
  • the image processing apparatus further comprises a control unit for controlling read and writing to and from these two buffer memories.
  • the control unit is operative in such a manner that while the image processing unit writes image data to either one of the two buffer memories, the compression unit selectively reads image data previously stored in the other one of the two buffer memories.
  • FIG. 1 schematically shows the configuration of a digital camera using an interlace sensor according to a first preferred embodiment of the present invention
  • FIG. 2 schematically shows the configuration of a digital camera using a progressive sensor according to the first preferred embodiment
  • FIG. 3 schematically shows the configuration of a digital camera according to a second preferred embodiment of the present invention
  • FIG. 4 schematically shows the configuration of a digital camera according to a third preferred embodiment of the present invention.
  • FIG. 5 schematically shows the configuration of a digital camera according to a fourth preferred embodiment of the present invention.
  • FIG. 6 shows process flow of regional data according to the fourth preferred embodiment
  • FIG. 7 schematically shows the configuration of a conventional digital camera using an interlace sensor
  • FIG. 8 schematically shows the configuration of a conventional digital camera using a progressive sensor
  • FIG. 9 shows conventional process flow of regional data.
  • FIG. 1 schematically shows a digital camera comprising an image processing apparatus 1 according to a first preferred embodiment of the present invention.
  • arrows indicate flow of image data captured by the digital camera.
  • the image processing apparatus 1 comprises a main processor 4 and a main memory 5 .
  • the image processing apparatus 1 is connected through a signal processing unit 3 to an interlace sensor 2 .
  • the main processor 4 comprises an SPU 6 , an RPU 7 including a line memory 8 , an image compression and expansion unit 13 , a local buffer A 9 (first buffer memory) and a local buffer B 10 (second buffer memory) serving to transmit data between the RPU 7 and the image compression and expansion unit 13 , a write control unit 11 and a read control unit 12 for selecting which one of the two local buffers A 9 or B 10 to use, a bus 14 for data transmission, and a DMA control unit 15 for controlling data transmission through the bus 14 between the main processor 4 and the main memory 5 .
  • the main memory 5 serves to temporarily store original image data 16 generated in the main processor 4 by data processing of odd and even fields sent from the interlace sensor 2 through the signal processing unit 3 , and compressed image data 17 sent from the image compression and expansion unit 13 .
  • the digital camera of the first preferred embodiment has the same configuration as that of the conventional digital camera.
  • the main processor 4 further comprises a CPU for various types of operations, an external memory card for storing captured image data, hardware serving as an interface for storage into the external memory card, and a storage device for storing a software program.
  • the SPU 6 receives image data from the interlace sensor 2 through the signal processing unit 3 to generate the original image data 16 on the main memory 5 .
  • the RPU 7 stores image data in the line memory 8 read from the original image data 16 and performs signal processing thereon such as interpolation, color space conversion and pseudo color suppression.
  • the digital camera of the first preferred embodiment sends the processed image data to the local buffers A 9 and B 10 in the main processor 4 , not to the line buffers A 24 and B 25 in the main memory 5 .
  • the processed image data of 8 lines is sent to the local buffer A 9 .
  • the RPU 7 thereafter continues to process subsequent image data of 8 lines read from the original image data 16 , and sends the processed image data to the local buffer B 10 .
  • the image data stored in the local buffer A 9 is sent to the image compression and expansion unit 13 for compression into a predetermined format such as JPEG.
  • the compressed image data is thereafter sent to the main memory 5 .
  • the RPU 7 still continues to process subsequent image data of 8 lines read from the original image data 16 and send the processed image data to the empty line buffer A 9 .
  • the image data stored in the local buffer B 10 is concurrently sent to the image compression and expansion unit 13 for compression into a predetermined format such as JPEG.
  • the compressed image data is thereafter sent to the main memory 5 . That is, while the RPU 7 sends image data either to the local buffer A 9 or to B 10 , image data stored either in the local buffer B 10 or in A 9 is sent to the image compression and expansion unit 13 to be compressed therein. These operations are repeatedly performed, whereby the compressed image data 17 is generated on the main memory 5 .
  • FIG. 2 shows the configuration of a digital camera using a progressive sensor 18 as a CCD sensor. Except for the progressive sensor 18 , the digital camera shown in FIG. 2 has the same configuration as that of the digital camera of FIG. 1 . In FIG. 2 , constituent elements which are operative in the same manner as those of FIG. 1 are designated by the same reference numerals.
  • the operation of the digital camera using the progressive sensor 18 is the same as that of the digital camera using the interlace sensor 2 , with exception that no original image data 16 is required, and data from the SPU 6 is directly sent to the RPU 7 to be processed therein.
  • the write and read control units 11 and 12 select which one of the two local buffers A 9 or B 10 to use by making connection to the RPU 7 and the image compression and expansion unit 13 .
  • the line buffers A 9 and B 10 are only required to be not less than image data processed at one time by the RPU 7 in horizontal line length. More specifically, as long as the local buffers A 9 and B 10 have a horizontal line not less than one horizontal line of pixels of the CCD sensors 2 and 18 , the local buffers A 9 and B 10 may be the same as or shorter than in horizontal line length the line memory 8 of the RPU 7 .
  • the horizontal line length of the local buffers A 9 and B 10 may contain 2600 pixels or 4096 pixels, as long as it is not less than the length of 2560 pixels.
  • the use of the local buffers A 9 and B 10 reduces the number of DMA transmissions of image data between the main processor 4 and the main memory 5 under control by the DMA control unit 15 until the compressed image data 17 is generated. That is, the conventional digital camera requires five transmissions in the use of the interlace sensor 2 and three transmissions in the use of the progressive sensor 18 as shown in FIGS. 7 and 8 , respectively, whereas with reference to FIGS. 1 and 2 , the digital camera of the first preferred embodiment only requires three transmissions and one transmission in the use of the interlace sensor 2 and the progressive sensor 18 , respectively. A burden placed on a bus band is lightened accordingly, whereby high-speed image processing is realized.
  • the first preferred embodiment further advantageously serves to reduce the frequency of use of the main memory 5 generally operating at a higher voltage than the main processor 4 as well as to reduce the number of operations of the DMA control unit 15 , thereby reducing power consumption of the image processing apparatus 1 as a whole. Additionally, provision of two local buffers allows continuous process flow from the SPU 6 to the image compression and expansion unit 13 , thereby advantageously realizing high-speed image processing.
  • the first preferred embodiment allows real-time data processing flow from the SPU 6 to the image compression and expansion unit 13 by way of the local buffers A 9 and B 10 without generating the original image data 16 once on the main memory 5 as shown in FIG. 2 .
  • the CCD sensors 2 and 18 have a horizontal line of pixels that exceeds in length the horizontal line length of the local buffers A 9 and B 10 , however, data sent from the RPU 7 cannot completely be stored in the local buffers A 9 and B 10 , whereby such sequential processing becomes impossible.
  • FIG. 3 schematically shows a digital camera comprising an image processing apparatus 1 according to the second preferred embodiment.
  • arrows indicate flow of image data captured by the digital camera.
  • the image processing apparatus 1 of the second preferred embodiment has the same configuration as that of the first preferred embodiment, whereas the local buffers A 9 and B 10 are operative in different ways between the first and second preferred embodiments.
  • constituent elements which are operative in the same manner as those of FIGS. 1 and 2 are designated by the same reference numerals.
  • the local buffers A 9 and B 10 are overflowed with data sent from the RPU 7 which prevents process flow.
  • the two local buffers A 9 and B 10 together serve as one continuous buffer memory as shown in FIG. 3 .
  • the horizontal line length of the local buffers A 9 and B 10 becomes responsive to data up to 5200 pixels per horizontal line which is twice the horizontal line length of each one of the local buffers A 9 and B 10 .
  • the use of the local buffers A 9 and B 10 as one continuous buffer memory does not allow the RPU 7 and the image compression and expansion unit 13 to follow sequential process flow by means of the alternate use of the two local buffers A 9 and B 10 as discussed in the first preferred embodiment.
  • a control signal is transmitted between the RPU 7 and the image compression and expansion unit 13 .
  • the RPU 7 first processes image data of one line read from the original image data 16 , and sends the processed image data to the buffer memory as a unity of the two local buffers A 9 and B 10 .
  • the RPU 7 stops the operation.
  • the image compression and expansion unit 13 serves to process the data read from the buffer memory.
  • the image compression and expansion unit 13 sends a control signal to the RPU 7 notifying completion of data readout.
  • the RPU 7 again processes image data read from the original image data 16 , and sends the processed image data to the buffer memory.
  • the second preferred embodiment is applicable even in the use of the progressive sensor 18 as a CCD sensor. More specifically, data from the SPU 6 is not directly sent to the RPU 7 , but is sent to the main memory 5 once to generate the original image data 16 .
  • the two local buffers A 9 and B 10 are conceptually shown as one buffer memory in physical connection in a horizontal direction.
  • the actual use of the local buffers A 9 and B 10 is such that under control by a software program, for example, the RPU 7 sends data to the local buffer A 9 , and overflow data from the local buffer A 9 is sequentially sent to the local buffer B 10 .
  • This process flow is also followed in reading to the image compression and expansion unit 13 .
  • the write and read control units 11 and 12 are collectively controlled, the second preferred embodiment is feasible in the configurations shown in FIGS. 1 and 2 . That is, the configurations of FIGS. 1 and 2 can be subjected to the process flows of the first and second preferred embodiments.
  • the second preferred embodiment has been described in the use of the two local buffers A 9 and B 10 as discussed in the first preferred embodiment.
  • one local buffer may be provided in the image processing apparatus 1 that has the same capacity as that of the line memory 8 of the RPU 7 .
  • the second preferred embodiment is also feasible by the use of such a local buffer and the foregoing control signal.
  • the local buffers A 9 and B 10 serve to transmit data from the RPU 7 to the image compression and expansion unit 13 .
  • the local buffers A 9 and B 10 may be operative for another processing performed in the main processor 4 .
  • the local buffers A 9 and B 10 are operative for image display on a display device 21 .
  • a digital camera may use an electronic viewfinder or a liquid crystal display.
  • image data sent from the CCD sensor 2 or 18 passes through the signal processing unit 3 and the SPU 6 , and is thereafter sent through the bus 14 to the image display unit 19 for conversion.
  • the converted image data is further sent through the signal conversion unit 20 to the display device 21 such as an electronic viewfinder or a liquid crystal display, whereby an image is reproduced on the display device 21 .
  • reproduction of the subject should be continuously made on the display device 21 , and should not be limited to a capturing period. Accordingly, data should be continuously sent to the image display unit 19 , thus heavily placing a burden on the bus 14 serving for this data transmission.
  • the local buffers A 9 and B 10 serve to transmit data to the image display unit 19 to lighten a burden on a bus band.
  • FIG. 4 schematically shows a digital camera comprising an image processing apparatus 1 according to the third preferred embodiment.
  • arrows indicate flow 45 of image data captured by the digital camera.
  • Constituent elements which are operative in the same manner as those of FIGS. 1, 2 and 3 are designated by the same reference numerals.
  • Image data sent from the CCD sensor 2 or 18 is subjected to the processing at the SPU 6 .
  • the processed image data is thereafter sent to the RPU 7 after generation of the original image data 16 in the use of the interlace sensor 2 , whereas in the use of the progressive sensor 18 , it is directly sent to the RPU 7 .
  • the captured image data from the RPU 7 is sent through the local buffers A 9 and B 10 to the image compression and expansion unit 13 and to the image display unit 19 .
  • the captured image data is subjected to the processing at the image compression and expansion unit 13 during image capturing, whereas in visual recognition of a subject to be captured, it is subjected to the processing at the image display unit 19 .
  • image data sent from the RPU 7 during visual recognition is skipped with 240 pixels high by 320 pixels wide, for example.
  • the skipped image data is subjected to the sequential process flow as discussed in the first preferred embodiment including writing from the RPU 7 and reading to the image display unit 19 by means of alternate use of the two local buffers A 9 and B 10 under control by the write and read control units 11 and 12 .
  • Continuity between the RPU 7 and the local buffers A 9 and B 10 , and continuity between the local buffers A 9 and B 10 , and the image compression and expansion unit 13 and the image display unit 19 , are controlled by purpose-built hardware and a purpose-built software program having a switching function of continuity such as the write and read control units 11 and 12 .
  • the output from the RPU 7 is sent to the local buffers A 9 and B 10 , and the output from the local buffers A 9 and B 10 is selectively sent either to the image compression and expansion unit 13 or to the image display unit 19 . That is, the local buffers A 9 and B 10 are selectively available either to the image compression and expansion unit 13 or to the image display unit 19 . More specifically, between the image compression and expansion unit 13 and the image display unit 19 , the local buffers A 9 and B 10 are allocated to the one placing a heavier burden on the bus 14 .
  • data from the RPU 7 is directly sent through the local buffers A 9 and B 10 to the image display unit 19 without data transmission through the bus 14 for image reproduction as required in the conventional digital camera.
  • a burden placed on a bus band is lightened accordingly, to thereby reduce power consumption.
  • the lightened burden on a bus band further advantageously leads to enhanced processing speed of the image processing apparatus 1 as a whole.
  • the original image data 16 is divided into left and right regions. These left and right regions are separately processed at the RPU 7 by means of the line memory 8 to generate the intermediate data 26 on the main memory 5 .
  • the intermediate data 26 is subsequently subjected to the processing at the image compression and expansion unit 13 to generate the compressed image data 17 .
  • Such process flow can handle the CCD sensors 2 and 18 having a horizontal line of pixels exceeding in length the horizontal line length of the line memory 8 of the RPU 7 , whereas a region for storing the high-volume intermediate data 26 is required in the main memory 5 that contains pixels corresponding in number to those of the CCD sensors 2 and 18 .
  • a region for storing the high-volume intermediate data 26 is required in the main memory 5 that contains pixels corresponding in number to those of the CCD sensors 2 and 18 .
  • FIG. 5 schematically shows a digital camera comprising an image processing apparatus 1 according to the fourth preferred embodiment.
  • arrows indicate flow of image data captured by the digital camera.
  • FIG. 6 is an explanatory view sequentially showing generation of data on the main memory 5 . With reference to FIGS. 5 and 6 , the operation of the digital camera according to the fourth preferred embodiment will be discussed below.
  • the original image data 16 is generated once in the main memory 5 regardless of whether the CCD sensor 2 or 18 is used.
  • the original image data 16 is divided and is then subjected to the processing at the RPU 7 by means of the line memory 8 .
  • the original image data 16 is divided into left and right regions in two columns. The left and right regions are each further horizontally divided into regions of 8 lines.
  • intermediate data R 1 ( 23 ) of 8 lines is generated in the main memory 5 that has a horizontal line length of pixels corresponding in number to those of the horizontal line of the CCD sensors 2 and 18 .
  • the image compression and expansion unit 13 handles data on a block unit basis of 8 pixels high by 8 pixels wide, and hence, is capable of handling data R 1 to generate data S 1 . These process steps are sequentially performed until the compressed image data 17 is generated.
  • the RPU 7 processes the regional data including P 1 , P 2 , . . . , and the resultant data R 1 , R 2 , . . . are subjected to the processing at the image compression and expansion unit 13 .
  • the CCD sensors 2 and 18 have a horizontal line of pixels exceeding in length the horizontal line length of the line memory 8 , process flow continues without waiting for generation of intermediate data of one frame on the main memory 5 as required in the conventional digital camera.
  • the fourth preferred embodiment further advantageously lowers working buffer capacity in the main memory 5 . Still further, provision of two lines buffers A 22 and B 23 in the main memory 5 , and alternate use of these buffers A 22 and B 23 as discussed with reference to the local buffers A 9 and B 10 in the first and second preferred embodiments realize image processing with still higher speed.
  • the CCD sensors 2 and 18 may have a horizontal line of pixels exceeding in length the line memory 8 of the RPU 7 as a result of higher pixel resolution of the CCD sensors 2 and 18 , in which case the fourth preferred embodiment devises reading of image data from the original image data 16 to realize image processing.
  • the fourth preferred embodiment requires a region for storing intermediate data in the main memory 5 , thereby placing a heavier burden on the bus 14 and causing higher power consumption than the first and second preferred embodiments using the local buffers A 9 and B 10 .
  • a fifth preferred embodiment of the present invention as a combination of the second and fourth preferred embodiments is responsive to these problems.
  • the fifth preferred embodiment employs the one shown in FIG. 3 discussed in the second preferred embodiment.
  • image processing only the reading flow of the original image data 16 discussed in the fourth preferred embodiment with reference to FIG. 6 is adopted.
  • the original image data 16 generated on the main memory 5 after passing through the CCD sensor 2 or 18 and the SPU 6 is read as the regional data P 1 and P 2 in this order as shown in FIG. 6 .
  • the regional data P 1 and P 2 thereby read are subjected to the processing at the RPU 7 by means of the line memory 8 .
  • the processed data are sent to the local buffers A 9 and B 10 , not to the line buffer A 22 in the main memory 5 .
  • the data R 1 ( 23 ) shown in FIG. 6 is generated in the local buffers A 9 and B 10 .
  • the RPU 7 stops reading of the original image data 16 .
  • the image compression and expansion unit 13 reads the data from the local buffers A 9 and B 10 and performs compression thereon.
  • the compressed data is thereafter sent as the compressed image data 17 to the main memory 5 .
  • the data S 1 shown in FIG. 6 is generated as the compressed image data 17 on the main memory 5 .
  • the image compression and expansion unit 13 sends a control signal notifying completion of data readout.
  • the RPU 7 continues to read the regional data P 3 and P 4 from the original image data 16 , and performs image processing thereon.
  • the processed data are sent to the local buffers A 9 and B 10 .
  • the fifth preferred embodiment has been discussed as a combination of the fourth and second preferred embodiments. As long as the CCD sensors 2 and 18 have a horizontal line of pixels that does not exceed in length the horizontal line length of the local buffers A 9 and B 10 , the first preferred embodiment may alternatively be combined with the fourth preferred embodiment. Still alternatively, a combination of the third and fourth preferred embodiments will be applicable.
  • the use of the local buffers A 9 and B 10 of the second preferred embodiment combined with the fourth preferred embodiment eliminates the need for generating intermediate data on the main memory 5 .
  • Such combination further advantageously lightens a burden to be placed on the bus 14 and reduces power consumption.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
  • Television Signal Processing For Recording (AREA)
  • Storing Facsimile Image Data (AREA)
  • Image Processing (AREA)
US10/811,840 2003-03-31 2004-03-30 Image processing apparatus Abandoned US20050008230A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US13/230,068 US8531546B2 (en) 2003-03-31 2011-09-12 Image processing apparatus having a buffer memory for image data storage
US13/944,238 US9082206B2 (en) 2003-03-31 2013-07-17 Image processing apparatus having a buffer memory for image data storage
US13/944,173 US9082205B2 (en) 2003-03-31 2013-07-17 Image processing apparatus for processing image data to produce compressed image data and display data

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2003093075A JP4245139B2 (ja) 2003-03-31 2003-03-31 画像処理装置
JP2003-093075 2003-03-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US13/230,068 Division US8531546B2 (en) 2003-03-31 2011-09-12 Image processing apparatus having a buffer memory for image data storage

Publications (1)

Publication Number Publication Date
US20050008230A1 true US20050008230A1 (en) 2005-01-13

Family

ID=33405953

Family Applications (4)

Application Number Title Priority Date Filing Date
US10/811,840 Abandoned US20050008230A1 (en) 2003-03-31 2004-03-30 Image processing apparatus
US13/230,068 Expired - Lifetime US8531546B2 (en) 2003-03-31 2011-09-12 Image processing apparatus having a buffer memory for image data storage
US13/944,173 Expired - Lifetime US9082205B2 (en) 2003-03-31 2013-07-17 Image processing apparatus for processing image data to produce compressed image data and display data
US13/944,238 Expired - Lifetime US9082206B2 (en) 2003-03-31 2013-07-17 Image processing apparatus having a buffer memory for image data storage

Family Applications After (3)

Application Number Title Priority Date Filing Date
US13/230,068 Expired - Lifetime US8531546B2 (en) 2003-03-31 2011-09-12 Image processing apparatus having a buffer memory for image data storage
US13/944,173 Expired - Lifetime US9082205B2 (en) 2003-03-31 2013-07-17 Image processing apparatus for processing image data to produce compressed image data and display data
US13/944,238 Expired - Lifetime US9082206B2 (en) 2003-03-31 2013-07-17 Image processing apparatus having a buffer memory for image data storage

Country Status (2)

Country Link
US (4) US20050008230A1 (ja)
JP (1) JP4245139B2 (ja)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060101116A1 (en) * 2004-10-28 2006-05-11 Danny Rittman Multifunctional telephone, walkie talkie, instant messenger, video-phone computer, based on WiFi (Wireless Fidelity) and WiMax technology, for establishing global wireless communication, network and video conferencing via the internet
US20060140498A1 (en) * 2004-12-28 2006-06-29 Fujitsu Limited Apparatus and method for processing an image
US20060146076A1 (en) * 2004-12-08 2006-07-06 Chung-Hsun Huang Image processing module with less line buffers
US20070065022A1 (en) * 2005-09-22 2007-03-22 Nec Electronics Corporation Image signal processing apparatus and method
US20080316221A1 (en) * 2007-06-19 2008-12-25 Aas Eric F Down-sampled image display
US20130083233A1 (en) * 2011-10-04 2013-04-04 Sony Corporation Image pickup unit
CN103140873A (zh) * 2010-09-29 2013-06-05 英泰克普拉斯有限公司 影像数据高速收发方法及装置
TWI413943B (zh) * 2009-02-11 2013-11-01 Silicon Motion Inc 影像處理系統及其影像處理方法
US10674154B2 (en) * 2009-10-05 2020-06-02 Beamr Imaging Ltd Apparatus and methods for recompression of digital images

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4925271B2 (ja) * 2006-08-21 2012-04-25 株式会社メガチップス 一眼レフデジタルカメラにおける連続撮像方法
EP2302845B1 (en) 2009-09-23 2012-06-20 Google, Inc. Method and device for determining a jitter buffer level
US8838680B1 (en) 2011-02-08 2014-09-16 Google Inc. Buffer objects for web-based configurable pipeline media processing
TW201242346A (en) * 2011-04-12 2012-10-16 Altek Corp Image capturing device and image processing method thereof
JP5103678B2 (ja) * 2011-04-27 2012-12-19 株式会社メガチップス 一眼レフデジタルカメラにおける連続撮像方法
TWI520598B (zh) * 2012-05-23 2016-02-01 晨星半導體股份有限公司 影像處理裝置與影像處理方法
US8917336B2 (en) * 2012-05-31 2014-12-23 Apple Inc. Image signal processing involving geometric distortion correction
US8928680B1 (en) 2012-07-10 2015-01-06 Google Inc. Method and system for sharing a buffer between a graphics processing unit and a media encoder
CN103414877A (zh) * 2013-08-01 2013-11-27 天津天地伟业数码科技有限公司 基于高速存储器动态突发长度的图像倍帧处理方法

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5694166A (en) * 1991-09-04 1997-12-02 Canon Kabushiki Kaisha Video camera with slower than standard read-out from CCD
US6002446A (en) * 1997-02-24 1999-12-14 Paradise Electronics, Inc. Method and apparatus for upscaling an image
US6137534A (en) * 1997-07-10 2000-10-24 Flashpoint Technology, Inc. Method and apparatus for providing live view and instant review in an image capture device
US6233010B1 (en) * 1990-11-20 2001-05-15 St. Clair Intellectual Property Consultants, Inc. Electronic still video camera with direct personal computer (PC) compatible digital format output
US20010019362A1 (en) * 2000-02-22 2001-09-06 Kenji Nakamura Digital camera and recording medium
US6400471B1 (en) * 1999-02-11 2002-06-04 Flashpoint Technology, Inc. Flexible architecture for image processing
US6563535B1 (en) * 1998-05-19 2003-05-13 Flashpoint Technology, Inc. Image processing system for high performance digital imaging devices
US6847388B2 (en) * 1999-05-13 2005-01-25 Flashpoint Technology, Inc. Method and system for accelerating a user interface of an image capture unit during play mode

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS61164377A (ja) 1985-01-16 1986-07-25 Sanyo Electric Co Ltd 符号，復号方式
JPH05128232A (ja) 1991-11-08 1993-05-25 Mitsubishi Electric Corp イメージのフアイリング方法およびイメージ処理装置
US5450599A (en) * 1992-06-04 1995-09-12 International Business Machines Corporation Sequential pipelined processing for the compression and decompression of image data
JPH06245013A (ja) 1993-02-19 1994-09-02 Murata Mach Ltd ファクシミリ装置
US7123772B2 (en) * 2001-03-28 2006-10-17 Texas Instruments Incorporated Image compression by differences within a strip
JPH09245157A (ja) 1996-03-12 1997-09-19 Canon Inc 画像形成装置
US5745603A (en) * 1996-08-26 1998-04-28 Hewlett-Packard Company Two dimensional context model obtained without a line buffer for arithmetic coding
KR100314801B1 (ko) * 1998-12-22 2002-01-15 박종섭 이미지센서에서화면을패닝및스켈링하기위한장치
JP3532781B2 (ja) * 1999-02-12 2004-05-31 株式会社メガチップス 画像入力装置の画像処理回路
JP3961736B2 (ja) 2000-02-04 2007-08-22 株式会社リコー 画像処理装置
JP2002125105A (ja) 2000-10-13 2002-04-26 Ricoh Co Ltd 画像処理装置、画像処理方法及び該方法を実行するプログラムを格納したコンピュータ読み取り可能な記憶媒体
JP2002237951A (ja) 2001-02-09 2002-08-23 Matsushita Electric Ind Co Ltd 映像信号処理回路
JP4112187B2 (ja) * 2001-03-01 2008-07-02 富士フイルム株式会社 画像処理方法および装置並びにプログラム
JP2003244529A (ja) * 2002-02-20 2003-08-29 Minolta Co Ltd デジタルカメラ

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6233010B1 (en) * 1990-11-20 2001-05-15 St. Clair Intellectual Property Consultants, Inc. Electronic still video camera with direct personal computer (PC) compatible digital format output
US5694166A (en) * 1991-09-04 1997-12-02 Canon Kabushiki Kaisha Video camera with slower than standard read-out from CCD
US6002446A (en) * 1997-02-24 1999-12-14 Paradise Electronics, Inc. Method and apparatus for upscaling an image
US6137534A (en) * 1997-07-10 2000-10-24 Flashpoint Technology, Inc. Method and apparatus for providing live view and instant review in an image capture device
US6563535B1 (en) * 1998-05-19 2003-05-13 Flashpoint Technology, Inc. Image processing system for high performance digital imaging devices
US6400471B1 (en) * 1999-02-11 2002-06-04 Flashpoint Technology, Inc. Flexible architecture for image processing
US6847388B2 (en) * 1999-05-13 2005-01-25 Flashpoint Technology, Inc. Method and system for accelerating a user interface of an image capture unit during play mode
US20010019362A1 (en) * 2000-02-22 2001-09-06 Kenji Nakamura Digital camera and recording medium

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060101116A1 (en) * 2004-10-28 2006-05-11 Danny Rittman Multifunctional telephone, walkie talkie, instant messenger, video-phone computer, based on WiFi (Wireless Fidelity) and WiMax technology, for establishing global wireless communication, network and video conferencing via the internet
US20060146076A1 (en) * 2004-12-08 2006-07-06 Chung-Hsun Huang Image processing module with less line buffers
US8269805B2 (en) * 2004-12-08 2012-09-18 Himax Technologies Limited Image processing module with less line buffers
US7583851B2 (en) * 2004-12-28 2009-09-01 Fujitsu Microelectronics Limited Apparatus and method for processing an image
US20060140498A1 (en) * 2004-12-28 2006-06-29 Fujitsu Limited Apparatus and method for processing an image
US20070065022A1 (en) * 2005-09-22 2007-03-22 Nec Electronics Corporation Image signal processing apparatus and method
US20080316221A1 (en) * 2007-06-19 2008-12-25 Aas Eric F Down-sampled image display
TWI413943B (zh) * 2009-02-11 2013-11-01 Silicon Motion Inc 影像處理系統及其影像處理方法
US10674154B2 (en) * 2009-10-05 2020-06-02 Beamr Imaging Ltd Apparatus and methods for recompression of digital images
CN103140873A (zh) * 2010-09-29 2013-06-05 英泰克普拉斯有限公司 影像数据高速收发方法及装置
US20130176443A1 (en) * 2010-09-29 2013-07-11 Intekplus Co., Ltd Method and device for transmitting/receiving image data at high speed
US9218050B2 (en) * 2010-09-29 2015-12-22 Intekplus Co., Ltd. Method and device for transmitting/receiving image data at high speed
US20130083233A1 (en) * 2011-10-04 2013-04-04 Sony Corporation Image pickup unit

Also Published As

Publication number Publication date
US9082206B2 (en) 2015-07-14
US8531546B2 (en) 2013-09-10
US9082205B2 (en) 2015-07-14
US20120002080A1 (en) 2012-01-05
JP4245139B2 (ja) 2009-03-25
US20130300898A1 (en) 2013-11-14
US20130300756A1 (en) 2013-11-14
JP2004304387A (ja) 2004-10-28

Similar Documents

Publication Publication Date Title
US9082206B2 (en) Image processing apparatus having a buffer memory for image data storage
JP4993856B2 (ja) 画像変換装置、画像変換のための直接メモリアクセス装置および画像変換を支援するカメラインターフェース
KR101128167B1 (ko) 화상 표시 장치
JP4263190B2 (ja) 映像合成回路
US6593965B1 (en) CCD data pixel interpolation circuit and digital still camera equipped with it
EP1531399A2 (en) Method and system for preventing data transfer bus bottlenecks through image compression
US7023455B2 (en) Apparatus and method for processing video signal
US6356306B1 (en) Digital camera capable of converting a progressive scan signal into an interlace scan signal
JP2000092375A (ja) 信号処理装置及びその制御方法
JP6722278B2 (ja) 画像処理装置
US6661452B1 (en) Digital camera capable of decreasing a required memory capacity
US7391932B2 (en) Apparatus and method for selecting image to be displayed
JP4158245B2 (ja) 信号処理装置
US8085345B2 (en) Image processing apparatus and image processing method
CN113658049A (zh) 一种图像转置的方法、设备和计算机可读存储介质
JP3178665B2 (ja) 画像サイズ変換方法とそのための装置
JP2000224540A (ja) 画像ファイル装置
JP2002101310A (ja) フィルタ処理装置及び方法
US20100254618A1 (en) Method for Accessing Image Data and Related Apparatus
JP2000092349A (ja) 画像処理装置
JP4211572B2 (ja) 撮像装置
JP2007124075A (ja) 画像処理装置、画像処理方法、撮像記録装置および撮像記録方法
JP2004328405A (ja) 動画記録再生装置、その静止画圧縮記録方法およびプログラム
JP4439746B2 (ja) 情報処理装置
JP2002237953A (ja) 画像データ処理装置およびその方法、ならびにカメラ・システム

Legal Events

Date Code Title Description
AS Assignment

Owner name: MEGA CHIPS COPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SASAKI, GEN;REEL/FRAME:015161/0450

Effective date: 20040322

AS Assignment

Owner name: MEGA CHIPS CORPORATION, JAPAN

Free format text: RECORD TO CORRECT THE ASSIGNEE'S NAME AND ADDRESS ON AN ASSIGNMENT PREVIOUSLY RECORDED ON REEL/FRAME 015161/0450;ASSIGNOR:SASAKI, GEN;REEL/FRAME:015856/0728

Effective date: 20040322

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION