JP6882007B2 - Image processing device and its control method and program - Google Patents

Description

The present invention relates to an image data coding technique.

In recent years, as the number of pixels of an image handled by an image processing device such as a digital camera has increased, the amount of image data to be processed has also increased. Even when the image processing circuit performs simple filtering on one image, the circuit requires a line memory that holds pixels for one horizontal line of the image to be processed, and the circuit scale increases. Resulting in. Therefore, in order to save line memory, the image to be processed is divided and processed. In addition, an increase in the amount of data puts pressure on the memory bandwidth and reduces the frame rate of moving images and the number of continuous shots of still images. Therefore, when the divided image is temporarily stored in the memory, a technique of compressing the divided image data has been proposed in Patent Document 1 and Patent Document 2. Patent Document 1 proposes to divide a divided image into a plurality of sub-block lines, compress each sub-block line, and determine a memory write address based on the original image size and the compression rate. Patent Document 2 proposes that the division position of the image data is randomly processed for each line, and a dummy area is added to the end of the divided image in order to match the image data with the pixel unit to be compressed.

Japanese Patent No. 5597175 JP-A-2015-220543

However, in the technique disclosed in Patent Document 1, since compression is performed for each sub-block line, if the horizontal size of the divided image is not an integral multiple of the sub-block line, it is necessary to add the insufficient pixels as the sub-block. There is. That is, even pixels that do not originally need to be processed are processed and compressed, which causes a decrease in processing performance and compression efficiency. For this reason, countermeasures have been desired for such problems. Further, in the technique disclosed in Patent Document 2, since a dummy region is added, it causes a decrease in compression efficiency, and a countermeasure against it has been desired.

In view of the above problems, the present invention is intended to provide a technique for suppressing a decrease in processing performance and compression coding efficiency.

In order to solve this problem, for example, the image processing apparatus of the present invention has the following configuration. That is,
An image processing device that encodes image data.
A dividing means for dividing the image data to be encoded into a plurality of divided image data, and
A coding means that encodes a predetermined number of consecutive pixel data as a unit, and
It has a control means for controlling the coding means and for encoding each of the divided image data obtained by the dividing means.
The control means
A determination means for determining whether or not the size of the divided image data to be encoded by the coding means is an integral multiple of the unit.
It said determining means, when the size of the divided image data of the encoding target is determined not to be an integral multiple of the unit, possess an additional means for adding the missing pixel for an integer multiple of said unit,
The additional means
It has an analysis means for performing a predetermined analysis on a region from the boundary of the divided image data to a predetermined distance and determining whether or not the region represents a low frequency component.
When the analysis result obtained that the region represents a low frequency component by the analysis means, a copy of the pixel located inside the boundary is generated as a pixel to be added.
When an analysis result that the region does not represent a low frequency component is obtained by the analysis means, a pixel generated by a predetermined random number is generated as an additional pixel .

According to the present invention, it is possible to suppress a decrease in processing performance and compression efficiency related to image data coding.

The block diagram of the image pickup apparatus in 1st Embodiment. The block diagram of the image processing part in 1st Embodiment. The figure which shows the pixel addition in 1st Embodiment. The figure which shows the example of the image division in 1st Embodiment. The flowchart which shows the determination process of the divided image size in 1st Embodiment. The flowchart which shows the additional pixel determination process in 1st Embodiment. The figure which shows the example of the image division in 2nd Embodiment. The flowchart which shows the determination process of the divided image size in 2nd Embodiment. The block diagram of the image pickup apparatus in 3rd Embodiment. The block diagram of the image processing part in 3rd Embodiment. The figure which shows the example of the image division in 3rd Embodiment. The flowchart which shows the additional pixel determination process in 3rd Embodiment. The block diagram of the image pickup apparatus in 4th Embodiment. The block diagram of the image processing part in 4th Embodiment. The figure which shows the example of the image division in 4th Embodiment. The flowchart which shows the compression unit determination process in 4th Embodiment. The block diagram of RDDMAC in the 4th Embodiment. The flowchart which shows the extension unit determination process in 4th Embodiment. The block diagram of the image processing part in 5th Embodiment. The figure which shows the example of the image division in 5th Embodiment. The flowchart which shows the compression unit determination process in 5th Embodiment. The flowchart which shows the extension unit determination process in 5th Embodiment.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the following, the image processing device according to the present embodiment will be described as being mounted as a part of the image pickup device. Further, although the image pickup device will be described as a digital camera, it is also possible to adopt an image pickup device such as a digital video camera, a smartphone, a camera-equipped mobile phone, or an in-vehicle camera.

[First Embodiment]
FIG. 1 is a block diagram showing an example of the configuration of the digital camera according to the first embodiment. The digital camera includes an image processing unit 100, a compression unit 101, a data transfer unit 102, a data bus 103, a control bus 104, a memory control unit 105, a memory 106, a non-volatile memory control unit 107, a non-volatile memory 108, and a CPU 109. Although not shown, the digital camera converts the received subject image into an electric signal to create image data, an image sensor such as a CCD or CMOS sensor, and an AD that converts an analog signal from the image sensor into a digital signal. It also has a converter, a compression / decompression unit that compresses or decompresses image data into, for example, a PEG or MPEG format, and an operation unit operated by a user.

The image processing unit 100 is connected to the data transfer unit 102, inputs image data obtained by imaging with an imaging element (not shown) via the data transfer unit 102, performs image processing, and transfers the processing result to data. Output to unit 102 or compression unit 101. As shown in FIG. 2, the image processing unit 100 includes an image processing core unit 200 and a pixel addition unit 201. The image processing core unit 200 has a plurality of processes such as pixel correction, black level correction, shading correction, scratch correction, magnification chromatic aberration correction, gamma correction, brightness / color generation processing, geometric deformation, noise reduction, and image resizing (enlargement / reduction). Is a block that performs appropriate image processing on the image data. The pixel addition unit 201 sets copy pixels or pixels of random values (random numbers) from the CPU 109 with respect to the image data processed by the image processing core unit 200 or the image data transferred from the data transfer unit 102. It's a block to add for a few minutes.

An example of adding pixels will be described with reference to FIGS. 3A and 3B. 3 (A) and 3 (B) show one horizontal line of image data, and mean a pixel to which a shaded portion is added. FIG. 3A is image data of the Bayer array, and FIG. 3B is image data of the YUV422 format. In the case of the Bayer array, copying is performed line-symmetrically at the boundary position in units of 2 pixels. In the case of the YUV422 format, the Y pixel is copied from the end and the UV pixel is copied in units of 2 pixels in line symmetry at the boundary position. Note that the addition of pixels shown in FIGS. 3A and 3B is an example, and it is possible to copy from a position other than the end, or to add a random value as a pixel instead of copying. ..

The compression unit 101 inputs the image data processed by the image processing unit 100 as a coding target, and is in units of sub-blocks such as a predetermined number of pixels (32 pixels in the embodiment) that are continuous in the horizontal direction. , Compressed and coded by differential pulse code modulation (DPCM) method. The number of pixels of the subblock, which is a coding unit, is not limited to 32 pixels, and may be 64 pixels, and the size (number of pixels) is not particularly limited. Further, the coding method may be another method.

The data transfer unit 102 has a plurality of Direct Memory Access controllers (DMACs) that transfer data, and includes a WRDMAC for writing to the memory 106 and an RDDMAC for reading. The image data is output to the bus 103 by WRDMAC and temporarily stored in the memory 106 via the memory control unit 105. The image data temporarily stored in the memory 106 is output from the memory 106 to the bus 103 by the RDDMAC, and the data is output to the image processing unit 100. The bus 103 is a data bus, and the bus 104 is a control bus.

The memory control unit 105 writes data to the memory 106 and reads data from the memory 106 in response to an instruction from the CPU 109 or the data transfer unit 102. The memory 106 is a storage device having a storage capacity sufficient for storing a predetermined number of still images, moving images for a predetermined time, data such as voice, constants for operation of the CPU 109, programs, and the like, and is composed of a DRAM or the like. Will be done.

The non-volatile memory control unit 107 writes data to the non-volatile memory 108 and reads data from the non-volatile memory 108 in response to an instruction from the CPU 109. The non-volatile memory 108 is a memory that can be electrically erased and recorded, and for example, EEPROM or the like is used. The non-volatile memory 108 stores constants, programs, and the like for the operation of the CPU 109.

The CPU 109 is composed of a microcomputer or the like that controls the operation of the digital camera, gives various instructions to each functional block constituting the digital camera, and executes various control processes. For example, the CPU 109 controls an image processing unit 100, a data transfer unit 102, a memory control unit 105, and a non-volatile memory control unit 107 connected via a bus 104. The CPU 109 realizes each process of the present embodiment by executing the program recorded in the non-volatile memory 108 described above. Further, the CPU 109 determines the division size at the time of the image data division processing. Here, an example of image division by the CPU 109 in the embodiment is shown in FIG. 4, and a procedure for determining the division size is shown in FIG.

The image division process in the embodiment will be described with reference to FIG. In the image division processing in the embodiment, the divided image data is determined at the center position in the horizontal direction of the image data to be encoded, and then the other divided image data is determined toward both ends in the horizontal direction. In the figure, the image data 400 shows an unencoded image obtained by imaging. In the embodiment, this image is divided into three as shown by a broken line to generate divided image data 401, 404, and 406. The dotted line blocks 403, 405, 408 and other dotted line blocks without reference numerals indicate subblocks that are compression processing units by the compression unit 101, and the horizontal size of the subblocks is the image processing unit. It depends on the size capacity of the line memory provided in 100.

The shaded areas 402 and 407 indicate pixels to be added by the pixel addition unit 201. Although FIG. 4 shows three divisions, the number of divisions is not limited to three, and three or more divisions may be used.

Next, the processing procedure of the CPU 109 that performs the process of determining the division size of the image data shown in FIG. 4 will be described with reference to the flowchart of FIG.

In S500, the CPU 109 determines the size of the divided image data 404 in the horizontal direction so that the dotted line block 405 fits well.

For example, in the embodiment, since the horizontal size of the subblock is 32 pixels, the horizontal size of the divided image data 404 is an integral multiple of 32 pixels. Specifically, the CPU 109 calculates the horizontal size b1 of the divided image data 404 so that the following mathematical formulas (1) and (2) hold.
b1% n1 = 0 ... (1)
m1 ≧ b1 ... (2)
Here, X% Y is a function that returns the remainder when the integer X is divided by the integer Y. Further, b1 is the number of horizontal pixels of the divided image data 404, m1 is the number of pixels that can be stored in the delay line, and n1 is the number of horizontal pixels of the subblock (dotted line block 403).

That is, the divided image data 404 is an integral multiple of the subblock, and the divided image data is equal to or less than the capacity of the delay line. In order to minimize the processing overhead and process efficiently, it is necessary to obtain the maximum b1 that satisfies the above conditions. Note that m1 is a fixed value because it is the number of pixels that can be stored in the delay line of the image processing core unit 200, and n1 is also a fixed value because it is a unit of subblock pixels.

In S501, the CPU 109 determines the size of the divided image data 401 and the size of the divided image data 406. Specifically, the horizontal size a1 of the divided image data 401 and the horizontal size c1 of the divided image data 406 are calculated so that the following mathematical formulas (3), (4), and (5) hold.
k1 = a1 + b1 + c1 ... (3)
m1 ≧ a1… (4)
m1 ≧ c1 ... (5)
Here, k1 is the number of horizontal pixels of the image data 400, a1 is the number of horizontal pixels of the divided image data 401, and c1 is the number of horizontal pixels of the divided image data 406. The number of horizontal pixels a1 of the divided image data 401 and the number of horizontal pixels c1 of the divided image data 406 may be a1 = c1, a1> c1 or a1 <c1. It can be set to the corresponding value.

In S502, the CPU 109 determines whether or not the number of horizontal pixels a1 of the divided image data 401 is divisible by the number of horizontal pixels n1 of the subblock (or is it an integral multiple of n1). When the CPU 109 determines that it is divisible, it advances the process to S504, and when it determines that it is not divisible, it advances the process to S503. When it is divisible, the number of horizontal pixels of the shaded portion 402 in FIG. 4, which represents the pixels to be added to the end portion of the divided image data 401, is 0. That is, the shaded portion 402 is unnecessary, in other words, there is no pixel addition processing by the pixel addition portion 201.

In S503, the CPU 109 calculates the number of pixels in the shaded area 402, that is, the number of pixels d1 added by the pixel addition unit 201 according to the following equation (6), and proceeds to the process in S504.
d1 = (a1 / n1 + 1) * n1-a1 ... (6)
In addition, "/" in equation (6) is a division sign which truncates after the decimal point. Here, it can be said that d1 to be calculated is a shortage that is less than an integral multiple of the number of pixels of the delay line.

Further, in S503, the CPU 109 performs a determination process (additional pixel determination process) of whether the pixel to be added is a copy pixel or a random pixel. This additional pixel determination process will be described later with reference to the flow chart of FIG.

In S504, the CPU 109 determines whether or not the number of horizontal pixels c1 of the divided image data 406 is divisible by the number of horizontal pixels n1 of the subblock. When the CPU 109 determines that it is divisible, the CPU 109 ends this division size determination process. If the CPU 109 determines that it is not divisible, the CPU 109 proceeds to S505. If it is divisible, there is no pixel addition process by the pixel addition unit 201.

In S505, the CPU 109 calculates the number of pixels in the shaded area 407 in FIG. 4, that is, the number of pixels added by the pixel addition unit 201, in the same manner as in step 503.

Next, the additional pixel determination process of the CPU 109 will be described with reference to the flow chart of FIG.

In S600, the CPU 109 determines whether the pixel added to the edge of the image is in the copy mode using the copy pixel. When the CPU 109 determines that the copy mode is set, the process proceeds to S601. When the CPU 109 determines that the copy mode is not set, the CPU 109 proceeds to S602. The copy mode indicates that the low image quality recording mode or the high compression rate mode is selected by the user from the operation unit (not shown) in the mode of the digital camera of the present embodiment. Further, there is a case where the copy mode is explicitly selected by the user in the mode of the digital camera of the present embodiment, or a case where pixels are added by the pixel addition unit 201 before processing by the image processing core unit 200. The advantage of using copy pixels before processing by the image processing core unit 200 is that pixels with a filter tap length required for filtering by the image processing core unit 200 are copied by image processing inside the image processing core unit 200. There is no need for it. Further, by holding and placing the pixels obtained by filtering the copy pixels in the image processing core unit 200 so as to be a unit of sub-block pixels, it is not necessary to add pixels after the processing of the image processing core unit 200. Furthermore, if the edges of the image are simply copy pixels, the compression efficiency of the subblocks will increase by including many copy pixels, and the compression ratio of nearby subblocks will deviate, which may affect the image quality. However, by using the filtered pixels, the compression ratio of the nearby sub-blocks does not deviate, and the difference in image quality for each sub-block can be suppressed.

In S601, the CPU 109 sets the additional pixel as a copy pixel and finishes the additional pixel determination process.

In S602, the CPU 109 is a pixel value of a pixel existing in a region (or range) from the left end side (right end side) of the divided left end image (or the divided right end image) to the inside of the image to a preset distance. Is analyzed, and the degree of low frequency component is obtained based on the analysis result. Then, in S603, the CPU 109 determines whether or not a large amount of low frequency components are contained.

For example, the CPU 109 obtains the absolute value of the difference between adjacent pixels of the pixels existing in the range, and if the absolute value of the difference is equal to or less than the threshold value, it determines that it is a low frequency component. It should be noted that this is an example, and the judgment may be made according to other logics.

If the CPU 109 determines in S603 that it contains a large amount of low frequency components, the process proceeds to S601. On the other hand, when the CPU 109 determines that the low frequency component is small (the high frequency component is large), the process proceeds to S604.

In S604, the CPU 109 determines that additional pixels are randomly generated, and ends the additional pixel determination process. It should be noted that all the pixels to be added may be random pixels, or the same random pixels may be used for the pixels to be added.

If it is a low frequency component in the additional pixel determination process, it is made into a copy pixel, and if it is not a low frequency component, it is made into a random pixel because it is compressed in subblock units, so it is better to compress it at a compression rate close to that of nearby subblocks. This is because it is possible to suppress the difference in image quality due to the difference in compression rate.

After that, the compression unit 101 inputs the divided image data from the image processing unit 100, and performs the compression coding process in sub-block units. The generated encoded data will be stored in the memory 106 via the data transfer unit 102.

As described above, according to the first embodiment, the size of the divided image is an integral multiple of the compression subblock as compared with the case where the division center image size is determined to be arbitrary. It is not necessary to adjust to the unit of, and the coding process can be performed smoothly, and the deterioration of the coding process performance and the compression efficiency can be suppressed. Further, as the number of divisions increases, the effect of this embodiment becomes more remarkable.

Although the above has been specifically described based on the first embodiment, it is needless to say that the description is not limited to the above embodiment and various changes can be made without departing from the gist thereof.

[Second Embodiment]
Next, the second embodiment will be described. Here, only the parts different from those of the first embodiment described above will be described, and the same parts will be designated by the same reference numerals and detailed description thereof will be omitted.

FIG. 7 is an example in which the image data 700 is divided into two divided image data 701 and 703 with the dotted line as the boundary. The dotted line blocks 702 and 705, and the dotted line block to which the drawing number is not added indicate a subblock to be compressed by the compression unit 101. The shaded area 704 indicates a pixel area to be added by the pixel addition unit 201.

Next, a method of determining the division size of the image data shown in FIG. 7 by the CPU 109 will be described with reference to the flowchart of FIG.

In S800, the CPU 109 determines the size of the split left end image data 701 in the horizontal direction so that the dotted line block 702 fits well. That is, if the unit of the sub-block pixel is 32 pixels, the horizontal size of the divided image data 701 is made an integral multiple of 32 pixels.

Next, the CPU 109 calculates the size b2 of the divided right end image data 703 in S801 so that the following mathematical formulas (7) and (8) hold.
b2% n2 = 0 ... (7)
m2 ≧ b2… (8)
Here, b2 is the number of horizontal pixels of the split left end image data 701, m2 is the number of pixels that can be stored in the delay line, and n2 is the number of horizontal pixels of the subblock (the number of horizontal pixels of the dotted line block 702). Note that m2 is a fixed value because it is the number of pixels that can be stored in the delay line of the image processing core unit 200, and n2 is also a fixed value because it is a unit of subblock pixels. It is efficient if b2 is set to the maximum value that satisfies the mathematical formulas (7) and (8) according to the system of the digital camera of the present embodiment.

In S801, the CPU 109 determines the size of the divided image data 703 and proceeds to process in S802. Specifically, the size c2 of the divided image data 703 is calculated so that the following mathematical formulas (9) and (10) hold.
k2 = b2 + c2 ... (9)
m2 ≧ c2… (10)
Here, k2 is the number of horizontal pixels of the image data 700, and c2 is the number of horizontal pixels of the divided right end image data 703.

In S802, the CPU 109 determines whether or not the number of horizontal pixels c2 of the divided right end image data 703 is divisible by the size n2 of the subblock. When the CPU 109 determines that it is divisible, the CPU 109 ends the division size determination process. If the CPU 109 determines that it is not divisible, the CPU 109 proceeds to S803. When c2 is divisible by n2, the number of horizontal pixels in the shaded area 704 becomes 0, and additional pixel processing is not required.

In S803, the CPU 109 calculates the number of pixels represented by the shaded area 704, that is, the number of pixels d2 added by the pixel addition unit 201 from the following equation (11).
d2 = (c2 / n2 + 1) * n2-c2 ... (11)
Further, whether the pixel to be added is a copy pixel or a random pixel is determined by the additional pixel determination process shown in FIG.

As described above, according to the second embodiment, in addition to the effects of the first embodiment, the divided image for determining the number of pixels to be added is one of the divided right end images, and the determination is made. Such processing and pixel addition processing are simplified, and deterioration of processing performance and compression efficiency can be further suppressed.

Although specifically described based on the second embodiment, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the gist thereof.

[Third Embodiment]
Next, a third embodiment will be described. Here, only the parts different from those of the first embodiment will be described, and the same parts will be designated by the same reference numerals and detailed description thereof will be omitted.

FIG. 9 is a block diagram showing an example of the configuration of the digital camera according to the third embodiment, and is a configuration in which an extension unit (decoding unit) 900 is added to FIG.

The decompression unit 900 reads the image data compressed by the compression unit 101 stored in the memory 106 via the data transfer unit 102. Then, the decompression unit 900 decompresses (decodes) the read compressed image data in units of subblock pixels compressed by the compression unit 101, and passes the decompression result image data to the image processing unit 100.

FIG. 10 is an internal configuration diagram of the image processing unit 100 according to the third embodiment. The difference from FIG. 2 is that the pixel deletion unit 1000 is added. The pixel deletion unit 1000 deletes pixels at an arbitrary location in the input image data, and passes the deleted image data to the image processing core unit 200. The pixel deletion unit 1000 may be provided in the extension unit 900.

FIG. 11 is a diagram in which the image data 1100 is divided into three divided image data 1101, 1105, and 1107 with the dotted line in the figure as a boundary. The dotted line blocks 1104, 1106, 1110 and the dotted line blocks to which the drawing numbers are not added indicate the horizontal size (compression coding unit) of the subblocks to be compressed by the compression unit 101. The shaded portions 1102, 1108 and the cross shaded portions 1103, 1109 indicate the pixels to be added by the pixel addition unit 201. Although FIG. 11 shows three divisions, the number of divisions may be three or more.

The division image size determination in the third embodiment is determined according to the flow of FIG. 5, but the pixels to be added by the pixel addition unit 201 are determined by the additional pixel determination process shown in FIG.

The program according to FIG. 12 is stored in the non-volatile memory 108 and is also executed by the CPU 109. Hereinafter, the additional pixel determination process by the CPU 109 will be described with reference to the figure.

In S1200, the CPU 109 determines as copy pixels the pixels required for filter processing when the pixels to be added are expanded by the expansion unit 900 and image processing is performed. The shaded areas 1102 and 1108 in FIG. 11 indicate the copy pixel region. That is, the left end and the right end for filtering the pixels for matching with the unit of the sub-block pixel required at the time of compression in the next image processing core unit 200 are set as copy pixels in advance. As a result, it is not necessary to add copy pixels in the next image processing as compared with the case of using random pixels, so that the processing performance can be improved.

In S1201, the CPU 109 determines as random pixels other than the number of copy pixels determined in step 1200 among the pixels to be added, and ends the additional pixel determination process. The cross-hatched portions 1103 and 1109 in FIG. 11 indicate random pixels, which are regions to be deleted by the pixel deletion unit 1000 after being expanded by the expansion unit 900. That is, since the cross shaded portions 1103 and 1109 are pixels that are not processed by the image processing core unit 200, it is not necessary to process unnecessary pixels, so that deterioration of processing performance can be suppressed.

As described above, according to the third embodiment, since the divided image size is the unit of the subblock pixel of compression, it is not necessary to match the unit of the subblock pixel at the time of compression, so that extra pixels are used. Processing is not required, and deterioration of processing performance and compression efficiency can be suppressed. Further, as the number of divisions increases, the effect of this embodiment becomes more remarkable. Furthermore, when filtering is performed in the next image processing, the processing performance is improved by making the pixels used in the filtering processing copy pixels at the time of compression, and unnecessary pixels are cut by the filtering after decompression. It is also possible to suppress a decrease in processing performance.

Although the present invention has been specifically described above based on the third embodiment, it is needless to say that the present invention is not limited to the above-described embodiment and various modifications can be made without departing from the gist thereof.

[Fourth Embodiment]
Next, a fourth embodiment will be described. FIG. 13 is a block diagram showing an example of the configuration of the digital camera according to the fourth embodiment. As shown in the figure, this apparatus includes an image processing unit 1300, RDDMAC1301, WRDMAC1302, compression unit 1303, decompression unit 1304, data bus 1305, control bus 1306, memory control unit 1307, memory 1308, non-volatile memory 1309, and non-volatile memory. It has 1310 and CPU 1311. Further, although not shown, an image sensor such as a CCD or CMOS sensor that converts a received subject image into an electric signal to create image data, an AD converter that converts an analog signal from the image sensor into a digital signal, and an image. It also has a compression / decompression unit that compresses or decompresses the data in, for example, PEG or MPEG format, and an operation unit that is operated by the user.

The image processing unit 1300 is connected to the RDDMAC 1301, the extension unit 1304, the WRDMAC 1302, and the compression unit 1303. The image processing unit 1300 inputs image data from the RDDMAC 1301, performs image processing, and outputs the processing result to the WRDMAC 1302 or the compression unit 1303.

As shown in FIG. 14, the image processing unit 1300 includes an image processing core unit 1400 and a pixel compression unit determination unit 1401. The image processing core unit 1400 executes a plurality of processes such as pixel correction, black level correction, shading correction, scratch correction, magnification chromatic aberration correction gamma correction, brightness / color generation processing, geometric deformation, noise reduction, scaling, and the like. This is a block that performs appropriate image processing on image data. The pixel compression unit determination unit 1401 determines the number of pixels of the subblock when the compression unit 1303 horizontally divides the image data into subblock units and compresses the image data. Detailed explanation will be performed later with reference to FIGS. 15 and 16. The pixel compression unit determination unit 1401 may be included in the compression unit 1303.

The RDDMAC 1301 is composed of a plurality of DMA (Direct Memory Access) controllers that read and transfer data, and includes an RDDMAC core unit 1700 and a pixel extension unit determination unit 1701 as shown in FIG. The image data temporarily stored in the memory 1308 is output from the memory 1308 to the RDDMAC 1301 via the data bus 1305 by the RDDMAC core unit 1700, and is transferred to the image processing unit 1300 or the extension unit 1304. The pixel expansion unit determination unit 1701 calculates the number of pixels in the subblock of the image data to be expanded. Detailed explanation will be performed later using the flow chart of FIG. The pixel extension unit determination unit 1701 may be included in the extension unit 1304.

The WRDMAC 1302 is composed of a plurality of DMA controllers that write and transfer data to and from the memory 1308. The image data is output to the data bus 1305 by the WRDMAC 1302 and temporarily stored in the memory 1308 via the memory control unit 1307.

The compression unit 1303 is a block that compresses the image data processed by the image processing unit 1300 in the horizontal direction in units of the number of pixels of a sub-block such as 32 pixels by a differential pulse code modulation (DPCM) method. The number of pixels of the sub-block is determined by the pixel compression unit determination unit 1401. The compression method may be another method.

The decompression unit 1304 reads the image data compressed by the compression unit 1303 stored in the memory 1308 via the RDDMAC 1301. Then, the decompression unit 1304 decompresses the read compressed image data in units of the number of pixels of the compressed subblock, and passes the decompression result image data to the image processing unit 1300. The number of pixels of the sub-block is determined by the pixel extension unit determination unit 1701.

The data bus 1305 is a bus for transferring image data, and the control bus 1306 is a control bus that accesses the image processing unit 1300, RDDMAC 1301, WRDMAC 1302, compression unit 1303, and decompression unit 1304 from the CPU 1311.

The memory control unit 1307 writes data to the memory 1308 and reads data from the memory 1308 in response to an instruction from the CPU 1311 or the RDDMAC 1301 or WRDMAC 1302. The memory 1308 is a storage device having a storage capacity sufficient for storing a predetermined number of still images, moving images for a predetermined time, data such as voice, constants for operation of the CPU 1311, programs, etc., and is composed of a DRAM or the like. Will be done.

The non-volatile memory control unit 1309 writes data to the non-volatile memory 1310 and reads data from the non-volatile memory 1310 in response to an instruction from the CPU 1311. The non-volatile memory 1310 is a memory that can be electrically erased and recorded, and for example, EEPROM or the like is used. The non-volatile memory 1310 stores constants, programs, and the like for the operation of the CPU 1311.

The CPU 1311 is composed of a microcomputer or the like that controls the operation of the digital camera, gives various instructions to each functional block constituting the digital camera, and executes various control processes. The CPU 1311 controls an image processing unit 1300, an RDDMAC 1301, a WRDMAC 1302, a compression unit 1303, an expansion unit 1304, a memory control unit 1307, and a non-volatile memory control unit 1309 connected via a control bus 1306. The CPU 109 realizes each process of the present embodiment by executing the program recorded in the non-volatile memory 1310 described above. Further, the CPU 1311 determines the division size at the time of the image data division processing. An example of image division by the CPU 1311 is shown in FIG.

FIG. 15 shows an example in which the image data 1500 is divided into three divided image data 1501, 1504, and 1507 with the dotted line in the figure as a boundary. The dotted line blocks 1502, 1505, and 1508 indicate sub-blocks to be compressed by the compression unit 1303, and the number of pixels in the horizontal direction thereof is the same.

Further, the number of pixels in the horizontal direction of the dotted line block 1503 located at the right end of the divided image data 1501 is obtained by subtracting the total number of pixels of all subblocks located on the left side of the dotted line block 1503 from the number of horizontal pixels of the divided image data 1501. Value.

When the number of pixels in the horizontal direction of the divided image data 1501 is N and the number of pixels in the horizontal direction of the coding processing unit is S, the relationship of the following equation is obtained.
N = S × i + r
The dotted line block 1503 represents a portion corresponding to the fraction “r” in the above equation. In addition, r is a value smaller than S, and is also a value within the range of the capacity of the line memory possessed by the compression unit 1303.

The method of calculating the number of pixels of the sub-block will be performed later using the flow chart of the compression unit determination process of FIG.

The concept of the number of pixels of the sub-blocks shown in the dotted line blocks 1505 and 1508 and the dotted line blocks to which the subsequent drawing numbers are not added is the same as that of the dotted line block 1502. The concept of the number of pixels of the sub-blocks shown in the dotted line blocks 1506 and 1509 located at the right end of the divided image data is the same as that of the dotted line block 1503.

A flow of compression unit determination processing for calculating the number of pixels of the compression unit, that is, the number of pixels of the subblock, in which the divided image data 1501 is compressed by the compression unit 1303 by the CPU 1311 will be described with reference to FIG. Although described with reference to the divided image data 1501, the divided image data 1504 and 1507 are also applied in the same manner.

In S1600, the CPU 1311 determines whether to process the first pixel of the dotted line block 1502 of the divided image data 1501 and the dotted line block (subblock) to which the subsequent drawing number is not added. When the CPU 1311 determines that the first pixel is to be processed, the CPU 1311 proceeds to S1601 for processing. When the CPU 1311 determines that the pixels other than the first pixel are to be processed, the compression unit determination process is terminated.

In S1601, the CPU 1311 acquires the processing pixel position in the horizontal direction and proceeds to process in S1602. The CPU 1311 knows in advance the number of horizontal pixels of the divided image data 1501. Therefore, in S1602, the CPU 1311 calculates the number of remaining pixels (also the number of uncoded pixels) in the horizontal direction of the divided image data 1501 from the pixel positions acquired in S1601, and proceeds to S1603.

In S1603, the CPU 1311 compares the remaining number of pixels calculated in S1602 with the threshold value A. When the CPU 1311 determines that the calculated number of pixels is equal to or greater than the threshold value, the processing proceeds to S1604, and when it determines that the calculated number of pixels does not reach the threshold value, the processing proceeds to S1605.

In S1605, the CPU 1311 sets the number of pixels of the compression unit as the threshold value A, notifies the compression unit of the number of pixels, and ends the compression unit determination process.

In S1605, the CPU 1311 sets the number of pixels of the compression unit to the remaining number of pixels calculated in S1601, notifies the compression unit 1303 of the number of pixels, and ends the compression unit determination process.

Next, according to FIG. 18, for the image data obtained by compressing the divided image data 1501, the expansion unit determination process for calculating the number of pixels of the expansion unit to be expanded by the expansion unit 1304, that is, the number of pixels of the subblock will be described.

Although the image data obtained by compressing the divided image data 1501 will be described, the same applies to the image data obtained by compressing the divided image data 1504 and 1507. Further, the decompression unit determination process is a process on the premise that the data size of the compressed subblock compressed by the compression unit 1303 is compressed so as to have a fixed length in terms of the number of pixels in the subblock. For example, if the number of pixels before compression is 1 pixel 14 bits, the number of pixels in the subblock is 32 pixels and uncompressed, it is 448 bits, and if it is compressed, it is 320 bits. If the number of pixels is 8 pixels and it is uncompressed, it is 112 bits, compressed. If so, it is compressed to 80 bits. That is, by setting the data size of the compressed subblock to a fixed length according to the number of pixels in the subblock, the horizontal pixel position can be grasped from the amount of data.

In S1800, the CPU 1311 determines whether to process the first pixel of the dotted line block (subblock) in the compressed image data of the divided image data 1501. When the CPU 1311 determines that the first pixel of the dotted line block is the processing target, the processing proceeds to S1801. Further, when the CPU 1311 determines that the pixel other than the first pixel is the processing target, the CPU 1311 ends the expansion unit determination process.

In S1801, the CPU 1311 acquires the processing position in the horizontal direction and advances the processing to S1802. In S1802, the CPU 1311 calculates the number of remaining pixels in the horizontal direction from the processing position in the horizontal direction, and proceeds to the processing in S1803.

In S1803, the CPU 1311 compares the remaining number of pixels calculated in S1802 with the threshold value A. When the CPU 1311 determines that the calculated number of pixels is larger than the threshold value A, the CPU 1311 advances the process to S1804. Further, when the CPU 1311 determines that the calculated number of pixels is equal to or less than the threshold value A, the CPU 1311 proceeds to S1805.

In S1804, the CPU 1311 sets the number of pixels of the expansion unit as the threshold value A, notifies the expansion unit 1304 of the number of pixels, and ends the expansion unit determination process.

In S1805, the CPU 1311 sets the number of pixels in the expansion unit to the remaining number of pixels calculated in S1801, notifies the expansion unit 1304 of the number of pixels, and ends the expansion unit determination process.

As described above, according to the fourth embodiment, when the divided image is divided into sub-blocks, the number of pixels of the sub-block at the right end of the divided image is made different, thereby suppressing a decrease in processing performance and compression efficiency. it can. Further, when the boundary data of the divided image is compressed, it is compressed without using copy pixels or random pixels, so that the code amount of the image boundary and the code amount other than the image boundary do not become large, and the divided boundary and the divided image are divided. It is also possible to suppress the influence of image quality differences other than the boundary.

Although the present invention has been specifically described above based on the fourth embodiment, it goes without saying that the present invention is not limited to the above-described embodiment and various modifications can be made without departing from the gist thereof.

[Fifth Embodiment]
Next, a fifth embodiment will be described. In the fifth embodiment, only the parts different from the fourth embodiment described above will be described, and the same parts will be designated by the same reference numerals and detailed description thereof will be omitted.

FIG. 19 is a configuration diagram of the image processing unit 1300 according to the fifth embodiment. As shown in the figure, the image processing core unit 1400, the pixel compression unit determination unit 1401, and the line buffer 1900 are included and connected to the compression unit 1303. The line buffer 1900 is composed of SRAM or the like, and has a double buffer configuration in which pixel data of two lines are stored in the horizontal direction. The pixel compression unit determination unit 1401 calculates the number of pixels of the subblock of the number of pixels of the compression unit to be compressed by the compression unit 1303 and the process of storing one line of image data in one of the double buffers, and the compression unit The process of outputting to 1303 is performed. Details will be described with reference to FIGS. 20 and 21.

The pixel compression unit determination unit 1401 and the line buffer 1900 may be included in the compression unit 1303. The compression unit 1303 performs data compression processing according to the number of pixels of the subblock notified from the pixel compression unit determination unit 1401, and compresses the data so that the data size after subblock compression has a fixed length. That is, even if the number of pixels in the subblock is different, the data size after compression is fixed. The data size after subblock compression can be set from the CPU 1311. At the time of compression, the number of pixels in the subblock is added to the header.

FIG. 20 shows an example in which the image data 2000 is divided into three divided image data 2001, 2004, and 2007 at the dotted line boundary. The illustrated dotted line blocks 2002, 2003, 2005, 2006, 2008, 2009 and the dotted line blocks to which the drawing numbers are not added indicate subblocks to be compressed by the compression unit 1303. The number of pixels of the dotted line block 2002, the dotted line block to which the drawing number following it is not added, and the sub-block shown in the dotted line block 2003 located at the right end are different from each other. However, depending on the pixel data in the divided image data 2001, the number of pixels in the subblock may be the same. That is, it means that the number of pixels of the sub-block is variable depending on the position of the divided image data 2001. The method of calculating the number of pixels of the sub-block in which the number of pixels is variable will be performed later using the flow chart of the compression unit determination process of FIG. 21. The concept of the number of pixels of the sub-blocks shown in the dotted line blocks 2005, 2006, 2008 and 2009 is the same as that of the dotted line block 2002.

A process of calculating the number of pixels of the compression processing unit of the compression unit 1303, that is, the number of pixels of the sub-block by the CPU 1311 will be described with reference to FIG. 21. Although the divided image data 2001 will be described as an example, the divided image data 2004 and 2007 are also applied in the same manner.

In S2100, the CPU 1311 takes a difference αn (n is an integer of 1 or more and has a value up to the number of subblocks N-1) between adjacent pixels, and holds an input pixel in one side A (B) of the line buffer 1900. Then, the process proceeds to S2101. Further, when one side A of the line buffer 1900 is used in odd-numbered lines, the other side B of the line buffer 1900 is used in even-numbered lines.

In S2101, the CPU 1311 compares the difference αn with the threshold value B. When the CPU 1311 determines that the difference αn is larger than the threshold value B, the CPU 1311 proceeds to S2102. Further, when the CPU 1311 determines that the difference αn is equal to or less than the threshold value B, the CPU 1311 proceeds to S2107 for processing.

In S2102, the CPU 1311 holds the difference αn and the number of pixels Hn of the subblocks (n is an integer of 1 or more and becomes a value up to the number of subblocks N), counts up (counts) the number of subblocks N, and S2013. Proceed to processing.

In S2103, the CPU 1311 determines whether or not the pixels for one line of the divided image data 2001 have been processed. When the CPU 1311 determines that the image processing for one line has been completed, the CPU 1311 advances the processing to S2104. If the CPU 1311 determines that the processing for one line has not been completed, the CPU 1311 proceeds to S2107.

In S2014, the CPU 1311 compares the number of subblocks N with the threshold value C, and determines whether or not the number of subblocks N is larger than the threshold value C. When the CPU 1311 determines that the number of subblocks N is larger than the threshold value C, the CPU 1311 proceeds to S2105. Further, when the CPU 1311 determines that the number of subblocks N is equal to or less than the threshold value C, the CPU 1311 proceeds to S2106.

In S2105, the CPU 1311 changes the number of subblocks N by combining the subblocks in ascending order of the value of the difference αn so that the number of subblocks N becomes the threshold value C. Then, the CPU 1311 advances the process to S2106.

In S2106, the CPU 1311 initializes the number of subblocks N. Then, the CPU 1311 advances the process to S2107.

In S2107, the CPU 1311 determines whether or not there is data to be output to one side B (A) of the line buffer 1900, that is, one side of the line buffer 1900 in which data is not stored in S2100. When the CPU 1311 determines that there is data to be output, the CPU 1311 proceeds to S2108 for processing. If it is determined that there is no data to be output, the compression unit determination process is terminated.

In S2108, the CPU 1311 outputs the pixels stored in one side B (A) of the line buffer 1900 to the compression unit, and when it becomes the head of the subblock, notifies the number of pixels Hn of the subblock and determines the compression unit. End the process.

Next, with reference to FIG. 22, the flow of the expansion unit determination processing for calculating the number of pixels of the expansion unit to be expanded by the expansion unit 1304, that is, the number of pixels of the subblock, will be described for the compressed image data of the divided image data 2001. .. Although the compressed image data of the divided image data 2001 will be described, the same applies to the compressed image data of the divided image data 2004 and 2007.

In S2200, the CPU 1311 determines whether or not it is the beginning of the subblock compressed image data. When the CPU 1311 determines that it is the first pixel, the process proceeds to S2201. When the CPU 1311 determines that the pixel is not the first pixel, the expansion unit determination process is terminated. The determination as to whether or not it is the beginning of the subblock compressed image data is determined by whether or not it is the beginning data of the divided image data 2001. Alternatively, it is determined whether the processed data size is the cumulative addition of the processed data sizes and the next data that is equal to the data size after subblock compression set in advance by the CPU 1311. If it is the first data, it is determined. Initialize the processed data size. That is, as described in the divided image data 2001 before compression, it is determined whether or not the dotted line block 2002, the dotted line block to which the drawing number following the dotted line block 2002 is not added, and the head of the subblock shown in the dotted line block 2003.

In step 2201, the CPU 1311 acquires the number of pixels in the subblock from the header of the subblock compressed image data, notifies the expansion unit 1304 of the number of pixels, and ends the expansion unit determination processing unit.

As described above, according to the fifth embodiment, the processing is performed by changing the number of pixels in the subblock to be compressed so that the number of subblocks in one line of the divided image data is equal to or less than the threshold value. It is possible to suppress a decrease in performance and compression efficiency. Further, when compressing the boundary data of the divided image, it is compressed without using copy pixels or random pixels. Therefore, since the code amount of the image boundary and the code amount other than the image boundary do not become large, the influence of the image quality difference between the divided boundary and the image quality other than the divided boundary can be suppressed. Furthermore, by determining the number of pixels in the subblock from the difference between adjacent pixels, the amount of code assigned to each pixel is made uniform, so that image deterioration due to compression can be suppressed.

The present invention has been specifically described above based on the fifth embodiment, but it goes without saying that the present invention is not limited to the above-described embodiment and various modifications can be made without departing from the gist thereof.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 ... image processing unit, 101 ... compression unit, 102 ... data transfer unit, 103 ... data bus, 104 ... control bus, 105 ... memory control unit, 106 ... memory, 107 ... non-volatile memory control unit, 108 ... non-volatile memory , 109 ... CPU

Claims (5)

An image processing device that encodes image data.
A dividing means for dividing the image data to be encoded into a plurality of divided image data, and
A coding means that encodes a predetermined number of consecutive pixel data as a unit, and
It has a control means for controlling the coding means and for encoding each of the divided image data obtained by the dividing means.
The control means
A determination means for determining whether or not the size of the divided image data to be encoded by the coding means is an integral multiple of the unit.
It said determining means, when the size of the divided image data of the encoding target is determined not to be an integral multiple of the unit, possess an additional means for adding the missing pixel for an integer multiple of said unit,
The additional means
It has an analysis means for performing a predetermined analysis on a region from the boundary of the divided image data to a predetermined distance and determining whether or not the region represents a low frequency component.
When the analysis result obtained that the region represents a low frequency component by the analysis means, a copy of the pixel located inside the boundary is generated as a pixel to be added.
An image processing apparatus characterized in that when an analysis result that the region does not represent a low frequency component is obtained by the analysis means, pixels generated by a predetermined random number are generated as additional pixels. The image according to claim 1, wherein the dividing means determines a divided image having a size that is an integral multiple of the unit in a direction from the center position of the image data to be encoded toward both ends. Processing equipment. The claim is characterized in that the dividing means determines a divided image having a size that is an integral multiple of the unit in a direction from one end to the other end of the image data to be encoded. The image processing apparatus according to 1. A control method for an image processing device that encodes image data.
A division step of dividing the image data to be encoded into a plurality of divided image data, and
A coding process that encodes a predetermined number of consecutive pixel data as a unit,
It has a control step of controlling the coding step and encoding each of the divided image data obtained in the dividing step.
The control step is
A determination step of determining whether or not the size of the divided image data to be encoded by the coding step is an integral multiple of the unit, and
In the determination step, when the size of the divided image data of the encoding target is determined not to be an integral multiple of the unit, possess an additional step of adding a missing pixel to an integer multiple of said unit,
The additional step is
It has an analysis step of performing a predetermined analysis on a region from the boundary of the divided image data to a predetermined distance and determining whether or not the region represents a low frequency component.
When the analysis result that the region represents a low frequency component is obtained by the analysis step, a copy of the pixel located inside the boundary is generated as an additional pixel.
When an analysis result that the region does not represent a low frequency component is obtained by the analysis step, a pixel generated by a predetermined random number is generated as an additional pixel.
A control method for an image processing device. A program for causing the computer to execute each step of the method according to claim 4 , which is read and executed by the computer.

Images