JP6884530B2 - Image processing device and first adjustment circuit - Google Patents

Description

The present invention relates to an image processing technique.

Patent Document 1 discloses a circuit for processing image data.

Japanese Unexamined Patent Publication No. 2006-293748

Due to the alignment constraint, not only necessary image data but also unnecessary data may be input to the circuit for processing image data, which is located after the circuit having the alignment constraint.

Therefore, the present invention has been made in view of the above points, and it is possible to prevent unnecessary data from being input to the circuit that processes the image data without changing the circuit having the alignment constraint. The purpose is to provide various technologies.

One aspect of the image processing apparatus includes a first processing circuit having an A-byte alignment constraint, a second processing circuit that performs first processing on input data, and a first adjustment circuit. With respect to the image data indicating the image, the processing circuit is input with the first data having a data length that is a multiple of the A bytes, including the data of the one line, for each line, and the first processing circuit is described. The second data, which is the first data, is output for each line, or the first data is subjected to the second processing for each line, and the first data after the second processing. The second data is output, and the first adjustment circuit discards the first unnecessary data included in the second data for each line, and the second data in which the first unnecessary data is discarded is used. A certain third data is input to the second processing circuit.

Further, in one aspect of the image processing apparatus, a determination circuit for determining the first data and the first unnecessary data is further provided based on the alignment constraint of the A byte and the information regarding the image.

Further, in one aspect of the image processing apparatus, the second processing circuit has a B-byte alignment constraint, and the first adjustment circuit has a data length of the third data that is a multiple of the B-byte. , The first unnecessary data included in the second data is discarded.

Further, in one aspect of the image processing apparatus, the second processing circuit performs the first processing on the third data for each line, and is the third data after the first processing. A third processing circuit that outputs data and performs a third processing on the input data, the fourth data for each line, and the second unnecessary data included in the fourth data are discarded. A second adjustment circuit to be input to the third processing circuit is further provided.

Further, in one aspect of the image processing apparatus, the first data, the first unnecessary data, and the second unnecessary data are determined based on the A-byte alignment constraint, the B-byte alignment constraint, and information on the image. A decision circuit is further provided.

Further, one aspect of the adjustment circuit is the first adjustment circuit included in the above-mentioned image processing device.

Further, one aspect of the adjustment circuit is a second adjustment circuit included in the above-mentioned image processing device.

It is possible to prevent unnecessary data from being input to the circuit that processes the image data without changing the circuit having the alignment constraint.

It is a figure which shows an example of the structure of an image processing apparatus. It is a figure which shows an example of how the memory stores image data. It is a figure which shows an example of an image. It is a figure which shows an example of the structure of an image processing circuit. It is a figure which shows an example of a necessary image. It is a figure which shows an example of the line correspondence read data and necessary line data. It is a figure which shows an example of the relationship between a necessary image and an image read from a memory. It is a figure which shows an example of the line correspondence read data and necessary line data. It is a figure which shows an example of the line correspondence read data and necessary line data. It is a figure for demonstrating an example of operation of a DMA controller. It is a figure for demonstrating an example of the operation of an image processing circuit. It is a figure which shows the state of reading out the necessary image data in a raster order. It is a figure which shows the state of reading out the necessary image data in the reverse raster order. It is a figure for demonstrating an example of the operation of an image processing circuit.

<Overall configuration of image processing device>
FIG. 1 is a diagram showing an example of the configuration of the image processing device 1. The image processing device 1 is mounted on, for example, a digital still camera.

As shown in FIG. 1, the image processing device 1 includes a CPU (Central Processing Unit) 2, a DMA (Direct Memory Access) controller 3, a memory 4, an image sensor 5, and an image processing circuit 6. It can be said that the image processing device 1 has a kind of circuit configuration.

The CPU 2 is a kind of processor and also a kind of circuit configuration. The CPU 2 executes a program stored in a memory such as a ROM (Read Only Memory) (not shown). When the CPU 2 executes a program, various functions are realized in the CPU 2. The CPU 2 can control the DMA controller 3.

Instead of the CPU 2, a hardware circuit including, for example, a logic circuit, which does not require a program (software) to realize the function, may be provided. Further, a part of the functions of the CPU 2 may be realized by a hardware circuit that does not require a program to realize the functions.

The DMA controller 3 has a kind of circuit configuration, and can read the data stored in the memory 4 from the memory 4 under the control of the CPU 2. The memory 4 is, for example, a RAM (Random Access Memory). In the memory 4, for example, the storage area of one address stores one byte of data. The data bus width between the DMA controller 3 and the memory 4 is N bytes. N is an integer and N ≧ 2. In this example, for example, N = 16.

The DMA controller 3 has an N-byte alignment constraint, in this example, a 16-byte alignment constraint. It can be said that the DMA controller 3 is a processing circuit having an N-byte alignment constraint. The DMA controller 3 reads data from the memory 4 in 16-byte units (128-bit units). By designating an address that is a multiple of 16 to the memory 4, the DMA controller 3 simultaneously stores 16 bytes of data stored in the storage area of 16 consecutive addresses starting from the designated address. Can be read. In addition, N may be other than 16.

The image sensor 5 converts light incident through an optical system (not shown) including a lens or the like into an electric signal to generate image data 100 indicating an image in which the subject is captured. The image data 100 generated by the image sensor 5 is stored in the memory 4.

The image processing circuit 6 performs image processing on the image data read from the memory 4 by the DMA controller 3. The DMA controller 3 can read a part of the image data 100 or all of the image data 100 from the memory 4.

The CPU 2 outputs the specific information 300 for specifying the storage area in which the data to be read by the DMA controller 3 is stored in the memory 4 to the DMA controller 3. The DMA controller 3 reads the data to be read from the memory 4 based on the specific information 300 and inputs the data to the image processing circuit 6.

The image data processed by the image processing circuit 6 is input to the output device. The output device is, for example, a display included in the digital still camera on which the image processing device 1 is mounted, or a display outside the digital still camera. The display as an output device can display the image indicated by the image data based on the input image data.

FIG. 2 is a diagram showing an example of how the memory 4 stores a plurality of pixel data PDs constituting the image data 100. In this example, as shown in FIG. 3, in the image 200 generated by the image sensor 5, for example, the pixels in the x-th column from the left and the y-th row from the top are represented as pixels (x, y). 0 ≦ x ≦ X and 0 ≦ y ≦ Y, and the total number of pixels of the image 200 is ((X + 1) × (Y + 1)).

Further, the pixel data PD of the pixel (x, y) is represented as the pixel data PD (x, y). The data length of each pixel data PD is K bytes. For example, K = 3, and the data length of the pixel data PD is 24 bits. The lower data PDL of the 0th to 7th bits of the pixel data PD (x, y) is represented as the lower data PDL (x, y) [7: 0]. The median data PDM of the 8th to 15th bits of the pixel data PD (x, y) is represented as the median data PDM (x, y) [15: 8]. The upper data PDU of the 16th to 23rd bits of the pixel data PD (x, y) is represented as the upper data PDU (x, y) [23:16]. Then, in the memory 4, the address of the area for storing the lower data PDL (x, y) [7: 0] is represented by the address A (x, y) [7: 0], and the middle data PDM (x, y). The address of the area for storing [15: 8] is represented by the address A (x, y) [15: 8], and the address of the area for storing the upper data PDU (x, y) [23:16] is represented by the address A (address A (x, y) [15: 8]. x, y) It is represented by [23:16].

As shown in FIG. 2, in the memory 4, a plurality of pixel data PDs constituting the image data 100 are stored in a raster order with respect to storage areas of a plurality of addresses consecutive in ascending order. One pixel data PD is stored in the storage areas of three consecutive addresses in ascending order. The lower data PDL is stored in the storage area of the smallest address of the three addresses, and the middle data PDM is stored in the storage area of the second smallest address of the three addresses, and the upper data PDU is stored. Is stored in the storage area of the largest of the three addresses.

As described above, the DMA controller 3 can read 16 bytes of data stored in the storage areas of 16 consecutive addresses starting with the address specified for the memory 4 at a time. In this example, since the pixel data PD is 3 bytes, the 16-byte data read by the DMA controller 3 at a time includes 5 pixel data PDs.

For example, if the DMA controller 3 specifies the address A (0,0) [7: 0] for the memory 4, the DMA controller 3 has five pixel data PD (0,0), PD (1, 0), PD (2,0), PD (3,0), PD (4,0) and 1-byte lower data PDL (5,0) [7: 0] are read at once. Further, if the DMA controller 3 specifies, for example, the address A (X, 0) [15: 8] for the memory 4, the DMA controller 3 has a 1-byte medium data PDM (X, 0) [. 15: 8], 1-byte high-order data PDU (X, 0) [23:16], and 4-pixel data PD (0,1), PD (1,1), PD (2,1), PD (3,1), 1-byte lower data PDL (4,1) [7: 0], and 1-byte middle data PDM (4,1) [15: 8] are read at once.

Hereinafter, the address of the data or the address corresponding to the data means the address of the area in which the data is stored in the memory 4. Further, the data of a certain address or the data corresponding to a certain address means the data stored in the storage area of the certain address.

<Configuration of image processing circuit>
FIG. 4 is a diagram showing an example of the configuration of the image processing circuit 6. As shown in FIG. 4, the image processing circuit 6 includes an adjusting circuit 61, an adjusting circuit 62, a processing circuit 71, and a processing circuit 72.

The DMA controller 3 reads data from the memory 4 and inputs the read data to the adjustment circuit 61. The data bus width between the adjustment circuit 61 and the DMA controller 3 is N bytes, 16 bytes in this example. The adjustment circuit 61 discards unnecessary data included in the input data input from the DMA controller 3 and outputs the input data to the processing circuit 71. Specific information 610 for specifying data to be discarded by the adjustment circuit 61 is input to the adjustment circuit 61 from the processing circuit 72. The adjustment circuit 61 discards unnecessary data included in the input data input from the DMA controller 3 based on the specific information 610. The operation of the adjustment circuit 61 will be described in detail later.

The data bus width between the adjustment circuit 61 and the processing circuit 71 is M bytes. M is an integer of 2 or more, for example, M = 8. The processing circuit 71 has an M-byte alignment constraint, in this example, an 8-byte alignment constraint. The adjustment circuit 61 inputs data to the processing circuit 71 in 8-byte units (64-bit units). The processing circuit 71 outputs each of the plurality of pixel data PDs included in the data input from the adjustment circuit 61 one pixel at a time in the raster order.

The data bus width between the processing circuit 71 and the adjustment circuit 62 is L bytes. For example, L = 6. The adjustment circuit 62 inputs the image data output from the processing circuit 71 to the processing circuit 72 by discarding unnecessary pixel data PD included in the image data. When the processing circuit 71 outputs the necessary pixel data PD, the adjustment circuit 62 inputs the pixel data PD to the processing circuit 72, and when the processing circuit 71 outputs the unnecessary pixel data PD, the adjustment circuit 62 inputs the pixel data PD. , The unnecessary pixel data PD is discarded and is not input to the processing circuit 72. Specific information 620 for specifying the pixel data PD to be discarded by the adjustment circuit 62 is input to the adjustment circuit 62 from the processing circuit 72. The adjustment circuit 62 discards unnecessary pixel data PD included in the image data output from the processing circuit 71 based on the specific information 620. The operation of the adjustment circuit 62 will be described in detail later.

The processing circuit 72 performs various image processing on the image data input from the adjustment circuit 62 to generate image data according to the output device. The processing circuit 72 is sometimes called a GDU (Graphic Display Unit) core.

The processing circuit 72 performs gamma correction, enlargement processing, and the like on the image data input from the adjustment circuit 62, for example. Then, the processing circuit 72 combines the image indicated by the processed image data with, for example, another image to generate a display image. The processing circuit 72 performs gamma correction, format conversion, and the like on the display image data indicating the display image to generate display image data according to the output device. Then, the processing circuit 72 outputs the generated display image data to the output device. Further, the processing circuit 72 generates a synchronization signal and outputs the synchronization signal to the output device. The output device displays the image indicated by the display image data based on the display image data and the synchronization signal input from the processing circuit 72.

The CPU 2 generates the above-mentioned specific information 610 and 620 and inputs them to the processing circuit 72. The processing circuit 72 stores the input specific information 610 and 620 in a register (not shown). Then, the processing circuit 72 inputs the specific information 610 and 620 to the adjusting circuit 61 and the adjusting circuit 62, respectively.

<Details of operation of DMA controller and image processing circuit>
The output device not only displays the entire image 200 indicated by the image data 100 in the memory 4, but may also display a part of the image 200. For example, when the image 200 is enlarged and displayed on the output device or the image 200 is scrolled and displayed, a part of the image 200 is displayed on the output device. Therefore, the processing circuit 72 may need image data (image data 100) showing the entire image 200, or may need image data showing a part of the image 200. Hereinafter, the image data required by the processing circuit 72 may be referred to as "necessary image data". Further, the image indicated by the required image data, that is, the image required by the processing circuit 72 may be referred to as a "necessary image". Further, the image 200 may be referred to as "captured image 200", and the image data 100 may be referred to as "captured image data 100".

On the other hand, the DMA controller 3 may not be able to read only the necessary image data from the memory 4 due to its alignment constraint. This point will be described below.

FIG. 5 is a diagram showing an example of the required image 210. In FIG. 5, a diagonal line rising to the right is shown in the required image 210. In FIG. 5, the line number of the captured image 200 is shown on the left side of the captured image 200. In this example, a continuous positive integer starting from No. 1 is assigned to a plurality of lines constituting the image, for example, in order from the top line. In the example of FIG. 5, the required image 210 includes a part of each line from the first to the fifth of the captured image 200.

Due to the 16-byte alignment constraint, the DMA controller 3 can specify only addresses that are multiples of 128 bits with respect to the memory 4. In other words, the DMA controller 3 can read data from the memory 4 only in units of 16 bytes. Therefore, when the lower data PDL of the pixel data PD of the first pixel (leftmost pixel) of each line of the required image 210 is stored in an address that is not a multiple of 128 bits, it is combined with the pixel data PD. At least a part of the pixel data PD of the pixel on the left side of the first pixel of each line will be read out.

If the upper data PDU of the pixel data PD of the last pixel (rightmost pixel) of each line of the required image 210 is stored in an address that is not a multiple of 128 bits, the last pixel of each line. At least a part of the pixel data PD of the pixel on the right side of the last pixel of each line is read out together with the pixel data PD of.

As described above, the DMA controller 3 may not be able to read only the image data required by the processing circuit 72 in the captured image data 100 from the memory 4. As a result, the output device may display an image other than the required image.

Therefore, in the image processing device 1, the DMA controller 3 reads the data including the necessary image data from the memory 4 so as to keep the alignment constraint. That is, data in multiples of 16 bytes is input to the DMA controller 3. Then, the adjustment circuit 61 and the adjustment circuit 62 of the image processing circuit 6 input only the necessary image data to the processing circuit 72 by discarding unnecessary data included in the data read from the DMA controller 3. Thereby, the necessary image data can be obtained without changing the circuit having the alignment constraint, that is, the DMA controller 3. The operations of the DMA controller 3 and the image processing circuit 6 will be described in detail below.

FIG. 6 is a diagram showing a state in which data 400 of a plurality of lines for a portion of the captured image 200 including the required image 210 of FIG. 5 are arranged in the order of their addresses. In FIG. 6, diagonal lines rising to the right are shown in the data 410 of each line of the required image 210. Further, in FIG. 6, the number of the line is shown in the upper left of the line data 400 of the captured image 200. Hereinafter, the line data of the image may be referred to as "line data". Further, the data 400 may be referred to as "imaging line data 400", and the data 410 may be referred to as "necessary line data 410".

The DMA controller 3 and the image processing circuit 6 perform processing for each line of the required image 210. The DMA controller 3 reads the data 420 including the required line data 410 from the memory 4 for each line of the required image 210 so as to keep the alignment constraint. In FIG. 6, the data 420 is shown a downward-sloping diagonal line. Hereinafter, the data 420 may be referred to as "line-corresponding read data 420".

A plurality of addresses corresponding to the line-corresponding read data 420 are continuous. Further, due to the alignment constraint of the DMA controller 3, the start address (smallest address) of the line-corresponding read data 420 is a multiple of 128 bits, and the number of a plurality of addresses corresponding to the line-corresponding read data 420 is 128 bits. It is a multiple. The data length of the line-corresponding read data 420 is a multiple of 16 bytes.

Among the plurality of required line data 410s of the required image 210, the difference between the start address of the line-corresponding read data 420 including the required line data 410 and the start address of the required line data 410 is constant. Further, among the plurality of required line data 410s of the required image 210, the difference between the end address of the line-corresponding read data 420 including the required line data 410 and the end address of the required line data 410 is constant. Therefore, the number of the plurality of addresses corresponding to the line-corresponding read data 420 is constant among the plurality of line-corresponding read data 420 including the plurality of required line data 410 of the required image 210. In other words, the data length is constant among the plurality of line-corresponding read data 420s.

When the line-corresponding read data 420 is longer than the required line data 410, the line-corresponding read data 420 includes data other than the required line data 410, that is, unnecessary data 450, as shown in FIG. This unnecessary data 450 is discarded by the adjustment circuit 61 and the adjustment circuit 62. As a result, only the required line data 410 is input to the processing circuit 72. That is, only the image data indicating the required image 210 is input to the processing circuit 72. If the start address of the line-corresponding read data 420 is smaller than the start address of the required line data 410, the line-corresponding read data 420 has an address smaller than the start address of the required line data 410, as shown in FIG. Unnecessary data 450 is included. Further, if the end address of the line-corresponding read data 420 is larger than the end address of the required line data 410, the line-corresponding read data 420 is larger than the end address of the required line data 410, as shown in FIG. The unnecessary data 450 of the address is included.

In the example of FIG. 6, the line-corresponding read data 420 is included in the imaging line data 400. That is, a part of the imaging line data 400 is the line-corresponding read data 420. Therefore, the image data (hereinafter, referred to as “virtual image data”) in which the plurality of line-corresponding read data 420 including the data 410 of the plurality of lines constituting the required image 210 are each line data is shown in FIG. Image 220 will be shown roughly. The image 220 is a part of the captured image 200 and includes the necessary image 210. In FIG. 7, an oblique line downward to the right is shown in the image 220.

However, although the data at the start address of the required line data 410 is always the lower data PDL of the pixel data PD, the data at the start address of the line-corresponding read data 420 is the pixel data PD due to the alignment constraint of the DMA controller 3. It is not always the lower data PDL. Similarly, although the data at the end address of the required line data 410 is always the upper data PDU of the pixel data PD, the data at the end address of the line-corresponding read data 420 is the upper data PDU of the pixel data PD. Not limited. Therefore, the line-corresponding read data 420 may include some data but not the remaining data for a certain pixel data PD. Therefore, even when the line-corresponding read data 420 is included in the imaging line data 400, the line-corresponding read data 420 does not include all of the pixel data PD, and the pixels of the pixel data PD are included. It may not be possible to show it correctly. As a result, the virtual image data including the plurality of line-corresponding read data 420 may not be able to correctly show a part of the captured image 200. Hereinafter, the plurality of line-corresponding read data 420 including the data 410 of the plurality of lines constituting the required image 210 may be collectively referred to as "required image-corresponding read data".

In the example of FIG. 6, one line-corresponding read data 420 is included in one imaging line data 400. However, depending on the line length of the required image 210 or the relative position of the required image 210 in the captured image 200, one line-corresponding readout data 420 may exist across the plurality of imaging line data 400.

For example, consider a case where the end address of the required line data 410 is close to the end address of the imaging line data 400 including the required line data 410. In this case, as shown in FIG. 8, the end address of the line-corresponding read data 420 including the required line data 410 may be included in a plurality of addresses of the next imaging line data 400. That is, a part of the line-corresponding read data 420 including the required line data 410 included in the imaging line data 400 may be included in the next imaging line data 400. Further, consider a case where the start address of the required line data 410 is close to the start address of the imaging line data 400 including the required line data 410. In this case, the start address of the line-corresponding read data 420 including the required line data 410 may be included in a plurality of addresses of the previous imaging line data 400. That is, a part of the line-corresponding read data 420 including the required line data 410 included in the image pickup line data 400 may be included in the previous image pickup line data 400.

Further, consider the case where the start address of the data 410 of the first line of the required image 210 is close to the start address of the captured image data 100. In this case, the line-corresponding read data 420 including the data 410 of the first line may include data having an address smaller than the start address of the captured image data 100 (data other than the captured image data 100). Further, consider the case where the end address of the data 410 of the last line (the line having the highest number) of the required image 210 is close to the end address of the captured image data 100. In this case, the line-corresponding read data 420 including the last line data 410 may include data (data other than the image data 100) having an address larger than the end address of the captured image data 100.

Further, when the line length of the required image 210 is large, the line-corresponding read data 420 including the data 410 of the line with the required image 210 includes a part of the data 410 of the line before the certain line. , A part of the data 410 of the line after the certain line may be included. FIG. 9 is a diagram showing an example of such a situation. In FIG. 9, for convenience of explanation, diagonal lines are shown only in the required line data 410 included in the central imaging line data 400 and the line-corresponding read data 420 including the required line data 410.

As shown in FIG. 9, the line-corresponding read data 420 including the data 410 of the line with the required image 210 includes a part 410a of the data 410 of the line before the line and a part 410a of the data 410 of the line before the line and after the line. Part of data 410 of the line 410b and is included. Even in such a case, as shown in FIG. 9, all the data included in the line-corresponding read data 420 other than the corresponding required line data 410 becomes unnecessary data 450. That is, even if the line-corresponding read data 420 includes a part of the required line data 410 other than the corresponding required line data 410, the line-corresponding read data 420 includes the part. Data is treated as unnecessary data.

As described above, the DMA controller 3 having the alignment constraint reads the line-corresponding read data 420 including the data 410 of the line for each line of the required image 210. Then, the image processing circuit 6 discards unnecessary data 450 included in the read line-corresponding read data 420 for each line of the required image 210. As a result, even when the DMA controller 3 reads unnecessary data from the memory 4 due to the alignment constraint, the unnecessary data can be discarded and only the necessary data can be obtained in the circuit after the DMA controller 3. .. The operation of the DMA controller 3 and the image processing circuit 6 will be described in more detail below.

<Details of DMA controller operation>
FIG. 10 is a diagram for explaining an example of the operation of the DMA controller 3. In FIG. 10, similarly to FIG. 6, the state in which the data 400 of each line for the portion including the required image 210 in the captured image 200 is arranged in the order of their addresses is shown. In the example of FIG. 10, a part of the imaging line data 400 is the line-corresponding read data 420, as in FIG.

The DMA controller 3 reads a plurality of line-corresponding read data 420 including a plurality of required line data 410 constituting the required image data from the memory 4 in order from the one having the smallest start address, as indicated by the arrow 500. In other words, the DMA controller 3 reads out a plurality of line-corresponding read data 420s constituting the required image-corresponding read data from the memory 4 in order from the line corresponding to the required line data 410 included in the plurality of line-corresponding read data 420s located on the upper side. Further, the DMA controller 3 reads the read data 420 corresponding to each line in units of 16 bytes in order from the one having the smallest address. The DMA controller 3 reads a plurality of line-corresponding read data 420 from the memory 4 as described above based on the specific information 300 generated by the CPU 2. The DMA controller 3 inputs a plurality of line-corresponding read data 420s to the adjustment circuit 61 in the order read from the memory 4.

The specific information 300 for specifying the storage area in which the required image-corresponding read data is stored in the memory 4 includes a start address STA, an end address ENA, a continuous transfer size RPT, and a jump size OFS shown in FIG. include. The start address STA is the start address of the line-corresponding read data 420 including the data 410 of the top line (first line) of the required image 210. The end address ENA is the end address of the line-corresponding read data 420 including the data 410 of the bottom line (last line) of the required image 210. The continuous transfer size RPT is the data length of the line-corresponding read data 420. The jump size OFS indicates the difference between the start addresses of the two line-corresponding read data 420 including the data 410 of the two consecutive lines in the required image 210. The jump size OFS is a value obtained by subtracting the start address of the line-corresponding read data 420 read before the start address of the line-corresponding read data 420. The jump size OFS indicates how far the start address of the line-corresponding read data 420 to be read thereafter is from the start address of the line-corresponding read data 420. Due to the 16-byte alignment constraint of the DMA controller 3, each of the start address STA, the end address ENA, the continuous transfer size RPT, and the jump size OFS is set to a multiple of 128 bits.

The CPU 2 has a start address STA and a start address STA based on the necessary image identification information for specifying the area in which the necessary image 210 is stored in the memory 4, the alignment constraint of the DMA controller 3, and the data length of the pixel data PD. Determine the continuous transfer size RPT. Then, the CPU 2 determines the end address ENA and the jump size OFS based on the determined start address STA and continuous transfer size RPT, the data length of the imaging line data 400, and the required image identification information. The jump size OFS matches the data length of the imaging line data 400. The required image specific information includes, for example, the address of an area in which the lower data PDL of the pixel data PD of the leftmost pixel of the uppermost line of the required image 210 is stored (data 410 of the uppermost line of the required image 210). The start address of), the data length of the required line data 410, and the number of lines of the required image 210 are included. The required image 210 changes according to an image or the like displayed by the output device. The CPU 2 generates the required image specific information according to the required image 210. It can be said that the required image specific information is information related to the required image 210. The method of determining the start address STA and the continuous transfer size RPT will be described in detail later.

<Details of image processing circuit operation>
FIG. 11 is a diagram for explaining the operation of the image processing circuit 6. FIG. 11 shows one line-corresponding read data 420. The line-corresponding read data 420 includes left-side unnecessary data 450L having an address smaller than the start address of the required line data 410 and right-side unnecessary data 450R having an address larger than the end address of the required line data 410. ..

The adjustment circuit 61 discards the left unnecessary data 450L included in the line-corresponding read data 420 each time the line-corresponding read data 420 is input. Further, each time the line-corresponding read data 420 is input, the adjustment circuit 61 discards a part of the data 450R1 of the right-side unnecessary data 450R included in the line-corresponding read data 420. This data 450R1 is called "first right side unnecessary data 450R1". Further, among the right-side unnecessary data 450R, data other than the first right-side unnecessary data 450R1 is referred to as "second right-side unnecessary data 450R2".

The start address of the second right-side unnecessary data 450R2 matches the start address of the right-side unnecessary data 450R. The end address of the first right-side unnecessary data 450R1 matches the end address of the right-side unnecessary data 450R. The address next to the end address of the second right-side unnecessary data 450R2 becomes the start address of the first right-side unnecessary data 450R1. Each time the line-corresponding read data 420 is input, the adjustment circuit 61 transmits the input line-corresponding read data 420 to the processing circuit 71 after discarding the left-side unnecessary data 450L and the first right-side unnecessary data 450R1 included in the input line-corresponding read data 420. input.

The adjustment circuit 61 identifies the left side unnecessary data 450L and the first right side unnecessary data 450R1 based on the specific information 610 input from the processing circuit 72. The specific information 610 includes a data length DL1 of the left unnecessary data 450L (hereinafter referred to as "left unnecessary data length DL1") and a data length DL2 of the first right right unnecessary data 450R1 (hereinafter, "first right unnecessary data length DL2". ") Is included. In the line-corresponding read data 420, the adjustment circuit 61 discards a portion of the line-corresponding read data 420 having a left-side unnecessary data length DL of 1 minute in the direction in which the address increases from the data of the start address as the left-side unnecessary data 450L. Further, the adjustment circuit 61 discards the portion of the line-corresponding read data 420 that is equal to the first right-side unnecessary data length DL 2 minutes in the direction in which the address becomes smaller from the data at the end address as the first right-side unnecessary data 450R1.

Hereinafter, the line-corresponding read data 420 in which unnecessary data is discarded in the adjustment circuit 61, which is input to the processing circuit 71, is referred to as "first input data 460". In this example, the first input data 460 is the line-corresponding read data 420 in which the left-side unnecessary data 450L and the first right-side unnecessary data 450R1 are discarded. Each time the line-corresponding read data 420 is input, the adjusting circuit 61 generates the first input data 460 based on the input line-corresponding read data 420 and inputs the first input data 460 to the processing circuit 71. The first input data 460 generated based on the line-corresponding read data 420 is composed of the required line data 410 and the second right-side unnecessary data 450R2 included in the line-corresponding read data 420.

Each time the first input data 460 is input, the processing circuit 71 outputs a plurality of pixel data PDs included in the input first input data 460 one pixel at a time in ascending order of address.

Here, as described above, the processing circuit 71 has an 8-byte alignment constraint. Therefore, the data length of the first input data 460 input to the processing circuit 71 for each line needs to be a multiple of 8 bytes. Further, in order for the processing circuit 71 to output the pixel data PD one by one, the first input data 460 must not include only a part of the data of a certain pixel data PD.

Therefore, the data length of the second right-side unnecessary data 450R2 included in the first input data 460 is such that the data length of the first input data 460 is a multiple of 8 bytes and the first input data 460 is one of the pixel data PDs. It is set to a value that does not include only the data of the part. As a result, the 8-byte alignment constraint of the processing circuit 71 can be observed, and the processing circuit 71 can output pixel data PD for each pixel. Line-corresponding read data 420 having a data length that is a multiple of 16 bytes is input to the DMA controller 3, which is a processing circuit having a 16-byte alignment constraint, whereas the processing circuit 71 having an 8-byte alignment constraint receives input data 420. , The first input data 460 having a data length that is a multiple of 8 bytes is input.

The processing circuit 71 performs a process of sequentially outputting a plurality of pixel data PDs constituting the first input data 460 to the first input data 460 one pixel at a time. Therefore, it can be said that the first input data 460 input to the processing circuit 71 is output from the processing circuit 71 as it is as the content of the data, although the input form and the output form are different from each other. That is, it can be said that the first input data 460 is output from the processing circuit 71.

The data at the end address of the required line data 410 is the upper data PDU of the pixel data PD, and the data at the start address of the second right unnecessary data 450R2 following the required line data 410 is the lower data PDL of the pixel data PD. Become. Then, the data at the end address of the second right-side unnecessary data 450R2 becomes the upper data PDU of the pixel data PD. The data length of the second right-side unnecessary data 450R2 is the data length of the pixel data PD, that is, a multiple of 3 bytes.

Each time the first input data 460 is input from the processing circuit 71, the adjusting circuit 62 inputs the input first input data 460 to the processing circuit 72 after discarding the second right-side unnecessary data 450R2 included therein. To do. When the processing circuit 71 outputs the unnecessary pixel data PD included in the second right-side unnecessary data 450R2, the adjustment circuit 62 discards the unnecessary pixel data PD. On the other hand, when the processing circuit 71 outputs the necessary pixel data PD included in the required line data 410, the adjusting circuit 62 inputs the necessary pixel data PD to the processing circuit 72. As a result, only the required line data 410 is input to the processing circuit 72. Therefore, the processing circuit 72 can receive only the necessary image data. As described above, since the processing circuit 71 outputs the input plurality of pixel data PDs one pixel at a time in ascending order from the one with the smallest address, the adjustment circuit 62 outputs the plurality of pixel data PDs constituting the necessary image data. , Output to the processing circuit 72 one pixel at a time in raster order.

The adjustment circuit 62 identifies the second right-side unnecessary data 450R2 based on the specific information 620 input from the processing circuit 72. The specific information 620 includes a number of PDNs of a plurality of pixel data PDs constituting the second right-side unnecessary data 450R2 (hereinafter, referred to as “unnecessary pixel data number PDN”). In the plurality of pixel data PDs constituting the first input data 460 output from the processing circuit 71, the adjustment circuit 62 is for the number of unnecessary pixel data PDNs from the pixel data PD having the largest address to the direction in which the address becomes smaller. The pixel data PD is discarded as the second right-side unnecessary data 450R2.

<How to determine the start address, etc.>
Next, the CPU 2 includes the start address STA and continuous transfer size RPT included in the specific information 300, the left unnecessary data length DL1 and the first right unnecessary data length DL2 included in the specific information 610, and the unnecessary data included in the specific information 620. A method of determining the number of pixel data PDN will be described. Hereinafter, the start address STA, the continuous transfer size RPT, the left side unnecessary data length DL1, the first right side unnecessary data length DL2, and the unnecessary pixel data number PDN may be collectively referred to as "target information".

The CPU 2 determines the target information based on the necessary image identification information, the alignment constraint of the DMA controller 3 and the processing circuit 71, and the data length of the pixel data PD. Unless otherwise specified, the unit of each value (excluding the number of unnecessary pixel data PDN) in the formulas appearing below is "byte".

First, the CPU 2 obtains the data length DL3 (see FIG. 11) of the first input data 460 by the following equation (1).

DL4 (see FIG. 11) in the formula (1) indicates the data length of the required line data 410 included in the required image identification information. Further, M_K in the equation (1) indicates the least common multiple of M of the alignment constraint of M bytes of the processing circuit 71 and K of K bytes which is the data length of the pixel data PD. Then, [a] in the equation (1) is a function that returns a value (integer) rounded down to the nearest whole number of a. The same applies to [a] in the following equation. In the formula (1), the smallest value among the values that are the data length DL4 or more of the required line data 410, a multiple of M, and a multiple of K is the data length DL3 of the first input data 460. It has become.

Next, the CPU 2 obtains the number of unnecessary pixel data PDN (see FIG. 11) by the following equation (2).

Further, the CPU 2 obtains the start address STA (see FIG. 10) by the following equation (3).

The ADR in the equation (3) indicates the start address of the data 410 of the uppermost line of the required image 210. The start address ADR is included in the required image identification information. In the formula (3), the maximum value among the values that are equal to or less than the start address ADR of the data 410 in the uppermost line of the required image 210 and are multiples of N is the start address STA.

Next, the CPU 2 obtains the left unnecessary data length DL1 (FIG. 11) by the following equation (4).

Next, the CPU 2 obtains the continuous transfer size RPT (see FIGS. 10 and 11) by the following equation (5).

In equation (5), the smallest value among the values that are equal to or greater than the sum of the left unnecessary data length DL1 and the data length DL3 of the first input data 460 and are multiples of N is the continuous transfer size RPT. It has become.

Then, the CPU 2 obtains the first right-side unnecessary data length DL2 (see FIG. 11) by the following equation (6).

For example, assume that N = 16, M = 8, K = 3, and ADR = 123. Then, it is assumed that the required line data 410 is composed of 100 pixel data PDs and DL4 = 300. In this case, DL3 = 312, PDN = 4, STA = 112, DL1 = 11, RPT = 336, DL2 = 13.

As described above, when the CPU 2 determines the target information, the CPU 2 inputs the specific information 300 including the start address STA and the continuous transfer size RPT to the DMA controller 3. Further, the CPU 2 inputs the specific information 610 including the left-side unnecessary data length DL1 and the first right-side unnecessary data length DL2 and the specific information 620 including the unnecessary pixel data number PDN into the processing circuit 72. The processing circuit 72 inputs the input specific information 610 and 620 to the adjusting circuit 61 and the adjusting circuit 62, respectively. The CPU 2 determines the start address STA, the continuous transfer size RPT, the left side unnecessary data length DL1, the first right side unnecessary data length DL2, and the number of unnecessary pixel data PDN according to the required image 210. The CPU 2 discards the line-corresponding read data 420, the adjustment circuit 61, and the adjustment circuit 62 based on the N-byte alignment constraint of the DMA controller 3, the M-byte alignment constraint of the processing circuit 71, and the required image identification information. It can be said that it functions as a determination circuit that determines the data to be output.

At least one of the start address STA, the continuous transfer size RPT, the left side unnecessary data length DL1, the first right side unnecessary data length DL2, and the number of unnecessary pixel data PDNs may be determined by the processing circuit 72. When all of this information is determined by the processing circuit 72, the processing circuit 72 functions as a determination circuit for determining the line-corresponding read data 420 and the data to be discarded by the adjusting circuit 61 and the adjusting circuit 62. Further, when only a part of the information is determined by the processing circuit 72, the CPU 2 and the processing circuit 72 determine the line-corresponding read data 420 and the data discarded by the adjusting circuit 61 and the adjusting circuit 62. It functions as a decision circuit to decide.

Further, the processing performed by the processing circuit 71 may be processing other than the above, and the processing performed by the processing circuit 72 may be processing other than the above.

Further, the data bus width between the processing circuit 71 and the adjusting circuit 61 may be 1 byte. That is, the processing circuit 71 does not have to have an alignment constraint. In this case, the adjustment circuit 61 can input data to the processing circuit 71 in units of 1 byte, which is the smallest data unit handled by the image processing device 1. Therefore, the first input data 460 The above-mentioned restrictions on the data length DL3 are eliminated. Therefore, the adjustment circuit 61 can output the line-corresponding read data 420 from the DMA controller 3 after discarding all unnecessary data included in the read data 420. As a result, the adjustment circuit 62 becomes unnecessary.

Further, in the above example, the image including a part of each of a plurality of continuous lines in the captured image 200 is regarded as the required image 210 (see FIG. 7), but the required image 210 is not limited to this. For example, an image including all of each of a plurality of consecutive lines in the captured image 200 may be regarded as the required image 210. Further, in the captured image 200, an image including at least a part of each of a plurality of lines skipped by one line may be regarded as the required image 210. Further, in the captured image 200, an image including at least a part of each of a plurality of lines skipped by two lines may be regarded as the required image 210.

Further, in the above example, the relationship between the number of bytes N of the alignment constraint of the DMA controller 3 and the number of bytes M of the alignment constraint of the processing circuit 71 is N> M, but N = M may be used. It may be N <M. Further, the relationship between N, M and L is not limited to the above example.

Further, in the above example, the image processing apparatus 1 is provided with one processing circuit (processing circuit 71) having an alignment constraint, but a plurality of processing circuits having an alignment constraint may be provided. In this case, an adjustment circuit is provided in front of each of the plurality of processing circuits having alignment constraints. Each adjustment circuit discards unnecessary data included in the input data and inputs it to the subsequent processing circuit so as to satisfy the alignment constraint of the subsequent processing circuit. For example, when the processing circuit 72 has an alignment constraint, the adjusting circuit 62 inputs the input data to the processing circuit 72 by discarding unnecessary data included in the processing circuit 72 so as to satisfy the alignment constraint of the processing circuit 72. To do.

Further, instead of the DMA controller 3, a processing circuit having an alignment constraint other than the DMA controller may be provided.

As described above, in the image processing device 1, the adjustment circuit 61 discards unnecessary data included in the line-corresponding read data 420 for each line of the required image 210, and the line-corresponding read in which the unnecessary data is discarded. The first input data 460, which is the data 420, is input to the processing circuit 71. Therefore, when the DMA controller 3 having an alignment constraint, which is located in front of the processing circuit 71, outputs unnecessary data, the adjustment circuit 61 located between the DMA controller 3 and the processing circuit 71 discards the unnecessary data. Can be done. Therefore, it is possible to prevent unnecessary data from being input to the processing circuit 71 that processes the necessary image data.

Further, since unnecessary data is discarded from the line-corresponding read data 420 read from the memory 4 by the DMA controller 3, it is possible to prevent unnecessary data from being input to the subsequent processing circuit without changing the DMA controller 3. be able to. Therefore, the alignment constraint of the DMA controller 3 can be determined in consideration of the efficiency of the entire system in which the image processing device 1 is mounted. As a result, the efficiency of the entire system can be improved. Further, since the adjustment circuit 61 different from the CPU 2 discards unnecessary data from the line-corresponding read data 420, it is possible to suppress an increase in the load on the CPU 2. Further, it is better to add the adjustment circuit 61 and the adjustment circuit 62 to the image processing circuit 6 than to change the DMA controller 3 to remove the alignment constraint in order to read only the necessary image data from the memory 4. , The development manpower of the image processing apparatus 1 can be reduced. Further, by changing the line-corresponding read data 420 read from the memory 4, it is possible to easily cope with the change of the alignment constraint of the DMA controller 3.

Further, in the image processing device 1, the adjusting circuit 62 inputs the first input data 460 to the processing circuit 72 for each line by discarding unnecessary data included in the first input data 460. Therefore, when the processing circuit 71 having an alignment constraint, which is located in front of the processing circuit 72, outputs unnecessary data, the adjusting circuit 62 located between the processing circuit 71 and the processing circuit 72 discards the unnecessary data. Can be done. Therefore, it is possible to prevent unnecessary data from being input to the processing circuit 72 that processes the necessary image data without changing the processing circuit 71 having the alignment constraint.

Further, the image processing device 1 is provided with a determination circuit (for example, CPU 2) that determines the line-corresponding read data 420 read by the DMA controller 3 and the data to be discarded by the adjustment circuit 61 and the adjustment circuit 62. Therefore, even when the required image 210 is changed, the line-corresponding read data 420 and the unnecessary data included therein can be automatically determined by the determination circuit according to the required image 210. Therefore, it is possible to easily cope with the change of the required image 210.

Further, the adjustment circuit 61 does not need to be included in the line-corresponding read data 420 so that the data length of the first input data 460 input to the processing circuit 71 having the alignment constraint of M bytes is a multiple of the M bytes. Discard the data. Therefore, unnecessary data included in the line-corresponding read data 420 can be discarded, and data satisfying the alignment constraint of the processing circuit 71 can be input to the processing circuit 71.

In the above example, the adjustment circuit 62 outputs a plurality of pixel data PDs constituting the required image data indicating the required image 210, one pixel at a time in the raster order as shown in FIG. However, the image processing device 1 may be operated so that the adjustment circuit 62 can output the plurality of pixel data PDs one pixel at a time in the reverse raster order as shown in FIG. As a result, the output device can display a mirror image of the required image 210. The operation of the image processing device 1 in this case will be described below.

Similar to the above, the DMA controller 3 reads out a plurality of line-corresponding read data 420s in ascending order of the start address. On the other hand, the DMA controller 3 reads out the read data 420 corresponding to each line in order from the one having the largest address in units of 16 bytes, as shown by the arrow 510 in FIG. FIG. 14 shows line-corresponding read data 420 including data 410 for the top line of the required image 210.

Further, the adjustment circuit 61 discards the right-side unnecessary data 451R and the first left-side unnecessary data 451L1 included in the left-side unnecessary data 451L from the line-corresponding read data 420. Then, the adjustment circuit 62 discards the second left-side unnecessary data 451L2 included in the left-side unnecessary data 451L from the first input data 460.

The start address of the first left unnecessary data 451L1 matches the start address of the left unnecessary data 451L. The end address of the second left unnecessary data 451L2 matches the end address of the left unnecessary data 451L. The address next to the end address of the first left-side unnecessary data 451L1 becomes the start address of the second left-side unnecessary data 451L2. The end address of the right-side unnecessary data 451R matches the end address of the line-corresponding read data 420.

The processing circuit 71 outputs a plurality of pixel data PDs constituting the input first input data 460 one pixel at a time in descending order of address. The adjustment circuit 62 discards the second left unnecessary data 451L2 from the first input data 460, and outputs a plurality of pixel data PDs constituting the required line data 410 obtained by the first input data 460 one pixel at a time in descending order of address. As a result, the adjustment circuit 62 outputs a plurality of pixel data PDs constituting the required image data one pixel at a time in the reverse raster order.

The data length DL3 of the first input data 460 shown in FIG. 14 can be obtained by using the above equation (1). The number of unnecessary pixel data PDs PDN1 constituting the second left-side unnecessary data 451L2 can be obtained in the same manner as the above-mentioned number of unnecessary pixel data PDNs (see equation (2)). In order to obtain the data length DL5 of the right-side unnecessary data 451R (hereinafter referred to as "right-side unnecessary data length DL5"), the CPU 2 is the end address of the line-corresponding read data 420 including the data 410 of the uppermost line of the required image 210. The starting address STA1 (see FIG. 14) is obtained by the following equation (7).

ADR1 in the formula (7) indicates the end address (see FIG. 14) of the data 410 in the uppermost line of the required image 210.

The CPU 2 obtains the right unnecessary data length DL5 by the following formula (8) using the start address STA1 obtained by the formula (7).

Next, the CPU 2 obtains the continuous transfer size RPT1 in this example by the following equation (9).

Then, the CPU 2 obtains the data length DL6 of the first left unnecessary data 451L1 by the following equation (10).

In the CPU 2, the image processing device 1 operates in the first operation mode in which a plurality of pixel data PDs constituting the required image data are input to the processing circuit 72 in raster order, or the plurality of pixel data PDs process. It is determined whether or not the image processing apparatus 1 operates in the second operation mode in which the circuits 72 are input in the reverse raster order. Then, the CPU 2 controls the DMA controller 3 and the image processing circuit 6 so that the DMA controller 3 and the image processing circuit 6 perform processing according to the determined operation mode. As a result, the output device can display the required image 210 as it is, or display a mirror image of the required image 210.

In the above example, the image processing device 1 is mounted on the digital still camera, but may be mounted on other devices. For example, the image processing device 1 may be mounted on an OA (Office Automation) device (copier, intelligent multifunction device, projector, etc.). Further, the image processing device 1 may be mounted on an entertainment device (a game machine such as a pachinko machine, a game machine provided in a game center or the like, a game machine for personal use, etc.). Further, the image processing device 1 may be mounted on a medical device (medical monitor or the like). Further, the image processing device 1 may be mounted on an electronic device for educational purposes (tablet, electronic blackboard, etc.).

As described above, the image processing apparatus 1 has been described in detail, but the above description is an example in all aspects, and the present invention is not limited thereto. Further, the various modifications described above can be applied in combination as long as they do not contradict each other. Then, it is understood that innumerable variations not illustrated can be assumed without departing from the scope of the present invention.

1 image detection device 2 CPU
3 DMA controller 4 Memory 61, 62 Adjustment circuit 71, 72 Processing circuit

Claims (4)

The first processing circuit with A-byte alignment constraint and
A second processing circuit that performs the first processing on the input data,
Equipped with a first adjustment circuit
With respect to the image data indicating the image, the first data having a data length that is a multiple of the A bytes, including the data of the one line, is input to the first processing circuit for each line.
The first processing circuit outputs the second data, which is the first data, for each line, or performs the second processing on the first data for each line, and the second processing. Output the second data, which is the first data later,
The first adjustment circuit discards the first unnecessary data included in the second data for each line, and uses the third data, which is the second data in which the first unnecessary data is discarded, as the first data. 2 Input to the processing circuit,
The second processing circuit has a B-byte alignment constraint.
The first adjustment circuit is an image processing device that discards the first unnecessary data included in the second data so that the data length of the third data is a multiple of the B bytes. The image processing apparatus according to claim 1.
The second processing circuit performs the first processing on the third data for each line, and outputs the fourth data, which is the third data after the first processing.
A third processing circuit that performs the third processing on the input data,
An image processing apparatus further comprising a second adjusting circuit for inputting the fourth data to the third processing circuit by discarding the second unnecessary data included in the fourth data for each line. The image processing apparatus according to claim 2.
An image processing apparatus further comprising a determination circuit for determining the first data, the first unnecessary data, and the second unnecessary data based on the A-byte alignment constraint, the B-byte alignment constraint, and information about the image. .. The first adjustment circuit included in the image processing apparatus according to any one of claims 1 to 3.

Images