KR102366523B1 - Image processing apparatus and image processing method - Google Patents

Abstract

The image processing apparatus of the present embodiment includes a tile division unit, a correction area calculation unit, an image area determination unit, a distortion correction unit, a cache area determination unit, and a data acquisition unit. The correction region calculating unit calculates the first region of the input image corresponding to the tile to be subjected to distortion correction. The image area determining unit determines a rectangular second area of the input image required for distortion correction. The data acquisition unit stores the image data of the second area in the storage unit. The distortion correction unit performs distortion correction on the image data of the second area. The cache area determining unit determines a cache area including image data to be reused among the second areas. The data acquisition unit, after distortion correction, leaves the image data in the cache area in the storage unit, and opens the storage area in the storage unit in which other image data is stored.

Description

Image processing apparatus and image processing method

The present invention relates to an image processing apparatus and an image processing method, and more particularly, to an image processing apparatus and method for performing distortion correction on an image affected by distortion aberration.

In the prior art, there is an image processing apparatus for correcting distortion aberration of a lens (see Japanese Patent Laid-Open No. 2009-43060). In this type of image processing apparatus, a region necessary for an output image after distortion correction is calculated, and image data of an input image region corresponding to the output image is read out from a memory loaded with an input image having distortion aberration. Perform distortion correction. In this case, for the purpose of reducing the circuit scale, the region of the output image is divided into a plurality of rectangular regions, and distortion correction processing is performed in units of rectangular regions. Here, the rectangular area is referred to as a tile (hereinafter referred to as a tile).

According to the above-described prior art, when distortion correction processing for each tile is performed, it is necessary to read image data of an input image area larger than the actual tile as input image data to be subjected to distortion correction from the memory. In this case, since the input image area overlaps between adjacent tiles, the image data of the input image area overlaps and is read out from the memory. Accordingly, there is a problem in that the frequency of reading image data from the memory in which the input image is loaded increases, and the processing load for reading the image data increases.

The problem of the background art is the content that the inventor possessed for the purpose of derivation of the present invention or acquired during the derivation of the present invention, and it cannot be said that it is necessarily known to the general public prior to the filing of the present invention.

Japanese Patent Laid-Open No. 2009-43060

SUMMARY Embodiments of the present invention provide an image processing apparatus and an image processing method capable of suppressing redundant reading of image data to be subjected to distortion correction.

An image processing apparatus according to an embodiment of the present invention includes: a division unit dividing an output image area into a plurality of tiles; a correction region calculating unit configured to calculate a first region of the input image corresponding to a tile to be subjected to distortion correction processing among the plurality of tiles; an image area determining unit that determines a rectangular second area of the input image including the image data of the first area; a data acquisition unit which acquires the image data of the second area and stores it as cache data in a storage unit; a distortion correction unit for performing the distortion correction processing on the image data of the second area stored in the storage unit; and a cache area determining unit that determines a cache area including image data to be used again as cache data from among the second area by referring to the location information of the first area.

The data acquisition unit, after the distortion correction processing is performed, leaves the image data of the cache area in the storage unit among the image data of the second area stored in the storage unit, and stores other image data in the storage unit. It has a configuration of an image processing apparatus in which the storage area of the storage unit is opened.

The data acquisition unit acquires, for example, the image data of the second area from which the image data of the cache area is excluded, and stores the image data in the storage unit.

The image area determining unit determines the cache area according to, for example, position information of a leading line of a tile adjacent to a tile to be subjected to the distortion correction processing among the plurality of tiles.

The data acquisition unit may determine whether or not it is possible to secure the cache area in the storage unit, for example, according to an integrated amount of image data stored as cache data in the storage unit, and secure the cache area. If not, the image data in the cache area is not stored in the storage unit.

An image processing method according to an embodiment of the present invention includes a first step of dividing an output image area into a plurality of tiles; a second step of calculating a first area of an input image corresponding to a tile to be subjected to distortion correction processing among the plurality of tiles; a third step of determining a rectangular second area of the input image including the image data of the first area; a fourth step of acquiring the image data of the second area and storing it as cache data in a storage unit; a fifth step of performing the distortion correction processing on the image data of the second area stored in the storage unit; and a sixth step of determining a cache area including image data to be used again as cache data from among the second area by referring to the location information of the first area.

In the fourth step, after the distortion correction processing is performed, the image data of the cache area is left in the storage unit among the image data of the second area stored in the storage unit, and other image data is stored in the storage unit. It has the structure of the image processing method which opens the storage area of the said memory|storage part.

According to the embodiments of the present invention, after the distortion correction processing is performed, of the image data of the second area stored in the storage unit, the image data of the cache area remains in the storage unit, and other images A storage area of the storage unit in which data is stored is opened.

That is, image data of the second area from which the image data of the cache area is excluded as image data to be used again is acquired and stored in the storage unit.

Accordingly, redundant reading of image data to be subjected to distortion correction can be suppressed.

1 is a block diagram showing the configuration of an image processing apparatus according to a first embodiment of the present invention.
2 is a flowchart for explaining an operation flow of the image processing apparatus according to the first embodiment of the present invention.
3 is a view for explaining an operation (tile division) of the tile division unit of the image processing apparatus according to the first embodiment of the present invention.
4 is a view for explaining the operation of the correction area calculating unit of the image processing apparatus according to the first embodiment of the present invention.
5 is a diagram for explaining the operation of the image area determining unit of the image processing apparatus according to the first embodiment of the present invention.
6A is a diagram for explaining a method of determining an input image area.
6B is a diagram for explaining an overlapping area of an input image area.
FIG. 7A is a diagram in which the influence of distortion aberration included in the shapes of the eleventh and twenty-first input tiles Q11 and Q21 illustrated in FIGS. 4 or 5 is exaggerated and expressed again.
FIG. 7B is a diagram in which the influence of distortion aberration included in the shapes of the twelfth and twenty-second input tiles Q12 and Q22 illustrated in FIGS. 4 or 5 is exaggerated and expressed again.
8 is a diagram for supplementary explanation of a problem in operation of the image processing apparatus according to the first embodiment of the present invention.
9 is a flowchart for explaining an operation flow of the image processing apparatus according to the second embodiment of the present invention.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

(Example 1)

1 is a block diagram showing the configuration of an image processing apparatus 100 according to a first embodiment of the present invention. The image processing apparatus 100 divides an output image area into a plurality of tiles and performs distortion correction on input image data for each tile, and includes a storage unit 110 , a memory 120 , and a data acquisition unit 130 . ), a distortion correcting unit 140 , a memory 150 , a tile dividing unit 160 , a correction area calculating unit 170 , an image area determining unit 180 , and a cache area determining unit 190 . Here, a tile refers to an image area serving as a processing unit for distortion correction (hereinafter referred to as a tile).

The storage unit 110 is a line memory for temporarily storing image data forming an input image (hereinafter referred to as "input image data"), and is composed of, for example, a plurality of SRAM (Static Random Access Memory). do. The storage unit 110 may be an external memory of the image processing apparatus 100 . The memory 120 into which the input image data is preloaded is, for example, a Synchronous Dynamic Random Access Memory (SDRAM). The memory 120 may be an external memory of the image processing apparatus 100 .

Here, the input image formed by the input image data is a captured image obtained by an imaging device (not shown) such as a digital camera, and includes the influence of optical properties including distortion aberration of a lens provided in the imaging device, there is. The input image data forming the input image is image data to be subjected to distortion correction processing by the distortion correction unit 140 of the image processing apparatus 100 .

The horizontal length of the tile is determined by the horizontal length of the line memory constituted by the storage unit 110 , for example, as long as 512 pixels. In addition, the vertical length of the tile is determined by the rear-end processing specification of the image processing apparatus 100, and when a codec processing of the Joint Photographic Experts Group (JPEG) standard is assumed as the rear-end processing, the vertical length of the tile is , for example 8 pixels. Of course, it is not limited to this example, and the tile size can be arbitrarily set.

The data acquisition unit 130 is an element for writing and reading input image data to and from the storage unit 110 and the memory 120 . The data acquisition unit 130 acquires input image data from the memory 120 in units of tiles and stores it as cache data in the storage unit 110 , and at the same time reads the input image data from the storage unit 110 and corrects the distortion. It is supplied to the government 140 . The data acquisition unit 130 includes a data supply unit 131 , a cache controller 132 , a read area determination unit 133 , a table management unit 134 , and a table 135 .

Here, the data supply unit 131 reads input image data from a storage area of the memory 120 specified by a read area, which will be described later, determined by the read area determination unit 133, and reads the input image data to the cache controller 132 . element to supply The cache controller 132 is an element for writing and reading input image data to and from the storage area of the storage unit 110 specified by management information to be described later supplied from the table management unit 134 .

The read area determination unit 133 determines, as a read area, a storage area in which input image data to be actually read from the memory 120 is stored among the input image areas to be described later determined by the image area determination unit 180 . element to do it. The table management unit 134 is an element for managing the storage area of the storage unit 110 in the table 135 . In the table 135, among the storage areas of the storage unit 110, management information indicating an empty area in which input image data can be loaded and a used area in which the input image data is loaded are registered so that they can be updated.

The distortion correction unit 140 disposed at the rear end of the data acquisition unit 130 applies the input image data read out from the storage unit 110 by the data acquisition unit 130 among the input image data stored in the storage unit 110 . It is an element for performing a predetermined distortion correction process for the . The distortion correction unit 140 generates an output image in which the influence of distortion aberration included in the input image is suppressed by, for example, interpolating the input image at a predetermined interpolation ratio. Of course, it is not limited to this example, and the distortion correction processing by the distortion correction unit 140 may be a correction processing according to an arbitrary algorithm.

The memory 150 is an element for loading image data (hereinafter referred to as "output image data") resulting from the distortion correction processing performed by the distortion correction unit 140, and is, for example, SDRAM. The memory 150 may be an external memory of the image processing apparatus 100 . Hereinafter, an image formed by the output image data is referred to as an "output image".

The tile division unit 160 is an element for dividing the display area of the entire output image (hereinafter, referred to as “output image area”) formed by the above-described output image data into a plurality of tiles serving as a processing unit for distortion correction. am. Hereinafter, a tile of an output image corresponding to one tile serving as a processing unit for distortion correction is referred to as an "output tile". Each size of the plurality of output tiles divided by the tile divider 160 is assumed to be the same. As described above, since the size of the output tile is determined by the horizontal length of the line memory constituted by the storage unit 110 and the processing specification at the rear end, the number of tile divisions performed by the tile division unit 160 . is uniquely determined according to the size of the output tile.

The tile divider 160 outputs address information in the address space of the memory 120 corresponding to the output tile region as information on the divided output tile region. In addition, the tile dividing unit 160 has a built-in counter for specifying each of the plurality of output tiles P11 to P44 in the distortion correction processing of the input image data.

The correction region calculating unit 170 is an input image region corresponding to one tile to be subjected to distortion correction processing by the distortion correction unit 140 among a plurality of tiles of the output image divided by the tile dividing unit 160 . It is an element for calculating position and phase information of (the first region). Here, the phase information is information for specifying a region of an input image corresponding to a tile of an output image, and is information indicating an intermediate position between pixels.

Hereinafter, an area (first area) of an input image corresponding to one tile to be subjected to distortion correction processing is referred to as an "input tile". The input tile (first region) of the input image corresponds to an image region in which pixels of the tile before distortion correction including the influence of distortion aberration exist. The input tile (the first area) will be described in detail. The correction area calculating unit 170 sequentially calculates the positions of the input tiles (first areas) of the input image corresponding to each of the plurality of tiles divided by the tile dividing unit 160 .

The image region determining unit 180 is configured to provide a rectangular input image region ( 2nd area) is a factor for determining. The input image region (second region) of the input image corresponds to an image region in which pixels corresponding to a set of input image data serving as a processing unit for distortion correction by the distortion correction unit 140 exist. Details of the input image area (second area) will be described later. Hereinafter, the set of input image data in the input image area (second area) is referred to as a "kernel".

The cache area determining unit 190 refers to the position of the input tile (first area) of the input image calculated by the correction area calculating unit 170 as cache data in the input image area (second area) of the input image. It is an element for determining a cache area containing input image data to be used again. The above-described data acquisition unit 130 caches the image data (kernel) of the input image region (second region) stored in the storage unit 110 after the distortion correction processing is performed by the distortion correction unit 140 . The input image data of the cache area determined by the area determining unit 190 is left in the storage unit 110 to be used again. In this case, the data acquisition unit 130 opens the storage area of the storage unit 110 in which image data other than the input image data of the cache area determined by the cache area determination unit 190 is stored to be an empty area. . The related content will be described in more detail. Next, the operation of the image processing apparatus 100 according to the first embodiment will be described according to the flowchart shown in FIG. 2 .

2 is a flowchart illustrating an operation flow of the image processing apparatus 100 according to the first embodiment of the present invention.

Here, as a premise for the operation of the image processing apparatus 100 , it is assumed that input image data to be subjected to distortion correction is loaded in the memory 120 in advance.

The tile dividing unit 160 divides the output image area corresponding to the display area of the entire output image obtained by distortion-correcting the input image into a plurality of tiles (step ST1). Here, the output image area to be divided into tiles is an area of the entire output image in which distortion aberration of a lens provided in an imaging device (not shown) such as a digital camera is corrected, and is a virtual input image that does not include the influence of distortion aberration. covers the entire area.

3 is a diagram for explaining an operation (tile division) of the tile division unit 160 of the image processing apparatus 100 according to the first embodiment of the present invention.

In the example of FIG. 3 , for convenience of explanation, the output image area P is divided into 16 output tiles P11 to P44 in 4 rows and 4 columns. However, the number of divisions (the number of tiles) of the output image area P can be arbitrarily set.

The image processing apparatus 100 sequentially targets each of the plurality of tile-divided output tiles P11 to P44 for distortion correction, but steps ST2 to ST10 of FIG. is carried out Accordingly, steps ST2 to ST10 are repeatedly performed as many as the number of tiles. Hereinafter, the repetition cycle of steps ST2 to ST10 is referred to as a "distortion correction cycle".

In the first embodiment, a plurality of output tiles P11 to P44 are managed by a counter (not shown) built into the tile divider 160, and the tile indicated by the count value of the counter is changed in the current distortion correction cycle. of distortion correction processing. Then, when the current distortion correction cycle ends, the count value of the counter is incremented, and the tile indicated by the incremented count value is subjected to distortion correction processing in the next distortion correction cycle. Here, for convenience of description, coefficient values 1, 2, 3, 4, ..., 13 , 14, 15, 16) are assigned.

The correction area calculating unit 170 provides an input image corresponding to each tile on the output image area for each of the plurality of output tiles P11 to P44 divided by the tile division unit 160 according to the count value of the counter. The positions of the input tiles (the first area) of are sequentially calculated (step ST2).

4 is a view for explaining the operation of the correction area calculating unit 170 of the image processing apparatus according to the first embodiment of the present invention. In FIG. 4, the dashed-dotted line indicates each area of the output tiles P11 to P44 shown in FIG. 3, and the solid line represents the input image corresponding to the plurality of output tiles P11 to P44 on the output image area. A plurality of input tiles Q11 to Q44 are shown.

The correction area calculating unit 170 uses a function representing optical characteristics of a lens of an imaging device that supplies input image data to be subjected to distortion correction loaded into the memory 120 according to the count value of the counter, Each position of the plurality of input tiles Q11 to Q44 of the input image corresponding to the output tiles P11 to P44 is sequentially calculated. The function describes the correspondence between the input image and the output image. For example, the amount of image distortion caused by the distortion aberration of the lens provided in the imaging device that supplies the input image data loaded into the memory 120 . is a function representing

Next, the image region determining unit 180 sequentially determines the input image regions R11 to R44 shown in FIG. 5 as a rectangular input image region (second region) of the input image according to the count value of the counter. (Step ST3). Here, the input image area (second area) of the input image is an image area including a set of input image data required for image distortion correction of each tile of the plurality of input tiles Q11 to Q44 of the input image. The image area determiner 180 is configured to perform a cache, which will be described later, according to position information of a head horizontal line of a vertical tile adjacent to a tile to be subjected to distortion correction processing among a plurality of input tiles Q11 to Q44 of the input image. An input image area corresponding to the area is determined.

5 is a diagram for explaining the operation of the image area determiner 180 of the image processing apparatus 100 according to the first embodiment of the present invention. In FIG. 5 , input tiles Q11 to Q44 indicated by solid lines are input tiles Q11 to Q44 of the input image shown in FIG. 4 . In the input image regions R11 to R44 indicated by dotted lines, pixels necessary for distortion correction of each tile exist.

Referring to FIG. 5 , for example, when the input tile Q11 of the input image corresponding to the output tile P11 of the above-described output image is distortion-corrected, the image region determiner 180 determines the input tile Q11. A rectangular input image area R11 (second area) including The vertex position of the input image region R11 is set so as to include input image data of predetermined neighboring pixels necessary for pixel interpolation of the input tile Q11 during distortion correction processing as described later. The respective vertex positions of the input image regions R12 to R44 are determined similarly for each of the other input tiles Q12 to Q44.

6A and 6B are diagrams for supplementary explanation of the operation of the image region determiner 180 of the image processing apparatus 100 according to the first embodiment of the present invention, and are the input image regions R11 to R44 described above. It is a drawing showing an example of a method of determining the position of each vertex of . 6A is a diagram for explaining a method of determining the input image area R11. 6B is a diagram for explaining an overlapping region between the input image region R11 and the input image region 21 .

Referring to FIG. 6A , for example, when determining the input image region R11, the image region determiner 180 determines that the input image region R11 surrounding the input tile Q11 of the input image is the input tile ( The position of each side of the input image area R11 is determined so as to surround the entirety of Q11). Specifically, the image region determining unit 180 determines that the minimum vertical distance between the leading horizontal line L11 of the input tile Q11 and the exterior of the input image region R11 is a distortion correction process (interpolation process). The position of the external appearance of the input image area R11 is determined so that the distance corresponding to the predetermined number of peripheral pixels required for the .

In addition, the image area determining unit 180 determines the vertical direction between the first horizontal line L21 of the input tile Q21 adjacent in the vertical direction of the input tile Q11 and the lower side of the input image area R11. The position of the lower side of the input image region R11 is determined so that the minimum distance becomes a distance corresponding to a predetermined number of peripheral pixels required for the distortion correction processing (interpolation processing).

The positions of the left and right sides of the input image area R11 are also determined in this way.

In this way, when the external appearance, the lower side, the left side, and each position of the right side of the input image region R11 are determined, the vertex positions of the input image region R11 are determined. The position of each vertex of the other input image regions R12 to R44 is also determined using the same method as for the above-described input image region R11.

Here, the input image region R11 surrounding the input tile Q11 is a region including input image data necessary for the distortion correction processing of the input tile Q11. The input image region R21 surrounding the input tile Q21 is a region including input image data required for distortion correction processing of the input tile Q21. When the input image region R11 and the input image region R21 are determined, an overlapping region S of the input image region R11 and the input image region R21 is generated as shown in Fig. 6B.

If input image data included in an image region individually specified by each of the input image regions R11 and R21 is read from the memory 120, the input image data in the overlapping region S is duplicated. is read out, and the processing load for reading the input image data from the memory 120 increases.

Therefore, in the first embodiment, the input image data of the overlapping area S is left as cache data in the storage unit 110, for example, in the distortion correction processing of the input tile Q11. Conversely, input image data that will not be used is not left as a cache. Then, in the distortion correction cycle for the distortion correction processing of the input tile Q21, when the input image data of the input tile Q21 included in the input image region R21 is read from the memory 120, the overlapping region ( The reading of the image data of S) is omitted. In other words, when the data acquisition unit 130 reads the input image data of the input tile Q21 from the memory 120 and stores it in the storage unit 110 , the storage unit in which the input image data of the overlapping area S is left. The image data of the input image area R21 (second area) is acquired from the memory 120 and stored in the storage unit 110 except for the image data of the cache area at (110).

Next, the read area determination unit 133 of the data acquisition unit 130 actually transfers the input image data required for distortion correction of the above-described input image areas R11 to R44 from the memory 120 according to the count value of the counter. It is sequentially determined as a read area for reading, and the input image data of this read area is requested from the data supply unit 131 (step ST4). Here, the read area determiner 133 may include input image areas R11 to R14 of the input tiles Q11 to Q14 of the first row among the input tiles Q11 to Q44 illustrated in FIG. 4 or 5 . ) is determined as a read area as it is, and for the input tiles Q21 to Q24 of the second row, the area corresponding to the above-described overlapping area S is excluded from each of the input image areas R21 to R24. is determined as the read area (step ST3).

In addition, the read area determiner 133 , with respect to the input tiles Q21 to Q24 of the second row, receives each input image of the input image areas R21 to R44 determined by the image area determiner 180 . The data supply unit 131 requests input image data excluding the input image data contained in the cache area of the storage unit 110 corresponding to the above-described overlapping area S from the data (step ST4). The same operation for the tile of the second row is applied to the input tiles Q31 to Q34 of the third row and the input tiles Q41 to Q44 of the fourth row.

Subsequently, the data supply unit 131 of the data acquisition unit 130 , according to the request of the read area determination unit 133 , applies the input image areas R11 to Q14 for the input tiles Q11 to Q14 of the first row. R14) is sequentially acquired from the memory 120, supplied to the cache controller 132, and stored in the storage unit 110 via the cache controller 132 (step ST5).

Also, according to the request of the read area determining unit 133 , the data supply unit 131 is configured to provide input image areas R21 to R44 for the input tiles Q21 to Q44 in the second to fourth rows. Among the included input image data, input image data excluding the input image data included in the cache area of the storage unit 110 corresponding to the overlapping area S is acquired from the memory 120 and stored in the storage unit 110 . (Step ST5).

In addition, as described above, when the correction area calculating unit 170 calculates each position of the input tiles Q11 to Q44 of the input image corresponding to the plurality of output tiles P11 to P44 of the output image, Information about the location is supplied to the table management unit 134 . The table management unit 134 refers to the information on the respective positions of the input tiles Q11 to Q44 of the input image supplied from the correction area calculating unit 170, and the input image data of each tile to be subjected to the distortion correction processing. An empty area of the storage area of the storage unit 110 is allocated to .

The table management unit 134 registers management information indicating the storage area of the storage unit 110 allocated to the input image data of the input tile Q11 in the table 135 , and updates the table 135 . In addition, the table management unit 134 supplies management information managed in the updated table 135 to the cache controller 132 . The same is true for the other input tiles Q12 to Q44.

Next, the cache controller 132 retrieves the input image data of the input tile Q11 of the input image to be subjected to the distortion correction processing from the storage area of the storage 110 indicated by the management information supplied from the table management unit 134 . It is read and supplied to the distortion correction unit 140 (step ST6). The same is true for the other input tiles Q12 to Q44.

Next, the distortion correction unit 140 performs a predetermined distortion correction process on the input image data of the input tile Q11 supplied from the cache controller 132 (step ST7), and the output image data in which the distortion aberration is corrected. to create Then, the distortion correction unit 140 stores the output image data of the input tile Q11 subjected to the distortion correction process in the memory 150 (step ST8). The same applies to other input tiles Q12 to Q44.

Subsequently, the cache area determiner 190 determines the plurality of output tiles P11 calculated by the correction area calculating unit 170 after the distortion correction processing for the input tile Q11 by the distortion correcting unit 140 is finished. to P44), the input image region R11 of the input tile Q11 (th 2), a cache area containing input image data to be used again as cache data is determined (step ST9). The same is true for the other input tiles Q12 to Q44.

Here, as described above, the cache area including the image data to be used again is an area corresponding to the overlapping area S shown in FIG. 6B . The cache area determining unit 190 specifies the overlapping area S using the position of the leading line of the input tile Q11 and the position of the leading line of the input tile Q21, and selecting the specified overlapping area S It is determined as a cache area containing input image data to be reused (step ST9).

Next, the table management unit 134 writes management information indicating that input image data to be reused is loaded in the cache area determined by the cache area determination unit 190 into the table 135 , and updates the table 135 . (step ST10). At this time, the table management unit 134 stores information for opening a storage area loaded with input image data other than the cache area among the input image data included in the input image area R11 of the input tile Q11 to the table 135 . ), and the table 135 is updated (step ST10).

The table management unit 134 supplies the management information managed in the updated table 135 to the cache controller 132 . The cache controller 132 stores the cache area (duplicate area (redundancy area) The input image data in the cache area corresponding to S)) is left in the storage unit 110 to be used again. In this case, the cache controller 132 opens a storage area of the storage unit 110 in which input image data other than the cache area is stored under the management of the table management unit 134 to be an empty area (or other than the cache area). (overwrites the input image data newly supplied from the data supply unit 131 to the storage area of the storage unit 110 in which the input image data is stored). Next, the above-described series of processing (steps ST1 to ST10) is repeatedly performed for the input tiles Q12 to Q14 in the first row, similar to the input tile Q11.

Next, processing for distortion correction targeting the input tiles Q21 to Q24 of the second row (steps ST1 to ST10) is performed. In this case, when the table management unit 134 of the data acquisition unit 130 acquires the input image data included in the input image area R21 of the input tile Q21 of the second row from the memory 120, the table ( 135), the overlap area S with the input image area R11 of the above-described input tile Q11 among the input image data included in the input image area R21 of the input tile Q21. It is recognized that the input image data is already stored in the storage unit 110 as cache data.

Accordingly, the table management unit 134 supplies the read area determination unit 133 with management information indicating that the input image data of the overlapping area S has already been stored in the storage unit 110 as cache data.

The read area determination unit 133 receives the management information supplied from the table management unit 134 and determines the read area of the input image area R21 of the input tile Q21, from the input image area R21 to the overlapping area ( The area remaining except S) is determined as a read area (step ST3), and the input image data of the read area is requested from the data supply unit 131 (step ST4).

The data supply unit 131 receives a request from the read area determination unit 133 and according to the read area determined by the read area determination unit 133 , the input image area R21 excluding the overlapping area S The input image data of the area is acquired from the memory 120 and stored in the storage unit 110 via the cache controller 132 (step ST5). Accordingly, the input image data of the overlapping region S between the input image region R11 and the input image region R21 is not repeatedly read out from the memory 120 , and the input tile Q21 is stored in the storage unit 110 . All of the input image data in the input image area R21 of are stored as cache data.

The cache controller 132 reads the input image data of the input tile Q21 from the storage 110 (step ST6). The distortion correction unit 140 receives the input image data of the input image region R21 of the input tile Q21 stored in the storage 110 from the cache controller 132 and uses the input image data of the input tile Q21 . Thus, distortion correction processing is performed (step ST7). The distortion correction unit 140 stores the distortion-corrected image data in the memory 150 (step ST8).

Thereafter, determination of the cache area corresponding to the overlapping area of the input image area R21 of the input tile Q21 and the configuration of the table 135 are performed in the same manner as in the first row described above (steps ST9 and S10). . Further, the above-described series of processing (steps ST1 to ST10) is repeatedly performed for the input tiles Q22 to Q24 of the second row in the same manner as for the input tile Q21.

When the distortion correction processing is completed for the input tiles Q21 to Q24 of the second row, the input tiles Q31 to Q34 of the third row and the input tiles Q41 to Q44 of the fourth row are described in the same manner. A series of processing (steps ST1 to ST10) is performed. Also in this case, similarly to the input tiles Q21 to Q24 of the second row described above, a read area excluding the overlapping area S is determined from the input image area, and the input image data of the input image represented by the read area is obtained. do.

Therefore, according to the first embodiment, since the input image data of the overlapping area S is not read over and over, the amount of input image data read from the memory 120 can be reduced. Accordingly, the load of the read processing for reading the input image data from the memory 120 can be reduced. In addition, since it is not necessary to unnecessarily read input image data from the memory 120, the processing speed can be improved and power consumption can be reduced.

Further, according to the first embodiment, since the storage area of the storage unit 110 in which input image data other than the cache area corresponding to the overlapping area S is stored is used as an empty area, an input image other than the cache area is used as an empty area. Since the data storage area is opened, the storage capacity of the storage unit 110 can be reduced. Accordingly, the circuit scale of the image processing apparatus 100 can be suppressed, and the apparatus cost can be reduced.

In addition, when distortion correction processing using the conventional tile division method is applied to an image including the influence of distortion aberration, the overlap region S illustrated in FIG. 6B is generated, which increases processing time and reduces power consumption. increases In addition, many cache storage devices are required for distortion correction processing.

(Second embodiment)

Next, a second embodiment of the present invention will be described.

In the above-described first embodiment, the integration amount of the input image data of the cache area of the storage unit 110 corresponding to the overlapping area S (strictly, the integration amount of the input image data of the cache area, the amount of distortion correction When the target data amount plus input image data for one tile) exceeds the storage capacity of the storage unit 110 , the storage unit 110 stores the input image data of each tile of the input image as cache data. A difficult situation may occur, and a case may occur that interferes with the distortion correction process by the distortion correction unit 140 .

A situation in which it is difficult to secure input image data necessary for distortion correction as cache data in the storage unit 110 will be specifically described with reference to FIGS. 7A and 7B .

7A and 7B are diagrams for explaining the operational problems of the image processing apparatus 100 according to the first embodiment of the present invention described above. drawings for

FIG. 7A is an exaggerated representation of the influence of distortion aberration included in the shapes of the eleventh and twenty-first input tiles Q11 and Q21 illustrated in FIGS. 4 or 5 described above. FIG. 7B is a diagram in which the influence of distortion aberration included in the shapes of the twelfth and twenty-second input tiles Q12 and Q22 illustrated in FIGS. 4 or 5 is exaggerated and expressed again.

As shown in Fig. 7A, the deformation of the input tiles Q11 and Q21 located in the peripheral region of the image where the influence of the distortion aberration becomes significant increases.

In contrast, as shown in FIG. 7B , the distortion of the input tiles Q11 and Q21 positioned in the region close to the center of the image with little influence of the distortion aberration is reduced. Accordingly, the area of the overlapping region S1 between the input image region R11 of the input tile Q12 located in the peripheral region of the image and the input image region R21 of the input tile Q21 is close to the center of the image. It shows a tendency to become larger than the area of the overlapping area|region S2 between the input image area|region R12 of the input tile Q12 located in the area|region, and the input image area|region R22 of Q22.

FIG. 8 is a diagram for supplementary explanation of the operational problems of the image processing apparatus 100 according to the first embodiment of the present invention. The relationship between the accumulated amount of input image data in the cache area of the corresponding storage unit 110 and the position of each tile in the horizontal direction of the input image is shown. In Fig. 8, the left direction of the horizontal axis indicates the left direction of the image, and the right direction of the horizontal axis indicates the right direction of the image.

In the example of FIG. 8 , 12 columns indicating the integration amount of input image data are shown. Each column is, for example, the head among the input tiles Q11 to Q14 of the first row illustrated in FIG. 4 or FIG. 5 . The accumulated amount of input image data in the cache area of the storage unit 110 corresponding to the overlap area S from the input tile Q11 to the tile to be subjected to the distortion correction processing is shown.

In Fig. 8, as shown by the inclination of the arrow (dotted line) indicated in the area A on the left side of the image, the amount of deformation of the tile increases in the area A on the left side of the image as illustrated in Fig. 7A. Therefore, the increase rate of the integration amount of the input image data in the cache area of the storage unit 110 tends to increase. The same tendency is shown also for the area|region C around the right side. On the other hand, in the area B near the center of the image, since the amount of deformation of the tile is small, the increase rate of the integration amount of the input image data in the cache area of the storage unit 110 tends to be small.

In the example of FIG. 8 , in the area C in the right direction, the increase rate of the integration amount of the input image data in the cache area of the storage unit 110 rapidly increases. Accordingly, when the subject of the distortion correction processing shifts from the area B to the tile located in the area C, the input image data stored as cache data in the storage unit 110 accelerates. In this case, the possibility that it is difficult to secure the input image data necessary for the distortion correction processing as cache data in the storage unit 110 sharply increases.

Accordingly, in the second embodiment, when the accumulated amount of input image data in the cache area corresponding to the overlapping area S exceeds the predetermined value TH, the data acquisition unit 130 determines the data corresponding to the overlapping area S. Without leaving the input image data in the cache area in the storage unit 110, all the storage areas of the storage unit 110 are allocated to the kernel. Here, the predetermined value TH is the amount of input image data obtained by multiplying the total storage capacity of the storage unit 110 by the safety factor, and as an example, a data amount of 80% of the total storage capacity of the storage unit 110 is the value it represents. However, the predetermined value TH can be arbitrarily set.

Next, the operation of the image processing apparatus 100 according to the second embodiment will be described according to the flow shown in FIG. 9 .

In the second embodiment, Figs. 1 to 5 of the above-described first embodiment are used. Also, except for step ST8A among the steps shown in Fig. 9, the operation flow of the image processing apparatus 100 according to the second embodiment is the same as that of the first embodiment. Therefore, the description will be focused on step ST8A here.

Schematically, in the second embodiment, the data acquisition unit 130 performs the following distortion correction processing according to the accumulated amount of input image data in the cache area corresponding to the overlap area S stored in the storage unit 110 . At the time, it is determined whether or not a cache area for the input image data in the overlapping area S can be secured in the storage unit 110 , and when the cache area cannot be secured, the cache area is stored in the storage unit 110 . The image data of the area is not stored in the storage unit 110 . Otherwise, it is the same as that of the first embodiment.

In detail, paying attention to step ST8A, as in the first embodiment, when the image data distortion-corrected by the distortion correction unit 140 is stored in the memory 150 (step ST8), the table management unit ( 134) refers to the table 135, and indicates whether or not the amount of data in the used storage area among all the storage areas of the storage unit 110 does not exceed the predetermined value TH, that is, the overlapping area to be used in the next distortion correction process. It is determined whether or not it is possible to secure a cache area for storing the input image data of (S) (step ST8A).

After the distortion correction processing for the input tile Q11 by the distortion correction unit 140 is finished, the cache region determiner 190 stores the overlapping regions S(S1, S2) to be used in the next distortion correction cycle. When the corresponding cache area can be secured (step ST8A; "Yes"), the position of the input tile of the input image corresponding to each tile of the plurality of output tiles P11 to P44 calculated by the correction area calculating unit 170 A cache area including image data to be used again as cache data among the input image areas R11 (second area) of the input tile Q11 determined by the image area determining unit 180 is determined with reference to the location information on (step ST9).

The table management unit 134 of the data acquisition unit 130 writes, in the table 135, management information indicating that input image data to be used again is loaded into the cache area determined by the cache area determination unit 190 at the same time. , writes management information indicating that the storage area loaded with input image data other than the cache area is opened to the table 135, and updates the table 135 (step ST10).

In contrast, in the total storage area of the storage unit 110, the amount of data in the used storage area exceeds the predetermined value TH, and a cache area corresponding to the overlapping area S to be used in the next distortion correction process is secured. If not possible (step ST8A; "No"), the table management unit 134 uses the entire storage area of the storage unit 110 to store the input image data (kernel) of the input image area of the distortion correction target tile. and, after the distortion correction processing, the input image data of the cache area corresponding to the overlapping area S is not left.

Other contents are the same as in the first embodiment.

Therefore, according to the second embodiment, when the cache area corresponding to the redundant area S cannot be secured in the storage unit 110, all the storage areas of the storage unit 110 are allocated to the kernel storage area. Therefore, the distortion correction processing by the distortion correction unit 140 is not affected. Accordingly, it is possible to stably perform the distortion correction processing for all tiles.

Also, according to the second embodiment, in a situation in which it is possible to secure the cache region corresponding to the overlapping region S in the storage unit 110 , as in the first embodiment, unnecessary input from the memory 120 is performed. The effect of suppressing the reading of image data can be acquired.

Although the present invention is expressed as an image processing apparatus in the above-described embodiment, the present invention can also be expressed as an image processing method. In this case, the image processing method according to the present invention includes a first step of dividing an output image area into a plurality of tiles; a second step of calculating a first area of the input image corresponding to a tile to be subjected to distortion correction processing among the plurality of tiles; a third step of determining a rectangular second area of the input image including the image data of the first area; a fourth step of acquiring the image data of the second area and storing it as cache data in a storage unit; a fifth step of performing the distortion correction processing on the image data of the second area stored in the storage unit; and a sixth step of determining a cache area including image data to be used again as cache data from among the second area by referring to the location information of the first area, wherein the fourth step also includes the distortion correction processing is performed, image processing of leaving the image data of the cache area in the storage unit among the image data of the second area stored in the storage unit and opening the storage area of the storage unit in which other image data is stored can be expressed as a method.

As described above, according to the present embodiments, after the distortion correction processing is performed, of the image data of the second area stored in the storage unit, the image data of the cache area remains in the storage unit, and other images The storage area of the storage unit in which the data is stored is opened.

That is, the image data of the second area from which the image data of the cache area as image data to be used again is excluded is acquired and stored in the storage unit.

Accordingly, redundant reading of image data to be subjected to distortion correction can be suppressed.

The present invention is not limited to the above-described embodiments, and various changes, modifications, and substitutions are possible without departing from the spirit of the present invention. For example, although the present invention is expressed as an image processing apparatus and method in the above-described embodiments, the present invention may also be expressed as, for example, a distortion correction apparatus, a distortion correction circuit, a distortion correction method, and the like.

The present invention can be used in various apparatuses that perform image processing.

100: an image processing device, 110: a storage unit;
120: memory, 130: data acquisition unit,
131: data supply unit, 132: cache controller;
133: read area determining unit, 134: table management unit;
135: table, 140: distortion correction unit,
150: memory, 160: tile division,
170: correction area calculation unit;
180: image area determining unit;
ST1 to ST10, ST8A: processing steps.

Claims (6)

a division unit dividing the output image area into a plurality of tiles;
a correction region calculating unit configured to calculate a first region of the input image corresponding to a tile to be subjected to distortion correction processing among the plurality of tiles;
an image area determining unit that determines a rectangular second area of the input image including the image data of the first area;
The image data of the second area is acquired and stored as cache data in a storage unit, and it is determined whether or not a cache area can be secured in the storage unit according to the accumulated amount of the image data stored as cache data in the storage unit and a data acquisition unit for allocating all storage areas of the storage unit as storage areas for the input image without storing the image data of the cache area in the storage unit, when the cache area cannot be secured;
a distortion correction unit for performing the distortion correction processing on the image data of the second area stored in the storage unit; and
a cache area determining unit that determines the cache area including image data to be used again as cache data from among the second area by referring to the location information of the first area;
The data acquisition unit, after the distortion correction processing is performed, leaves the image data of the cache area in the storage unit among the image data of the second area stored in the storage unit, and stores other image data in the storage unit. An image processing apparatus that opens a storage area of a storage unit. According to claim 1, wherein the data acquisition unit,
The image processing apparatus of claim 1, wherein the image data of the second area from which the image data of the cache area is excluded is acquired and stored in the storage unit. The method of claim 1, wherein the image area determining unit comprises:
and determining the cache area according to position information of a leading line of a tile adjacent to a tile to be subjected to the distortion correction processing among the plurality of tiles. The method according to claim 2, wherein the image area determining unit comprises:
and determining the cache area according to position information of a leading line of a tile adjacent to a tile to be subjected to the distortion correction processing among the plurality of tiles. delete a first step of dividing an output image area into a plurality of tiles;
a second step of calculating a first area of the input image corresponding to a tile to be subjected to distortion correction processing among the plurality of tiles;
a third step of determining a rectangular second area of the input image including the image data of the first area;
a fourth step of acquiring the image data of the second area and storing it as cache data in a storage unit;
a fifth step of performing the distortion correction processing on the image data of the second area stored in the storage unit;
a sixth step of determining a cache area including image data to be used again as cache data from among the second area by referring to the location information of the first area; and
In accordance with the accumulated amount of image data stored as cache data in the storage unit, it is determined whether or not the cache area can be secured in the storage unit, and if the cache area cannot be secured, the cache is stored in the storage unit. an eighth step of determining not to store the image data of the region, and allocating all storage regions of the storage unit as storage regions of the input image;
In the fourth step, after the distortion correction processing is performed, the image data of the cache area is left in the storage unit among the image data of the second area stored in the storage unit, and other image data is stored in the storage unit. An image processing method of opening a storage area of the storage unit.

Images