JP2010176547A - Controller included in image processor, control method and control processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a controller included in an image processor, which reduces time required for memory access and rapidly transforms a fisheye image or the like into a planar image, a control method and a control processing program therefor. <P>SOLUTION: The controller is included in the image processor that includes: an input image frame memory for storing the data of an input image imaged with a wide angle lens; a cache memory for storing data of blocks of which the number is smaller than the number of all blocks of a plurality of blocks into which the input image is divided in a predetermined size; and an output image frame memory for storing the data of an output image for which an object is projected on a plane. When the data of a block including a specified pixel in an input image corresponding to a pixel in the output image is not stored in the cache memory, the data of the block is obtained from the input image frame memory, is stored in a cache memory, and is obtained from the corresponding cache memory to be transformed into pixel data in the output image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technical field such as an image processing apparatus and method for converting an input image captured using a wide-angle lens (for example, a fisheye lens) or an omnidirectional mirror into an output image projected onto a plane, for example.

  2. Description of the Related Art Conventionally, an apparatus that performs fish-eye correction processing that converts a fish-eye image captured by, for example, a fish-eye camera into a planar image for display and outputs the image is known (for example, Patent Document 1).

  In this fisheye correction process, for example, a fisheye image input from a fisheye camera is stored in the input image frame memory, and each pixel of an image (size: m × n) to be obtained as an output result is input image. The pixel corresponding to the input image coordinate obtained by the calculation is acquired from the input image frame memory, and is stored in the output image frame memory. Yes. This process is repeated for all the pixels of the output image in raster order, so that a desired corrected image is finally obtained.

JP 2000-235645 A

  However, the conventional fish-eye correction process has a problem that it takes a long time to access the memory when acquiring pixels in the input image, which becomes a bottleneck of the entire system.

  Therefore, the present invention has been made in view of such problems and the like, and an image processing apparatus capable of reducing the time required for memory access and quickly converting a fisheye image or the like into a planar image or the like. It is an object to provide a control device, a control method, and a control processing program that are included.

  In order to solve the above problems, the invention according to claim 1 is an input image frame memory for storing data of an input image captured using a wide-angle lens (for example, a fisheye lens) or an omnidirectional mirror, Of a plurality of blocks obtained by dividing the input image into a predetermined size, a cache memory for storing data of a smaller number of blocks than the total number of blocks, and a surface (plane or curved surface) of an object based on the input image An output image frame memory for storing data of the output image projected on the image processing device, the control device included in the image processing device, wherein the pixel in the input image corresponding to one pixel in the output image is specified The input image when the data of the block including the specified pixel is not stored in the cache memory. First storage processing for acquiring the data of the block from the frame memory and storing the data of the block in the cache memory, acquiring the data of the specified pixel from the cache memory and converting it into the pixel data in the output image A conversion process and a second storage process for storing the converted pixel data in the output image frame memory are sequentially executed for all the pixels in the output image.

  According to a second aspect of the present invention, in the control device according to the first aspect, the specifying process calculates the coordinates of a pixel in the input image corresponding to the coordinates of one pixel in the output image. When the pixel in the input image is specified and the coordinates of the pixel in the input image are not an integer value, the conversion processing acquires data of a plurality of pixels located in the vicinity of the coordinate from the cache memory. Then, the data is converted into pixel data in the output image by performing an interpolation operation using the data.

  According to a third aspect of the present invention, in the control device according to the first or second aspect, for each of the blocks constituting the input image, data of each pixel included in the block is 1 on the input image frame memory. It is stored so as to fit in a line.

  According to a fourth aspect of the present invention, in the control device according to any one of the first to third aspects, in the specifying process, each block obtained by dividing the output image into a predetermined size is a block unit raster. In addition to focusing on the order, each pixel included in the focused block is focused on in the raster order of the pixel unit, and the pixel in the input image is specified for each focused pixel.

  According to a fifth aspect of the present invention, in the control device according to any one of the first to fourth aspects, the input image is a fish-eye image.

  According to a sixth aspect of the present invention, there is provided an input image frame memory for storing data of an input image captured using a wide-angle lens (for example, a fisheye lens) or an omnidirectional mirror, and the input image having a predetermined size. A cache memory for storing data of blocks smaller than the total number of blocks among a plurality of divided blocks, and an output for storing data of an output image obtained by projecting an object on a surface based on the input image A control method included in an image processing apparatus comprising: an image frame memory; a specifying step of specifying a pixel in the input image corresponding to one pixel in the output image, wherein the specified pixel is included When the block data is not stored in the cache memory, the data of the block is input from the input image frame memory. A first storage step of acquiring and storing the data of the block in the cache memory, a conversion step of acquiring the data of the specified pixel from the cache memory and converting it into pixel data in the output image, and the conversion The second storing step of storing the data of the pixels in the output image frame memory is sequentially executed for all the pixels in the output image.

  An invention of a control processing program according to a seventh aspect is characterized in that a computer is caused to function as the control device according to any one of the first to fifth aspects.

  According to the present invention, an input image is divided into blocks each having a predetermined size, data access is performed in blocks, the read block data is stored in the cache memory, and the pixel data is stored in the cache memory. Therefore, the time required for memory access can be shortened, and an input image such as a fisheye image can be quickly converted into a flat image or the like.

It is a figure which shows the structural example of the image processing apparatus which concerns on this embodiment. It is a figure which shows one pixel in an input image corresponding to one pixel in an output image. It is a conceptual diagram which shows an example of the input image divided | segmented into the block unit. It is a figure which shows the mode of the acquisition of the pixel in the input image which exists in the input image frame memory 13 in order to produce | generate a certain line of an output image. It is a figure which shows a mode when an input image is stored in the input image frame memory. In a prior art example, it is a figure which shows the acquisition order of the data of each pixel in an input image corresponding to each pixel in an output image. In this embodiment, it is a diagram showing the acquisition order of data of each pixel in the input image corresponding to each pixel in the output image. 3 is a flowchart illustrating an example of a fisheye correction process executed by an LSI circuit 12;

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The embodiment described below is an embodiment when the present invention is applied to an image processing apparatus.

[1. Overview of image processing apparatus configuration and functions]
First, the configuration and functional overview of the image processing apparatus according to the present embodiment will be described with reference to FIG.

  FIG. 1 is a diagram illustrating a configuration example of an image processing apparatus according to the present embodiment.

  As shown in FIG. 1, the image processing apparatus 1 includes an input image captured using a host controller 11, an LSI (Large Scale Integration) circuit 12 as an example of a control apparatus, and a fish-eye lens (an example of a wide-angle lens). An input image frame memory (frame buffer) 13 for storing data, and an object is projected on a plane (which may be a curved surface, but in this embodiment, a plane is taken as an example) based on the input image An output image frame memory 14 or the like for storing output image data is provided.

  The host controller 11 calculates and sets each parameter necessary for fisheye correction calculation according to the cutout position, cutout image size, and magnification of the correction processing (fisheye correction processing) for the fisheye image as the input image.

  The LSI circuit 12 includes a fisheye correction arithmetic circuit 121, an input image frame memory controller 122, a cache memory 123, an image interpolation circuit 124, and an output image frame memory controller 125.

  Based on the parameters set by the host controller 11, the fisheye correction arithmetic circuit 121 corresponds to each pixel in the output image (that is, the planar image region to be obtained as a result) in the input image (fisheye image). In order to specify the pixel, the corresponding coordinate (X, Y) in the input image of each pixel of the output image is calculated. FIG. 2 is a diagram illustrating one pixel in the input image corresponding to one pixel in the output image.

  For example, a plane that touches an arbitrary point on a virtual spherical surface of a fisheye lens for capturing a fisheye image is defined, and each coordinate on the plane is converted into each coordinate in the fisheye image by known coordinate conversion, and the above-described correspondence is made. Coordinates (X, Y) are calculated.

  The input image frame memory controller 122 stores input image data from a camera module or the like equipped with a fisheye lens in the input image frame memory 13. The input image frame memory controller 122 also stores pixel data (here, R (red), G (green), and B (blue)) at the corresponding coordinates (X, Y) calculated by the fisheye correction arithmetic circuit 121. Data indicating each brightness) is acquired from the frame image stored in the input image frame memory 13 via the cache memory 123 and sent to the image interpolation circuit 124.

  The image interpolation circuit 124 converts pixel data from the cache memory 123 into pixel data in the output image, and sends the pixel data to the output image frame memory controller 125.

  Here, when the corresponding coordinates calculated by the fisheye correction arithmetic circuit 121 are integer values, the pixel data from the cache memory 123 is directly used as the pixel data in the output image, and is output to the output image frame memory controller 125. Sent. On the other hand, since the corresponding coordinates calculated by the fisheye correction arithmetic circuit 121 are not necessarily integer values, when the corresponding coordinates are not integer values, the image interpolation circuit 124 passes the corresponding coordinates through the input image frame memory controller 122. By acquiring data of a plurality of pixels located in the vicinity of the coordinates from the cache memory 123 and performing an interpolation operation (for example, a known interpolation operation such as a bilinear method or a bicubic method) using the data, The data is converted into pixel data, and the converted data is sent to the output image frame memory controller 125.

  The output image frame memory controller 125 stores the pixel data from the image interpolation circuit 124 in the output image frame memory 14.

  The above processing is sequentially executed for all the pixels in the output image, for example, repeatedly performed in raster order, so that an output image is constructed on the output image frame memory 14 and a frame that matches the external device (monitor, recorder, etc.). Data is output. For example, when output to a monitor or the like, the output image frame memory controller 125 has two pieces of frame data, and constructs the next frame image while outputting the already interpolated frame data together with the vertical and horizontal synchronization signals. .

[2. Detailed functions of LSI circuit 12]
Next, functional details of the LSI circuit 12 will be described.

(2-1. How to use cache memory)
First, a method of using the cache memory 123 will be described with reference to FIG.

  FIG. 3 is a conceptual diagram illustrating an example of an input image divided into block units.

  As described above, since memory access requires a long time, it is necessary to reduce the number of memory accesses as much as possible in order to increase the system speed. A burst transfer function for exchanging a plurality of data in one access is known as one of the means for efficiently performing memory access, but in the data acquisition at the time of fisheye correction processing, the output image Each pixel in the input image (fisheye image) corresponding to each pixel arranged on one line (one row) in (planar image) is not arranged on one line (in other words, it is not a continuous address). In many cases, the row address (Row Address) at the time of accessing the input image frame memory 13 changes frequently, so that it cannot be effectively used as it is.

  Therefore, in the present invention, as shown in FIG. 3, the input image is divided into blocks each having a predetermined size, and data access is performed in units of blocks (in this example, one block is composed of 8 × 8 pixels). In addition, by storing (registering) the data of the block once read in the cache memory 123, the time required for memory access can be shortened. Note that the cache memory 123 does not store all blocks of the divided input image because of the storage capacity, but stores data of a smaller number of blocks than the total number of blocks.

  Here, an embodiment relating to data access in units of blocks will be described with reference to FIG.

  FIG. 4 is a diagram showing how pixels are acquired in the input image existing on the input image frame memory 13 in order to generate a certain line of the output image, from the start point to the end point of the arrow in FIG. It is assumed that the data of the pixels located so far are acquired.

  First, the input image frame memory controller 122 uses, for example, a pixel indicated as “1” in FIG. 4B as the first pixel in the corresponding coordinates (X, Y) calculated by the fisheye correction arithmetic circuit 121. And try to acquire the pixel. At this time, when data of 1-2 blocks (see FIG. 4A) including the specified target pixel is stored in the cache memory 123, the input image frame memory controller 122 reads the data from the cache memory 123. Data of the pixel is acquired. In this case, since it is not necessary to perform memory access from the input image frame memory controller 122 to the input image frame memory 13, the time required to acquire pixel data can be greatly reduced.

  On the other hand, when the data of 1-2 blocks including the specified pixel is not stored in the cache memory 123, the input image frame memory controller 122 acquires the data of the 1-2 blocks from the input image frame memory 13. Then, after the data of the block is stored in the cache memory 123, the data of the specified pixel is acquired from the cache memory 123.

  After the first pixel data is acquired, it is necessary to continuously acquire the pixel data, for example, in the order of the numbers shown in FIG. 4B (2 → 3 → 4 → 5...). However, since the data of 1-2 blocks including these pixels are already stored in the cache memory 123, the data of these pixels can be acquired from the cache memory 123 without accessing the memory.

  Even if the target pixel to be acquired is shifted from the 1-2 block to the 2-2 block or to another block, if the data of the target block is stored in the cache memory 123, the input image frame memory controller The operation 122 acquires the data of the target pixel from the cache memory 123 and reads the data from the input image frame memory 13 and stores it in the cache memory 123 for each target block if it is not stored in the cache memory 123. repeat. At this time, as described above, since the corresponding coordinates (acquired pixel coordinates) obtained by the calculation by the fisheye correction calculation circuit 121 are not necessarily integer values, the data of a plurality of pixels is read from the cache memory 123 as necessary. Acquired and interpolation calculation processing is performed. When a predetermined number or more of blocks (caches) are stored in the cache memory 123, the data of the oldest block among the currently stored blocks is overwritten. The number of blocks to be stored is preferably 4 or more in consideration of the interpolation calculation processing by the image interpolation circuit 124.

  By using the block unit cache in this way, the number of memory accesses can be greatly reduced as compared with the case where data of individual pixels is acquired from the input image frame memory 13.

(2-2. Method of storing data in input image frame memory)
Next, a method for storing data in the input image frame memory 13 will be described with reference to FIG.

  FIG. 5 is a diagram illustrating a state when an input image is stored in the input image frame memory 13.

  As described above, in the present invention, pixel data is read from the input image frame memory 13 in units of blocks. However, when the data of each pixel in the input image is stored in the input image frame memory 13 as they are in raster order, the unit of blocks is read. Is read from the input image frame memory 13, multiple memory accesses are required.

  Therefore, in the present invention, for each block constituting the input image, the data of each pixel included in the block is stored so as to fit on one line on the input image frame memory 13.

  When the input image frame memory controller 122 stores the input image from the camera module or the like in the input image frame memory 13, as shown in FIGS. 5 (a) and 5 (b), the same block (for example, 0-0 block) ) Data (for example, (0,0), (1,0),... (7,0), (0,1)...) Are included in FIG. As shown in (), the data is stored so as to fit in one line. Thereby, the input image frame memory controller 122 can read one block of data from the input image frame memory 13 and store it in the cache memory 123 by one memory access using burst transfer.

(2-3. Order of obtaining data from input image frame memory)
Next, the data acquisition order from the input image frame memory 13 will be described with reference to FIGS.

  6 and 7 are diagrams showing the order of obtaining data of each pixel in the input image corresponding to each pixel in the output image. FIG. 6 shows a conventional example, and FIG. 7 shows this embodiment.

  When performing image transformation / conversion such as fisheye correction, as shown in FIG. 6 (a), the pixel data corresponding to each pixel in the image is converted into the input image in the raster order (arrow order) of the output image. A method of obtaining an output image by reading from the stored input image frame memory 13 is general.

  At this time, as shown in FIG. 6, pixels on the input image corresponding to a certain line of the output image and several lines located before and after the line are often located in close proximity. In other words, as described above, when a block unit cache is used when acquiring pixel data from an input image, the same block is used when acquiring pixel data corresponding to a certain line and several lines before and after that line. There is a high possibility that it is necessary (FIG. 6 (b)).

  However, since the number of blocks that can be stored (held) as a cache is finite, when acquiring pixel data from the input image in the raster order of the output image, the data of the block that has been read once in a certain line is being processed There is a problem that data of another block is overwritten.

  For example, when the maximum number of blocks that can be stored in the cache memory 123 is 3, and considering the case of FIG. 6, in order to acquire pixel data corresponding to the (1) line on the output image, (1) on the input image When acquiring pixel data from the start point to the end point of the line, the cache memory 123 stores the pixel data in the order of blocks A, B, and C in the input image. When the data of the block D becomes necessary, the data of the block A is overwritten and the data of the block D is stored. Next, when the data of the block E becomes necessary, the data of the block B is overwritten and the data of the block E is stored.

  When (1) all the pixel data on the line are acquired and (2) the pixel data on the line is acquired, the data of the block A is required again, but the data of the block A is already overwritten. Therefore, it is necessary to read from the input image frame memory 13 again, and there is a waste of reading data once read from the input image frame memory 13 again. In this embodiment, the cache update is targeted for the oldest cache at that time, but the cache update method is not limited to this.

  Therefore, in the present invention, the output image (that is, the output image region to be obtained as a result) is divided into a predetermined size, and each divided block is focused on in the raster order of the block unit and included in the focused block. Focusing on each pixel in raster order, the pixel in the input image corresponding to the focused pixel is identified and its data is acquired.

  For example, as shown in FIG. 7A, first, the block a in the output image is focused, and each pixel included in the block a is focused in the raster order (the arrow order in FIG. 7A). Pixels in the input image corresponding to the pixels are acquired from the input image frame memory 13 and stored in the cache memory 123.

  Then, when the pixels corresponding to all the pixels in the block a are obtained from the input image and the block a is finished, these blocks are in the order of blocks b, c, d, e,..., L shown in FIG. Pixel data corresponding to the pixels in the block is acquired from the input image. As a result, the probability that the acquired pixels are continuously located on the same block increases, and as a result, the number of times of reading the block data required to generate the output image is reduced, and the time required for memory access Can be shortened.

  For example, the input image frame memory controller 122 first acquires pixel data corresponding to the pixels included in the block a in the output image from the blocks A and B in the input image shown in FIG. When the acquisition of all the pixel data included in is completed, the pixel data corresponding to the pixel included in the block b in the output image is acquired from the blocks B and C in the input image. Similarly, pixel data corresponding to the pixels included in the blocks c and d in the output image are acquired from the blocks C, D, and E in the input image.

  Here, assuming that the maximum number of blocks that can be stored in the cache memory 123 is 3, in the method of acquiring pixel data from the input image in units of blocks as described above, the pixel data included in the same block is read a plurality of times. The pixel data for three lines of the output image can be acquired without any problem.

[3. Fisheye Correction Processing in LSI Circuit 12]
Next, fisheye correction processing executed by the LSI circuit 12 will be described with reference to FIG.

  FIG. 8 is a flowchart showing an example of fisheye correction processing executed by the LSI circuit 12. In the following description, the size of one block when the output image is divided into blocks is (i × j), and the number of blocks in the horizontal and vertical directions as a result of the block division is m + 1 and n + 1, respectively. .

  The process shown in FIG. 8 is started by a start command from the host controller 11, for example.

  First, the input image frame memory controller 122 fetches input image data from a camera module or the like and stores it in the input image frame memory 13 (step S1). At this time, the block unit data is stored in the input image frame memory 13 so as to be arranged on one line.

  Next, the input image frame memory controller 122 determines the vertical block position Y (any value from Y = 1 to n + 1) in the output image (step S2), and the horizontal block position X in the output image. (Any value from X = 1 to m + 1) is determined (step S3). Thereby, the block to which attention is paid is determined. By repeating the processing of steps S2 and S3, as described in “2-3. Order of obtaining data from input image frame memory”, each block in the output image is focused on in the raster order of the block unit (that is, If the blocks are sequentially focused from the left end to the right end and reach the right end block, the block moves to the lower left block and repeats this).

  Next, the input image frame memory controller 122 determines the vertical coordinate y (any value from y = 1 to j) in the block of interest by the processing in steps S2 and S3 (step S4), and the block. The horizontal coordinate x (any value from x = 1 to i) is determined (step S5). Thereby, one pixel included in the noted block is noticed. By repeating the processes in steps S4 and S5, as described in “2-3. Order of obtaining data from input image frame memory”, each pixel included in the focused block is in the raster order of the pixel unit. It will be noticed.

  Thus, information such as the coordinate (x, y) of the focused pixel is sent to the fisheye correction calculation circuit 121, and the fisheye correction calculation circuit 121 corresponds to the coordinate (x, y) of the focused pixel. The corresponding coordinates (X, Y) of the pixels in the input image are calculated as described above (step S6). Thereby, the pixel in the input image corresponding to the focused pixel is identified.

  Next, the input image frame memory controller 122 determines whether or not the data of the specified block including the pixel in the input image (block in the input image) is stored in the cache memory 123 (step S7). When the data of the block is not stored in the cache memory 123 (step S7: NO), the process proceeds to step S8, and when the data of the block is stored in the cache memory 123 (step S7: YES). The process proceeds to step S10.

  In step S8, the input image frame memory controller 122 acquires (reads) the data of the specified block including the pixel in the input image from the input image frame memory 13 and stores it in the cache memory 123 (step S9). ), Go to step S10.

  In step S <b> 10, the input image frame memory controller 122 acquires the specified pixel data in the input image from the cache memory 123 and sends it to the image interpolation circuit 124.

  Next, the image interpolation circuit 124 converts the pixel data in the input image specified above into pixel data in the output image (step S11), that is, the corresponding coordinates calculated by the fisheye correction arithmetic circuit 121 are adjusted. If it is a numerical value, the pixel data acquired from the cache memory 123 is used as it is as the pixel data in the output image. If the corresponding coordinate is not an integer value, the interpolation calculation process is performed as described above, and the output image Pixel data is obtained and sent to the output image frame memory controller 125.

  Next, the output image frame memory controller 125 stores the pixel data in the output image frame memory 14 (step S12).

  The processing from the above steps S2 to S12 is repeatedly performed as shown in FIG. 8, so that an output image as a final result is constructed on the output image frame memory 14.

  Then, the output image frame memory controller 125 creates frame data based on the output image constructed on the output image frame memory 14, and outputs this to an external device such as a monitor (step S13). Thus, the output image obtained by the fisheye correction process is displayed on a monitor, for example.

  Note that the fish-eye correction process shown in FIG. 8 may be performed by a hardware circuit mounted in the LSI circuit 12, or a microcomputer mounted in the LSI circuit 12 may execute a predetermined program ( Software) may be performed.

  As described above, according to the above embodiment, the input image is divided into blocks each having a predetermined size, data access is performed in units of blocks, and the read block data is stored in the cache memory 123. Since the output image is generated by acquiring pixel data from the cache memory 123, the time required for memory access can be shortened, and an input image such as a fisheye image can be quickly converted into a planar image. it can.

  Also, the output image is divided into a predetermined size, and each divided block is focused on in the raster order of the block unit, and each pixel included in the focused block is focused on in the raster order to correspond to the focused pixel. Since the pixel data in the input image is obtained from the cache memory 123, it is possible to prevent waste of reading the data once read from the input image frame memory 13 in the above-described aspect.

  In the above embodiment, a fisheye image captured using a fisheye lens has been described as an example. However, other than this, for example, an image captured using an omnidirectional mirror (on a curved surface of an object) The present invention can also be applied to a projected image).

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 11 Host controller 12 LSI circuit 13 Input image frame memory 14 Output image frame memory 121 Fisheye correction arithmetic circuit 122 Input image frame memory controller 123 Cache memory 124 Image interpolation circuit 125 Output image frame memory controller

Claims (7)

An input image frame memory for storing data of an input image captured using a wide-angle lens or an omnidirectional mirror;
Among a plurality of blocks obtained by dividing the input image into a predetermined size, a cache memory for storing data of a smaller number of blocks than the total number of blocks;
An output image frame memory for storing data of an output image obtained by projecting an object on a surface based on the input image;
A control device included in an image processing apparatus comprising:
A specifying process for specifying a pixel in the input image corresponding to one pixel in the output image;
First, when the block data including the specified pixel is not stored in the cache memory, the block data is acquired from the input image frame memory and the block data is stored in the cache memory. Storage processing,
A conversion process for acquiring the data of the identified pixel from the cache memory and converting the data to the pixel data in the output image; and
A second storage process for storing the converted pixel data in the output image frame memory;
Is sequentially executed for all the pixels in the output image. The control device according to claim 1,
The specifying process specifies a pixel in the input image by calculating a coordinate of a pixel in the input image corresponding to a coordinate of one pixel in the output image,
When the coordinates of the pixel in the input image are not integer values, the conversion process obtains data of a plurality of pixels located in the vicinity of the coordinate from the cache memory, and performs an interpolation operation using the data A control device for converting into pixel data in the output image. The control device according to claim 1 or 2,
A control device, wherein for each block constituting the input image, data of each pixel included in the block is stored so as to fit in one line on the input image frame memory. In the control device according to any one of claims 1 to 3,
In the specific processing, attention is paid to each block obtained by dividing the output image into a predetermined size in the order of the raster in block units, and attention is paid to each pixel included in the focused block in the order of raster in pixel units. A control device that identifies a pixel in the input image for each pixel. The control device according to any one of claims 1 to 4,
The control apparatus according to claim 1, wherein the input image is a fish-eye image. An input image frame memory for storing data of an input image captured using a wide-angle lens or an omnidirectional mirror;
Among a plurality of blocks obtained by dividing the input image into a predetermined size, a cache memory for storing data of a smaller number of blocks than the total number of blocks;
An output image frame memory for storing data of an output image obtained by projecting an object on a surface based on the input image;
A control method included in an image processing apparatus comprising:
A specifying step of specifying a pixel in the input image corresponding to one pixel in the output image;
First, when the block data including the specified pixel is not stored in the cache memory, the block data is acquired from the input image frame memory and the block data is stored in the cache memory. Storage step,
A conversion step of acquiring the specified pixel data from the cache memory and converting the data into pixel data in the output image; and
A second storing step of storing the converted pixel data in the output image frame memory;
Is sequentially executed for all the pixels in the output image.   A control processing program for causing a computer to function as the control device according to any one of claims 1 to 5.

Images