JP2012133521A - Image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for properly correcting distortion of an image having the distortion while suppressing cost increase in a distortion correction processing device.SOLUTION: An image processing device (4) comprises: a calculation part (11) for correcting image distortion included in an input image and creating an output image; an input side memory (12) for performing AD conversion on an input image signal indicating the input image, and for holding obtained input image data; an output side memory (14) for holding output image data corresponding to the output image; a memory (13) for calculation having a storage area for storing pixel data; and correction processing memory copy table data (15) which is referred to when the image distortion is corrected. The calculation part (11) performs processing of pixel data corresponding to multiple pixels to be processed by single data processing, when the multiple pixels to be processed are indicated by processing object pixel information (25).

Description

  The present invention relates to an image processing apparatus.

  With the widespread use of automobiles equipped with back guide monitor cameras and an increase in surveillance cameras installed for the purpose of crime prevention, etc., there has been a demand for lower prices for cameras that take moving pictures. Further, a camera used for a back guide monitor camera or a surveillance camera is required to have a wide field of view.

  When a subject is imaged with a camera equipped with an optical lens, geometric distortion (distortion aberration) occurs in an image that passes through the lens to some extent. The imaging lens provided in the imaging apparatus ensures a desired field of view, magnification, and the like by combining a plurality of lenses. The distortion generated in the captured image increases as the number of optical lenses included in the used imaging lens decreases. It is also known that the wider the lens, the greater the distortion that occurs in the image.

  If distortion correction is not performed on an image captured by a wide-angle lens having a large distortion, the peripheral portion of the image remains largely distorted. Such an image is difficult for the user to see. As a technique for distortion correction processing for correcting this distortion aberration, a processing technique for optically correcting an image by increasing the number of lenses incorporated in a photographing lens is known.

  However, generally, when the number of lenses incorporated in the photographing lens increases, the price of the photographing lens increases, and accordingly, the price of the imaging device also increases. In order to satisfy the demand for price reduction of the imaging apparatus, it is necessary to reduce the number of optical lenses included in the photographing lens. In order to perform distortion correction while suppressing an increase in cost, correction by digital processing is more suitable than optical processing. A distortion correction processing technique using digital data processing is known (for example, see Patent Document 1).

  Japanese Patent Application Laid-Open No. 2004-133260 discloses a technique that can realize distortion correction corresponding to the shift even when the lens center is shifted from the reference position of the imaging surface in addition to the distortion correction. FIG. 1 is a block diagram showing a configuration of an image distortion correction processing apparatus disclosed in Patent Document 1. As shown in FIG. Referring to FIG. 1, in the image distortion correction processing apparatus disclosed in Patent Document 1, a controller 102 is connected to a camera 101, and a display 103 is connected to the controller 102. The controller 102 includes an AD converter 104 connected to the camera 101, a calculation unit 105 connected to the AD converter 104, and a DA converter 106 connected between the calculation unit 105 and the display 103. is doing. Further, the controller 102 is provided with a memory 107 and a ROM 108 connected to the calculation unit 105.

  A memory 107 is for storing an image signal captured by the camera 101. In addition, the ROM 108 stores in advance a necessary pixel movement amount for each address of the output image for correcting image distortion due to lens distortion of the camera 101 as a map.

  FIG. 2 is a diagram illustrating a map area and a use area of the ROM when the lens center of the camera 101 and the reference position of the imaging surface match in the image distortion correction processing apparatus described in Patent Document 1. As shown in FIG. 2, the ROM 108 stores in advance the necessary movement amount of pixels in the map area 110 wider than the map data use area 109 corresponding to the output image.

  In the image distortion correction processing apparatus described in Patent Document 1, when distortion correction is performed, the image signal of the line at the upper end of the screen is sequentially captured by the camera 101. The captured image signal is A / D converted by the AD converter 104 and then written to the memory 107 via the arithmetic unit 105. Next, the image signal of the second line from the upper end of the screen is captured, and the image signal of the lower line is sequentially captured and written in the memory 107 in the same manner.

  The calculation unit 105 reads from the map data in the use area 109 of the ROM 108 the required vertical and horizontal movement amounts of each pixel for correcting image distortion due to lens distortion of the camera 101. The arithmetic unit 105 reads out an image signal corresponding to the uppermost line of the output image from the memory 107 in accordance with the movement amount read from the ROM 108, sequentially performs D / A conversion with the DA converter 106, and outputs it to the display 103. To do. Similarly, the calculation unit 105 reads out an image signal corresponding to the second line from the upper end of the output image according to the movement amount of each pixel, and outputs the image signal.

  In this way, image signals are taken in order from the uppermost line of the screen to the lower line and written into the memory 107. Image signals corresponding to each line of the output image are sequentially read from the memory 107 and output to the display 103 according to the amount of movement of each pixel. When the writing of the image signal for the line at the lower end of the screen is completed, the image signal for the line at the upper end of the screen is taken in again in a cyclic manner. Similarly, when the output of the image signal of the lower end line of the output image is completed, the image signal corresponding to the upper end line of the output image is read again from the memory 107 and output in a cyclic manner.

JP 2001-186409 A

  Distortion correction is a non-linear image deformation process that has a heavy processing load, and has a property that the larger the distortion, the heavier the processing load. For example, the amount of movement of each pixel in the vertical and horizontal directions when performing distortion correction is larger in an image with large distortion than in an image with small distortion. In addition, the time required for the distortion correction process is longer for an image with a large distortion compared to an image with a small distortion.

  Patent Document 1 describes that the calculation unit 105 can obtain an output image subjected to distortion correction processing by moving pixels according to map data for correcting distortion of an image stored in the ROM 108. Yes. In the technique described in Patent Document 1, the pixel movement processing capability of the calculation unit 105 determines the distortion correction processing capability.

  A high-performance arithmetic processing circuit is required to perform an appropriate distortion correction process on a moving image shot by an imaging apparatus having an inexpensive wide-angle lens with large distortion. However, such an arithmetic processing circuit is expensive and causes an increase in the cost of the distortion correction processing device.

  The problem to be solved by the present invention is to provide a technique for appropriately correcting distortion of an image having distortion while suppressing an increase in cost of the distortion correction processing apparatus.

  [Means for Solving the Problems] will be described below using the numbers used in [DETAILED DESCRIPTION]. These numbers are added to clarify the correspondence between the description of [Claims] and [Mode for Carrying Out the Invention]. However, these numbers should not be used to interpret the technical scope of the invention described in [Claims].

In order to solve the above-described problem, an arithmetic unit (11) that corrects image distortion included in an input image to generate an output image, and input image data obtained by AD conversion of an input image signal indicating the input image An input side memory (12) that holds output image data, an output side memory (14) that holds output image data corresponding to the output image, an arithmetic memory (13) having a storage area for storing pixel data, and image distortion. An image processing apparatus (4) comprising correction processing memory copy table data (15) referred to when correction is performed is configured.
The correction processing memory copy table data (15) preferably includes a plurality of records in which the processing order is determined. Each of the plurality of records relates to an instruction field (22) indicating information related to an instruction executed by the arithmetic unit (11), and at least one processing target pixel processed in one data process in response to the instruction. It is preferable to include processing target pixel information (25) indicating information.
Then, when a plurality of processing target pixels are indicated in the processing target pixel information (25), the calculation unit (11) processes pixel data corresponding to the plurality of processing target pixels in one data process.

  In the image processing apparatus (4), the processing target pixel information (25) includes a reference source information field (24) indicating reference source information holding pixel data read by the calculation unit (11), and a calculation unit (11 ) Includes a writing destination information field (23) indicating writing destination information when writing pixel data.

  In the image processing device (4), the input image data includes a plurality of input image pixel data, and the input side memory (12) assigns an input side memory address corresponding to each of the plurality of input image pixel data in a one-to-one relationship. It is preferable to have it. The output image data preferably includes a plurality of output image pixel data, and the output side memory (14) preferably has an output side memory address corresponding to each of the plurality of output image pixel data. .

  In the image processing device (4), when the command is a read command, the calculation unit (11) reads the input image pixel data by specifying the input-side memory address based on the reference source information and reading the read image pixel data. It is preferable to write the data as transfer target pixel data in the storage area of the calculation memory (13) based on the write destination information. When the command is a write command, the storage area of the calculation memory (13) is specified based on the reference source information, the transfer target pixel data is read, and the read transfer target pixel data is written as output image pixel data. It is preferable to specify the output side memory address based on the previous information and write to the output side memory (14).

  To briefly explain the effects obtained by typical inventions among the inventions disclosed in the present application, the distortion correction processing apparatus is appropriately corrected for distortion while suppressing an increase in cost of the distortion correction processing apparatus. Technology can be provided.

  When the distortion correction processing is realized by image copy by pixel address conversion, the memory copy processing is a bottleneck of processing. The present invention includes a memory copy processing table optimized for distortion correction system hardware in order to speed up the memory copy processing. Thereby, memory copy processing can be performed by sequential memory access.

FIG. 1 is a block diagram showing a configuration of a conventional image distortion correction processing apparatus. FIG. 2 is a diagram showing a correspondence between a ROM map area and a use area in a conventional image distortion correction processing apparatus. FIG. 3 is a block diagram illustrating the configuration of the imaging system 1 according to the first embodiment. FIG. 4 is a table illustrating the configuration of the pixel address conversion data 16. FIG. 5 is a block diagram illustrating the configuration of the input image memory 12. FIG. 6 is a block diagram illustrating the configuration of the output image memory 14. FIG. 7 is a table illustrating the configuration of the correction processing memory copy table data 15. FIG. 8 is a diagram illustrating the correspondence between an original image (an image before correction) and a corrected image (an image after correction). FIG. 9 is a block diagram illustrating the operation of the image processing device 4 in the imaging system 1 of the present embodiment. FIG. 10 is a block diagram illustrating a method for creating the correction processing memory copy table data 15. FIG. 11 is a block diagram illustrating the configuration of the imaging system 1 of the second embodiment.

[First Embodiment]
Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description thereof will be omitted.

  FIG. 3 is a block diagram illustrating the configuration of the imaging system 1 according to the first embodiment of the present invention. Referring to FIG. 3, the imaging system 1 includes a camera 2, an A / D converter 3, an image processing device 4, a D / A converter 5, and a display 6. The image processing apparatus 4 includes an input image memory 12, a calculation unit 11, correction processing memory copy table data 15, a calculation memory 13, and an output image memory 14.

  The image processing device 4 executes distortion correction processing on input image data obtained by A / D converting the image signal supplied from the camera 2. The image processing device 4 supplies output image data as the execution result to the D / A converter 5. The D / A converter 5 D / A converts the output image data and supplies it to the display 6. In the present embodiment, the components constituting the image processing device 4 (the calculation unit 11, the input image memory 12, the calculation memory 13, and the output image memory 14) may be provided in one IC chip. Good. The A / D converter 3 receives an analog image signal corresponding to the camera image, converts it into digital data, and delivers the digital input image data as a conversion result to the input image memory 12.

  The input image memory 12 is a storage device represented by an input buffer memory or the like. The input image memory 12 temporarily stores input image data output from the camera 2. The input image memory 12 of this embodiment can hold image data for at least one screen.

  The arithmetic unit 11 has a function of handling a plurality of pieces of pixel data in a single process when a plurality of pieces of pixel data having continuous pixel addresses are stored in a storage area having continuous memory addresses. A computing memory 13 is connected to the computing unit 11. The computing unit 11 performs memory copy processing from the input image memory 12 to the output image memory 14 based on the information of the correction processing memory copy table data 15.

  The arithmetic memory 13 is used as a memory copy buffer during the memory copy process. In the image processing apparatus 4 of the present embodiment, the calculation unit 11 transfers data from the input image memory 12 to the output image memory 14 based on information in the correction processing memory copy table data 15. The image processing apparatus 4 according to the present embodiment realizes distortion correction processing with a light processing load by performing memory copy processing based on the correction processing memory copy table data 15. The calculation memory 13 of the present embodiment is preferably capable of holding image data having the maximum data amount that can be handled by the calculation unit 11 in one process.

  The output image memory 14 is a storage device represented by a VRAM or the like. The output image memory 14 of this embodiment can hold image data for at least one screen. The output image memory 14 holds image data after distortion correction. The D / A converter 5 performs D / A conversion on the distortion-corrected image data (output image data) supplied from the output image memory 14 and passes it to the display 6 as an analog video signal for display. The video is displayed on the display 6 by the received analog video signal.

  The correction process memory copy table data 15 is a table data storing a memory copy method and procedure so that the memory copy process can be efficiently performed when the distortion correction process is realized by moving the pixels by the memory copy process. . The correction processing memory copy table data 15 is configured based on information of the pixel address conversion data 16. By performing memory copy using the correction processing memory copy table data 15, the processing time is reduced and the distortion correction processing capability is improved.

  FIG. 4 is a table illustrating the configuration of the pixel address conversion data 16. The correction processing memory copy table data 15 according to the present embodiment includes information equivalent to the pixel address conversion data 16 used in an image processing apparatus or the like that performs distortion correction by converting pixel addresses of an image having distortion. . As shown in FIG. 4, the pixel address conversion data 16 holds the correspondence between the pixel address before the pixel address conversion and the pixel address after the execution as data.

  FIG. 5 is a block diagram illustrating the configuration of the input image memory 12 of the imaging system 1 of this embodiment. The input image memory 12 holds the pixel address of the input image data supplied from the A / D converter 3 in a one-to-one correspondence with the memory address. The input image memory 12 stores a plurality of pixel data constituting the input image data in a predetermined storage area.

  FIG. 6 is a block diagram illustrating the configuration of the output image memory 14 of the imaging system 1 of this embodiment. The output image memory 14 holds the pixel address of the output image data supplied to the D / A converter 5 in a one-to-one correspondence with the memory address. The output image memory 14 stores a plurality of pixel data constituting the output image data in a predetermined storage area.

  FIG. 7 is a table illustrating the configuration of the correction processing memory copy table data 15 of the imaging system 1 of this embodiment. In the correction processing memory copy table data 15, processes for performing memory copying are arranged in a table. The arithmetic unit 11 performs memory copying between the input image memory 12 and the arithmetic memory 13 and between the arithmetic memory 13 and the output image memory 14 in accordance with the arrangement of the correction processing memory copy table data 15. Process.

  Referring to FIG. 7, the correction processing memory copy table data 15 includes a plurality of records. Each of the plurality of records includes a treatment order field 21, an instruction field 22, and processing target pixel information 25. In the treatment order field 21, a memory copy processing list is described. The command field 22 describes memory copy related command information.

  The processing target pixel information 25 includes a writing destination information field 23 and a reference source information field 24. The write destination information field 23 describes used copy buffer / register information or copy reference source / reference destination pixel address data information. Similarly, in the reference source information field 24, used copy buffer / register information or copy reference source / reference destination pixel address data information is described.

  In the treatment order field 21, the contents of processing performed at one time in the memory copy processing and the order of the processing are described as a memory copy processing list. In the command field 22, memory copy related command information such as Read and Write is written as memory copy related command information. Note that when the memory copy process is optimized, it is more efficient to disassemble the Copy instruction into Read and Write, and more Read and Write instructions can be seen.

  In the write destination information field 23, in the case of memory copy processing between the input image memory 12 and the operation memory 13, the operation memory that is the use copy buffer / register / FIFO is used as the use copy buffer / register information. Thirteen pieces of information are written. In the case of memory copy processing between the arithmetic memory 13 and the output image memory 14, the write destination information field 23 stores the copy reference source / reference destination pixel address data information in the output image memory 14. Address information is displayed.

  In the reference source information field 24, in the case of memory copy processing between the input image memory 12 and the arithmetic memory 13, the address information of the input image memory 12 is used as copy reference source / reference destination pixel address data information. Is written. Further, in the case of memory copy processing between the arithmetic memory 13 and the output image memory 14, the reference source information field 24 has an operation that is used copy buffer / register / FIFO as used copy buffer / register information. The information in the memory 13 is displayed.

  As described above, when a plurality of pixel data having consecutive pixel addresses are stored at successive memory addresses, the calculation unit 11 of the present embodiment can handle the plurality of pixel data in one process. It has a function.

  In the imaging system 1 of the present embodiment, if the image processing device 4 can handle pixel data for a plurality of pixels at a time, the processing speed when performing the distortion correction processing can be increased. The correction processing memory copy table data 15 of this embodiment has memory copy related command information in the command field 22. Based on the memory copy related command information, the process optimized for the function of the image processing apparatus 4 is realized.

  As described above, the image processing apparatus 4 of the present embodiment optimizes the memory copy processing procedure in accordance with the distortion correction function, and holds the correction processing memory copy table data 15. For example, the case where memory copy (pixel address conversion) is performed on a plurality of pixel addresses of pixel data at a certain pixel address is illustrated. At this time, the pixel data of the one pixel address is once written in the arithmetic memory 13 such as a register, and the memory is written to the output image memory 14 from the arithmetic memory 13 a plurality of times. When the time is short, the correction processing memory copy table data 15 is optimized so as to achieve such an operation.

  Also, it is possible to perform memory copying at a higher speed when the pixel data of the one pixel address is read from the reference image source memory 12 as a reference source a plurality of times and the memory data is written in the output image memory 14 a plurality of times. If possible, the correction processing memory copy table data 15 is optimized so as to achieve such an operation.

  Referring to FIG. 7, the plurality of records of the correction processing memory copy table data 15 include records in which a plurality of pixel addresses are written. The computing unit 11 processes a plurality of pixel data with one memory copy related command based on the correction processing memory copy table data 15. The input image memory 12 and the output image memory 14 according to the present embodiment simultaneously support data processing for a plurality of pixel data. The computing unit 11 shortens the distortion correction processing time by processing a plurality of pixel data with one memory copy related command.

  In FIG. 7, in the k-2th process, four pixel data having consecutive pixel addresses are read from the input image memory 12 so that the addresses are consecutive in the four registers as the calculation memory 13. The operation of storing is illustrated. In the next k-1th process, the pixel data held in the four registers of the arithmetic memory 13 is read out in a single process, and written in the output image memory 14 so that the pixel addresses are continuous. Is going.

  Further, as shown in FIG. 7, in the k-th processing, the k + 1-th processing, and the k + 2-th processing, the pixel data whose pixel address is (156, 2) is used as the pixel of the output image. The memory copy is performed so that the addresses are (154, 2), (155, 2), (156, 2). Further, the memory copy is performed so that the pixel data of the pixel address of the input image is (157, 2) and the pixel address of the output image is (157, 2), (158, 2). Further, the memory copy is performed so that the pixel data of the pixel address of the input image is (158, 2) and the pixel address of the output image is (159, 2), (160, 2). Further, the memory copy is performed so that the pixel data of the pixel address of the input image is (159, 2) and the pixel address of the output image is (161, 2).

  As described above, since the original data of the write data handled in the (k + 1) th process and the (k + 2) th process is the same, the register is read only once in the kth process. By writing it in the (k + 1) th process and the (k + 2) th process, the efficiency is improved so that it is not necessary to read out each time.

  FIG. 8 is a diagram illustrating the correspondence between the original image (image before correction) and the corrected image (image after correction) according to the present embodiment. As illustrated in FIG. 8, the image data with the pixel address (5, 6) of the original image is the pixel data with the pixel address (7, 8) of the corrected image. Similarly, the image data whose pixel address of the original image is (18, 5) is pixel data whose pixel address of the corrected image is (19, 7). When both the x coordinate and the y coordinate of the pixel coordinate change greatly, it becomes difficult to read out the pixel data of the continuous pixel address with only the information shown in the pixel address conversion data 16, and the pixel address conversion data Only the information indicated by 16 makes it difficult to write continuous pixel data in the output image memory 14.

  As shown in FIG. 7, the image processing apparatus 4 according to the present embodiment has the memory copy process rearranged so that the memory copy process necessary for the distortion correction process is already as continuous as possible. Thus, efficient memory access becomes possible. In addition, regarding memory copy processing with the same read address of the write source, a plurality of writes can be performed at one time, thereby reducing the total number of memory read and write processing, thereby improving the efficiency of the memory access processing.

FIG. 9 is a block diagram illustrating the operation of the image processing device 4 in the imaging system 1 of the present embodiment. FIG. 9A illustrates an operation when the correction processing memory copy table data 15 is not used for comparison. FIG. 9B shows an optimization example of the hardware function of the memory copy process.
In FIG. 9, the bus length of image data is 64 bits in data bit width, and the minimum access bit width is

This is an optimization example in the case of a hardware function of 16 bits. The bit width corresponds to the internal bit length of the calculation unit 11 and the connection bit length of the component connected to the calculation unit 11.

  As shown in FIG. 9A, when the data amount for one pixel, which is the minimum unit of an image, is 16 bits, if memory copy is performed for one pixel, the data is 64-bit wide at a time. Data is read and written, but valid data is 16 bits.

  On the other hand, as shown in FIG. 9B, when the data address is four consecutive pixels, it is possible to read / write data of 64 bits at a time, and the memory copy process Efficiency can be improved. In the memory copy process, two types of processes are required: a read process and a write process, that is, a read process from the input image memory 12 and a write process to the output image memory 14. If the data accessed every time is a continuous address, in this case, it is possible to process a maximum of four pixel data at a time.

  Compared with the prior art, the effect of adding processing instruction information so that it can actually handle a function that can handle a plurality of data in the present invention is that the memory access valid data of the present invention of FIG. It is not possible to fill all of them, and the memory access effective data rate is lowered when non-continuous access data is generated. Therefore, not only processing instruction information is added to a function that can handle data for a plurality of data, but also processing rearrangement processing is necessary for efficient processing so that it can be accessed by continuous addresses. .

  As described above, the image processing apparatus 4 according to the present embodiment has the correction processing memory copy processing table data optimized for the distortion correction system hardware, thereby realizing the optimization of the memory copy processing and the distortion correction. Processing capacity is improved.

  In the present embodiment, the correction processing memory copy table data 15 is preferably created in advance. Hereinafter, a method of creating the correction processing memory copy table data 15 will be described. In addition, there is no restriction | limiting in the preparation timing of the correction process memory copy table data 15 in this embodiment. For example, even when the correction processing memory copy table data 15 is created whenever necessary, the effects of the present application can be achieved.

  FIG. 10 is a block diagram illustrating a method for creating the correction processing memory copy table data 15. The camera image data 32 is actually photographed by the camera 2 to be corrected using a printed matter on which straight or grid-like figures are drawn as the correction image 31. The captured camera image data 32 is taken in, and the correlation between the correction image 31 and the camera image data 32 is analyzed by the external computer 33 to analyze and correct the distortion. The pixel address conversion data 16 for realizing correction by pixel movement by cutting and pasting data is created.

  In this distortion correction, it is possible to correct without knowing the optical characteristics of the camera by photographing the correction image 31 through the camera to determine the position where the image distorted by the lens should be originally located. become. Further, if correction data is created after the apparatus is attached, the deviation of the center of the lens, the attachment error, and the like are also corrected.

  The correction processor architecture information 34 includes the memory data length of the image data, the bus data length, the image memory operation specifications, the image capture timing, the image output timing, and the like from the input image memory 12 to the output image memory 14. Use hardware function information that is relevant for the copy process.

  Using this correction processor architecture information 34 and the pixel address conversion data 16, the external computer 33 performs optimization calculation for the memory copy method and order so that the memory copy process of pixel movement by the memory copy process is accelerated. Correction processing memory copy table data 15 which is a memory copy processing method optimized for hardware functions is created. This data is stored in the correction processing memory copy table data 15.

[Comparative example]
Below, the comparative example of this invention is demonstrated. In the technique of Patent Document 1 described above, although there is no specific description in the document, the image data subjected to the distortion correction process is temporarily transferred to the controller before the DA converter 106 because of the output timing to the display 103. In some cases, the data is stored in a storage area inside the memory 102. It can be said that the image distortion correction process described in Patent Document 1 is performed during the memory copy process from the memory 107 to the storage area mounted immediately before the DA converter 106.

  Memory copy processing differs in processing time between sequential access data copy at consecutive addresses and random access data copy at discontinuous addresses. Compared with the memory copy processing at continuous addresses, the memory copy processing capability decreases as it becomes discontinuous. Therefore, even if the same number of memory copy processes are performed, the longer the process is at discontinuous addresses, the longer the time required for the memory copy process and the lower the distortion correction processing capability.

  On the other hand, rearrangement of image data at discontinuous addresses is necessary for an image having a large distortion aberration (a high wide-angle ratio). Further, the more complicated correction processing such as perspective correction and enlargement correction processing is performed, the rearrangement of image data at discontinuous addresses becomes necessary. For this reason, in the prior art, the distortion correction processing capability is reduced compared to random data copying at discontinuous addresses, which takes time.

  In addition, if there is only information for pixel address conversion data, the distortion correction system hardware is a continuous address, even if it is a function that can process a plurality of pixel data at once if it is a continuous address As a result, the contents of the memory copy process cannot be changed, and the process is performed only in units of one pixel, so that an efficient process may not be performed.

  The imaging system 1 according to the present embodiment described above can process a plurality of pieces of pixel data at a time in order to solve the shortage of pixel movement processing capability while supporting digital processing using pixel address conversion data as distortion correction processing. By providing such a configuration, the pixel movement processing capability is improved.

[Second Embodiment]
Below, 2nd Embodiment of this invention is described. FIG. 11 is a block diagram illustrating the configuration of the imaging system 1 of the second embodiment. The imaging system 1 of the second embodiment constitutes a display device 41 with an image correction in which the display 6 and the image processing device 4 of the first embodiment are integrated. The display device with image correction 41 includes an image processing device 4, a display interface (C 8) 42, and a display 43. The display 43 according to the second embodiment is a display device with a digital signal interface specification instead of an analog signal interface specification.

  As shown in FIG. 11, the imaging system 1 of the second embodiment includes a camera 2 and a display device 41 with image correction. Since the other components except for the display interface (C8) 42 and the display 43 provided in the display device with image correction 41 have the same configuration and operation as in the first embodiment, detailed description thereof is omitted. .

  The VRAM that is the output image memory 14 holds image data after distortion correction. The display interface (C8) 42 delivers a display digital video signal from the VRAM to the display 43. The received video is displayed on the display 43.

  The second embodiment is different from the first embodiment in the image output interface after the distortion correction processing. If the image output interface is different, the image data transmission method, transmission timing, and transmission cycle change according to the signal interface standard. In such a case, the time required for the distortion correction processing time is different, and it may be necessary to transmit data at a faster timing. In that case, it is necessary to complete the distortion correction within the short timing within the necessary processing time. The image processing apparatus 4 according to the present embodiment has the correction processing memory copy processing table data optimized for the distortion correction processing system hardware, so that the memory copy processing can be optimized and the distortion correction processing capability can be improved. Can be planned. Therefore, the present invention can be applied to applications that require a reduction in distortion correction processing time.

  As shown in the above-described embodiments, according to the present invention, the time required for the distortion correction processing can be shortened, and the correction processing is approximately several times faster as a whole compared to the conventional correction processing device. A device can be configured. For example, a case where a hardware function capable of handling a maximum of four color data is configured and the input / output image is 16-bit color will be described below. In this case, optimization to the hardware function is performed. Can handle up to 4 parallel data, which is 4 times more effective. Actually, even if it is not possible to handle all four processes in parallel, the effective value is about several times as effective.

  Memory access optimization is achieved by rearranging the instruction order and the number of data for reading and writing instructions to perform memory copy, so that the number of reading and writing instructions can be reduced by about a few percent compared to before optimization. . Therefore, when both effects are combined, a speed increase of about 3 times can be realized.

  The imaging system 1 including the image processing apparatus 4 of the present embodiment is applicable not only to a system that implements processing using a distortion correction module, but also to a system that performs distortion correction processing using a car navigation system processor.

  In the above-described embodiment, the calculation memory 13 is described as an example. For example, when the calculation unit 11 is realized by DMA or the like, the calculation memory 13 may be a FIFO. In the correction processing memory copy table data 15 when the arithmetic memory 13 is a FIFO, the data for a plurality of pixels is read out to the FIFO that can hold the data at once, rather than performing memory copy via the FIFO in units of one pixel. The processing time is faster in the burst transfer that writes data to. For this reason, the processing content is optimized so as to achieve such processing.

  In addition, for example, in the market, the input image and the output image have a resolution higher than the screen resolution that has been processed by the pixel movement process with the conventional pixel address conversion data, the perspective correction that is a known technique, the enlargement correction process, etc. More complicated distortion correction processing is required. In such complicated distortion correction processing, distortion correction processing by a dedicated arithmetic unit is generally used. The distortion correction processing by the dedicated arithmetic unit is more expensive to implement than the pixel movement processing method using pixel address conversion data. By applying the technique of the present application to the complicated distortion correction processing, it is possible to improve the distortion correction processing capability while suppressing an increase in cost.

  The embodiment of the present invention has been specifically described above. The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention.

DESCRIPTION OF SYMBOLS 1 ... Imaging system 2 ... Camera 3 ... A / D converter 4 ... Image processing apparatus 5 ... D / A converter 6 ... Display 11 ... Calculation part 12 ... Memory for input image 13 ... Memory for calculation 14 ... Memory for output image 15 ... Correction processing memory copy table data 16 ... Pixel address conversion data 21 ... Treatment order field 22 ... Command field 23 ... Write destination information field 24 ... Reference source information field 25 ... Processing target pixel information 31 ... Correction image 32 ... Camera image Data 33 ... External computer 34 ... Correction processing device architecture information 41 ... Display device with image correction 42 ... Display interface (C8)
43 ... Display 101 ... Camera 102 ... Controller 103 ... Display 104 ... AD converter 105 ... Arithmetic unit 106 ... DA converter 107 ... Memory 108 ... ROM
109 ... Usage area 110 ... Map area

Claims (7)

An arithmetic unit that corrects image distortion included in the input image and generates an output image;
An input side memory for holding input image data obtained by AD converting an input image signal indicating the input image;
An output side memory for holding output image data corresponding to the output image;
An arithmetic memory having a storage area for storing pixel data;
Correction processing memory copy table data referred to when correcting the image distortion,
The correction processing memory copy table data is:
It has multiple records whose processing order is determined,
Each of the plurality of records is
An instruction field indicating information on an instruction executed by the arithmetic unit;
Processing target pixel information indicating information on at least one processing target pixel to be processed in one data processing in response to the command;
With
The computing unit is
An image processing apparatus that processes pixel data corresponding to the plurality of processing target pixels in one data process when a plurality of processing target pixels are indicated in the processing target pixel information. The image processing apparatus according to claim 1.
The processing target pixel information is
A reference source information field indicating reference source information holding pixel data read by the arithmetic unit;
A write destination information field indicating write destination information when the calculation unit writes pixel data;
An image processing apparatus. The image processing apparatus according to claim 2, wherein the input image data includes a plurality of input image pixel data,
The input side memory is
Each of the plurality of input image pixel data has an input side memory address corresponding to one to one,
The output image data includes a plurality of output image pixel data,
The output side memory is
An image processing apparatus having an output side memory address corresponding to each of the plurality of output image pixel data on a one-to-one basis. The image processing apparatus according to claim 3.
The computing unit is
When the command is a read command,
Read the input image pixel data by specifying the input side memory address based on the reference source information,
The read input image pixel data is written to the storage area of the arithmetic memory as transfer target pixel data based on the write destination information.
When the command is a write command,
Identify the storage area of the arithmetic memory based on the reference source information and read the transfer target pixel data,
The read processing target pixel data is output image pixel data, the output side memory address is specified based on the write destination information, and the transfer target pixel data is written to the output side memory. The image processing apparatus according to claim 4.
The computing unit is
When a plurality of pieces of reference source information are indicated in the processing target pixel information, a plurality of pieces of pixel data corresponding to the plurality of pieces of reference source information are read out in one data process,
An image processing apparatus that writes a plurality of pixel data in a region corresponding to the plurality of write destination information in one data process when a plurality of write destination information is indicated in the processing target pixel information. The image processing apparatus according to claim 5, further comprising:
A data bus having a data bus width capable of simultaneously transferring two or more pixel data,
The image processing apparatus, wherein the arithmetic unit, the input side memory, the output side memory, and the arithmetic memory are connected via the data bus. The image processing apparatus according to claim 6.
The input side memory is
It has a storage capacity to store input image pixel data for one screen,
The output side memory is
An image processing apparatus having a storage capacity for storing output image pixel data for one screen.

Images