JP6295619B2 - Image processing apparatus and method, and electronic apparatus - Google Patents

Description

  The present invention relates to an image processing apparatus and method for performing predetermined image transformation processing on input image data and outputting output image data, and an electronic apparatus including the image processing apparatus.

  A wide-angle lens such as a fish-eye lens is often used in video equipment such as a video conference system and a surveillance camera because an image with a wide angle of view can be obtained with a single lens. In a wide-angle lens, the distortion rate increases toward the periphery of the lens due to the distortion characteristics of the lens, and image distortion is significant. Since the visibility of the image is poor as it is, in many cases, the input image is subjected to image processing for correcting distortion. As one means for realizing this image processing, input image coordinate values and sub-pixel values corresponding to each output pixel are input as deformation parameters, and bilinear interpolation, bicubic interpolation, etc. are performed on the designated input pixels. A method of generating an output pixel by performing an interpolation calculation process is generally used.

  In Patent Documents 1 and 2, when performing high-speed image correction processing on an input image obtained through a fisheye lens and moving object detection in a monitoring system using a fisheye lens, a portion with a high distortion rate and a portion with a low distortion rate are used. Changing the weighting factor of matching is disclosed. Thereby, a moving body detection is performed accurately.

  Japanese Patent Application Laid-Open No. H10-228561 discloses encoding of wide-angle image data having a variable magnification according to a direction, such as an image captured with a fisheye lens, by JPEC compression. Here, since compression code data that preferentially stores high-frequency component information in a specific direction that tends to be lost due to scaling can be obtained, an image with a good balance of resolution (image quality) as a whole is obtained by decompressing it. Can be played.

  However, a DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory) is often used as a frame memory that temporarily stores an input image obtained through a lens. However, if the process of reading out the input pixel corresponding to the output pixel from the DDR SDRAM is to be executed sequentially for each output pixel, the random access to the DDR SDRAM is caused especially in arbitrary image deformation, and a huge amount of processing is performed. There was a problem of requiring time.

  On the other hand, in video equipment for video conferencing or the like, it is required to shorten the frame delay from image input to image output as much as possible in order to minimize synchronization deviation between video and audio. In the image processing for distortion correction, if it is attempted to deal with the reduction of the frame delay, the timing of starting the image input, starting the image processing, and starting the output processing of the corrected image to the video display device is simplified. It is possible to adjust the time so that the picture does not break. Alternatively, a method may be considered in which the respective processes are started when the number of lines that have been written to the frame memory of the input image and the number of lines that have been corrected by the image processing circuit have reached a certain value. However, because of the distortion aberration described above, the required reference image size differs greatly in each output image area, and there is a problem that the amount of reduction in frame delay cannot be optimized (minimized).

  In the above Patent Documents 1 to 3, there is no disclosure or suggestion of a technique for reducing frame delay in image deformation processing.

  An object of the present invention is to solve the above problems and provide an image processing apparatus capable of minimizing frame delay in image conversion processing.

An image deformation circuit according to a first aspect of the present invention is an image deformation circuit that performs predetermined image deformation processing on input image data stored in an image memory to form output image data,
The output image data is divided into a plurality of output image areas, and a part of the input image data necessary to form each output image area is read as reference image data and written to the image memory, and images are output in units of output image areas. In an image deformation circuit that performs deformation processing,
A calculation circuit for calculating the number of pixels in the sub-scanning direction of the reference image in the entire input image data of the reference image data from the control information for reading the reference image data corresponding to each output image area from the image memory;
A first comparison circuit that compares the number of pixels in the sub-scanning direction of the reference image calculated by the calculation circuit with the number of pixels in the written sub-scanning direction in which the input image data has been written to the image memory;
When the first comparison circuit determines that the number of pixels in the sub-scanning direction of the reference image is larger than the number of pixels in the written sub-scanning direction, it negates a read request signal for the reference image to the image memory. And a logic circuit for activating a pending signal.

An image deformation method according to a second aspect of the present invention is an image deformation method for an image deformation circuit that performs predetermined image deformation processing on input image data stored in an image memory to form output image data. And
The image transformation circuit divides the output image data into a plurality of output image areas, reads a part of the input image data necessary to form each output image area as reference image data, writes the image data in an image memory, Perform image transformation processing in units of output image areas,
The above image transformation method is
Calculating the number of pixels in the sub-scanning direction of the reference image in the entire input image data of the reference image data from the control information for reading the reference image data corresponding to each output image area from the image memory;
A first comparison circuit comparing the calculated number of pixels in the sub-scanning direction of the reference image with the number of pixels in the written sub-scanning direction in which the input image data has been written to the image memory;
A step of activating a pending signal for negating a read request signal for a reference image to the image memory when it is determined that the number of pixels in the sub-scanning direction of the reference image is larger than the number of pixels in the written sub-scanning direction It is characterized by including.

  An electronic apparatus according to a third aspect of the present invention includes the image deformation circuit.

  According to the present invention, the pending for negating the read request signal of the reference image to the image memory when it is determined that the number of pixels in the sub-scanning direction of the reference image is larger than the number of pixels in the written sub-scanning direction. Activate the signal. Therefore, the frame delay can be made as small as possible in the image conversion process.

1 is a block diagram illustrating a configuration of an image processing apparatus including an image deformation circuit 10 according to Embodiment 1 of the present invention. It is a block diagram which shows the structure of the image deformation circuit 10 of FIG. It is a figure which shows an example of the deformation | transformation parameter used in the image deformation | transformation circuit 10 of FIG. FIG. 3 is a sequence diagram showing an operation of the image deformation circuit 10 in FIG. 2. FIG. 3 is a block diagram showing a partial configuration of the sequencer circuit 12 of FIG. 2. It is a block diagram which shows the structure of a part of sequencer circuit 12A of the image deformation circuit 10 concerning Embodiment 2 of this invention. It is a figure which shows operation | movement of the image deformation circuit 10 which concerns on Embodiment 3 of this invention. It is a block diagram which shows the structure of a part of sequencer circuit 12B of the image deformation circuit 10 which concerns on Embodiment 4 of this invention.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In addition, in each following embodiment, the same code | symbol is attached | subjected about the same component.

Embodiment 1. FIG.
FIG. 1 is a block diagram showing a configuration of an image processing apparatus including an image deformation circuit 10 according to the first embodiment of the present invention. In order to solve the above-described problems, the image processing apparatus according to the first embodiment divides an output image into a plurality of areas (hereinafter referred to as output image areas), and input images required in the respective output image areas. An image of a partial area (hereinafter referred to as a reference image) is cut out. Then, after the reference image data is once taken into an SRAM that can be read / written at high speed, an output pixel is generated by a correction calculation.

  1, the image processing apparatus according to the first embodiment includes a sensor interface circuit 1, an image signal processor (hereinafter referred to as ISP) 2, a write direct memory access controller (hereinafter referred to as write DMAC) 3, and an interconnect. Circuit 4. The image processing apparatus further includes a frame memory 5 that is a DDR SDRAM, a read direct memory access controller (hereinafter referred to as a read DMAC), and an image deformation circuit 10.

  The sensor interface circuit 1 receives input image data from, for example, a CCD sensor such as a camera, converts it into RAW data that is input image data in a predetermined format, and outputs it to the ISP 2. The ISP 2 performs predetermined image processing (ISP processing) on the input RAW data, and then outputs the processed image data to the frame memory 5 via the write DMAC 3 and the interconnect circuit 4. The ISP processing mainly includes processing in units of pixels such as correction processing of an optical system such as a lens and correction of scratches caused by variations in image sensors.

  The write DMAC 3 is a circuit for temporarily writing image data after image processing by the ISP 2 into the frame memory 5. The interconnect circuit 4 operates as an arbitration circuit between circuits connected to the frame memory 5, and the frame memory 5, the image deformation circuit 10, and the read DMAC 6 are connected. The image deformation circuit 10 performs arbitrary image deformation processing such as distortion correction processing on the reference image data based on the deformation parameter and the reference image data from the interconnect circuit 4, and converts the processed image data into the interconnect. The data is output to the frame memory 5 via the circuit 4. The read DMAC 6 reads the image data after the image transformation process from the frame memory 5 via the interconnect circuit 4 and outputs it to various video display devices.

  In the image processing apparatus configured as described above, image data after ISP processing, image data after arbitrary deformation processing, and deformation parameters for arbitrary image deformation processing are stored in the frame memory 5 and processed. It is configured as follows. The image data after the ISP processing is updated every frame according to the input image data from the CCD sensor. Further, the image data after the image deformation process is updated every frame according to the image data that is the result of the image deformation process. Here, the deformation parameters for an arbitrary image conversion process are updated as necessary. If the same deformation process is performed every frame, there is no need to update.

  FIG. 2 is a block diagram showing a configuration of the image deformation circuit 10 of FIG. In FIG. 2, an image transformation circuit 10 includes a correction arithmetic circuit 11, SRAM circuits 21A, 21B, 22A, 22B, 23A, 23B, a sequencer circuit 12, an interface circuit 13, a setting register circuit 14, and a register control circuit. 15.

  The interface circuit 13 is connected to the frame memory 5 via the interconnect circuit 4 and exchanges information with the frame memory 5 regarding the deformation parameter, the reference image data, and the corrected image data. Specifically, the interface circuit 13 inputs deformation parameters and reference image data from the frame memory 5, and outputs image data after image deformation processing to the frame memory 5. The register control circuit 15 performs read / write control on the setting register circuit 14 for the correction arithmetic circuit 11 and the sequencer circuit 12 based on a control signal from the CPU 7. The sequencer circuit 12 controls the overall operation of the image deformation circuit 10. The image transformation circuit 10 is supplied with a clock signal, a reset signal, and an operation control signal from an external circuit. The correction arithmetic circuit 11 reads the deformation parameters and the reference image data from the SRAM circuits 21A, 21B, 22A, and 22B, generates output pixels, and outputs them to the SRAM circuits 23A and 23B.

In this embodiment, the SRAM circuits 21A, 21B, 22A, 22B, 23A, and 23B built in the image deformation circuit 10 are provided with SRAM circuits corresponding to the A and B surfaces of the image data. That is, it is as follows. These SRAM circuits 21A, 21B, 22A, 22B, 23A, and 23B are provided between the sequencer circuit 12 and the correction arithmetic circuit 11.
(1) The SRAM circuit 21A is an SRAM circuit (RAMPIX, A side in the drawing) that stores deformation parameters of image data on the A side.
(2) The SRAM circuit 21B is an SRAM circuit (RAMPIX, B side in the drawing) that stores deformation parameters of image data on the B side.
(3) The SRAM circuit 22A is an SRAM circuit (RAMREF, A surface in the drawing) that stores the reference image data of the A surface.
(4) The SRAM circuit 22B is an SRAM circuit (RAMREF, B side in the drawing) that stores reference image data for the B side.
(5) The SRAM circuit 23A is an SRAM circuit (RAMREV, A side in the drawing) that temporarily stores and stores the image data after the A side correction calculation.
(6) The SRAM circuit 23B is an SRAM circuit (RAMREV, B side in the drawing) that temporarily stores and stores the image data after the B side correction calculation.

  FIG. 4 is a sequence diagram showing the operation of the image deformation circuit 10 of FIG. As shown in FIG. 4, the read access to the frame memory 5 of the deformation parameter of the output image area (n = 1, 2, 3,...) Of the (n + 1) th area and the reference image data is performed on the A-side image data. Correction processing is performed on the B-side image data. Here, the correction calculation processing is performed on the n-th output image area (n = 1, 2, 3,...) Where DDR SDRAM read access has been performed first. Specifically, the correction operation for the A surface is performed when the image data after the correction for the B surface is written, the deformation parameter for the B surface is read, and the reference image data for the B surface is read. On the other hand, correction of the B surface is performed while writing the image data after correction of the A surface, reading the deformation parameters of the A surface, and reading the reference image data of the A surface. The above processing operation is to improve processing performance by allowing the frame memory 5 to be accessed without interruption.

  FIG. 3 is a diagram showing an example of deformation parameters used in the image deformation circuit 10 of FIG. As shown in FIG. 3, the transformation parameters specify reference image cutout data for cutting out reference image data from input image data, and a coordinate value and a subpixel value of a reference image for generating one pixel of output image data. It consists of reference pixel designation data. Here, the reference image cut-out data and the reference pixel designation data (group) are prepared for each area of the output image area, and processing such as arranging them in the storage circuit according to the order of the output image areas to be processed is performed. In this embodiment, the frame memory 5 for storing input / output image data is commonly used as a storage circuit for arranging parameters. Here, the input image data, the corrected image data, and the deformation parameters are arranged in the frame memory 5 with an address offset added thereto.

  The reference image cut-out data is composed of the first DDR SDRAM address of the reference image data, the number of main scanning pixels and the number of sub-scanning pixels of the reference image data. Here, the coordinate values in the reference pixel designation data are expressed by address values (reference pixel designation addresses) of the SRAM circuits (RAMREF) 22A and 22B that store the reference image data. The reference pixel designation data further includes a subpixel value (X subpixel, Y subpixel) for designating the reference pixel in detail, and multiplication of the data after the correction calculation, although detailed description is omitted in this specification. It is composed of blending coefficients for

  In the present embodiment, the reference pixel designation data is normally given to each output pixel. The present invention is not limited to this, and one reference pixel designation data may be given to the output 16 pixels and restored by linear interpolation so as to correspond to each output pixel within the image deformation circuit 10.

  FIG. 5 is a block diagram showing a partial configuration of the sequencer circuit 12 of FIG. The circuit in FIG. 5 is a circuit for generating a reference image read request signal Srr to the frame memory 5. The generation circuit includes registers 41 to 44 (storage circuit), a necessary sub-scanning direction pixel number calculation circuit 31, a comparison circuit 32, and an AND gate 33.

The necessary sub-scanning direction pixel number calculation circuit 31
(1) The first DDR SDRAM address Atr of the reference image data corresponding to each output image area stored in the register 42 and obtained from the reference image cut-out data;
(2) The number Nsr of sub-scanning direction pixels of the reference image data corresponding to each output image area stored in the register 43 and obtained from the reference image cut-out data;
(3) The number of pixels in the sub-scanning direction necessary for the entire input image data of the reference image corresponding to each output image area from the number Nmi of the main scanning direction pixels of the input image data stored in the register 44 and obtained by register setting or the like (Required number of pixels in the sub-scanning direction) Nsc is calculated. Specifically, the necessary sub-scanning direction pixel number calculation circuit 31 calculates the following equation (1).

[Equation 1]
Nsc = (Atr / Nmi) + Nsr (1)

  Further, the number Nsw of pixels written in the sub-scanning direction of the input image data to the frame memory 5 is stored in the input data register 41 from the write DMAC 3 via the interconnect circuit 4 and then output to the comparison circuit 32. The comparison circuit 32 compares the calculated required number of sub-scanning direction pixels Nsc with the written number of sub-scanning direction pixels Nsw. If Nsc> Nsw, the reference circuit data read request command signal to the frame memory 5 is output. The pending signal Spen for negating is activated. The pending signal Spen is input to the inverting input terminal of the AND gate 33. The reference image read request signal Srrm is input to another input terminal of the AND gate 33, and the reference image read request signal Srr is output from the output terminal of the AND gate 33 to the frame memory 5. Thus, the comparison result in the comparison circuit 32 is used for sequence control of the image deformation circuit 10. Here, the number Nsw of pixels in the written sub-scanning direction is counted up, and the image deformation process is temporarily stopped until the state of Nsc> Nsw is resolved.

  With this configuration, it is possible to simultaneously start the processing of the write DMAC 3 in the previous stage of the image deformation circuit 10 and the processing of the image deformation circuit 10. Further, if the reference image data required in the output image area in the middle is not written to the frame memory 5, the processing of the image deformation circuit 10 is temporarily stopped, so that the frame delay can be performed without failure of the image deformation processing. Can be made as small as possible.

Embodiment 2. FIG.
FIG. 6 is a block diagram showing a partial configuration of the sequencer circuit 12A of the image deformation circuit 10 according to the second embodiment of the present invention. Compared to the sequencer circuit 12 according to the first embodiment, the sequencer circuit 12A according to the second embodiment further includes a register 45, a comparison circuit 34, and an AND gate 35.

  In FIG. 6, the register 45 stores the number Nsi of pixels in the sub-scanning direction of the input image data (entire) obtained by register setting or the like, and then outputs it to the comparison circuit 34. The comparison circuit 34 compares the written sub-scanning direction pixel number Nsw from the register 41 with the sub-scanning direction pixel number Nsi of the input image data. If Nsw ≧ Nsi, the AND gate 35 performs the pending signal Spen in FIG. Is made inactive. Here, the output signal from the comparison circuit 34 is input to the inverting input terminal of the AND gate 35, and the output signal from the comparison circuit 32 is input to another input terminal of the AND gate 35. The AND gate 35 generates a pending signal Spen and outputs it to the inverting input terminal of the AND gate 33.

  With such a configuration, even when the necessary sub-scanning direction pixel number Nsc calculated by the necessary sub-scanning direction pixel number calculating circuit 31 exceeds the sub-scanning direction pixel number Nsi of the entire input image data. Thus, the frame delay can be made as small as possible without failure of the image deformation circuit.

Embodiment 3. FIG.
FIG. 7 is a diagram showing the operation of the image deformation circuit 10 according to the third embodiment of the present invention. In addition to the configuration of the image deformation circuit 10 according to the first or second embodiment, the image deformation circuit 10 according to the third embodiment includes a write destination of each output image area when the corrected image data is written to the frame memory 5. The first DDR SDRAM address Atw is determined by the following equation (2).

[Equation 2]
Atw = (offset address of the entire corrected image data)
+ (Number of main scanning pixels in the output image area) x (number of current areas -1)
+ (Number of pixels in the output image area) x (number of current areas / number of areas in the main scanning direction in the output image data) (2)

  With such a configuration, the write destination start DDR SDRAM address Atw of each output image area is calculated using the expression (2) based on the information of the output image area. Can be sequentially processed from an output image area having a small value.

  FIG. 7 shows the operation of processing in order from the output image area having the smallest number of pixels in the required sub-scanning direction in the configuration of the third embodiment. Here, the numbers in the figure indicate the order of processing of the output image area.

  By adopting such a configuration, it is possible to sequentially process from an output image area having a small number of pixels in the required sub-scanning direction, so that the frame delay can be further reduced as compared with the first or second embodiment. Is possible.

Embodiment 4 FIG.
FIG. 8 is a block diagram showing a partial configuration of the sequencer circuit 12B of the image deformation circuit 10 according to the fourth embodiment of the present invention. As shown in FIG. 8, the sequencer circuit 12B according to the fourth embodiment is different from the sequencer circuit 12A according to the second embodiment shown in FIG.
(1) Instead of one input data register 41, two input data registers 41A and 41B are provided.
(2) A selection circuit 36 is provided after the input data registers 41A and 41B and before the comparison circuits 32 and 34.

  In FIG. 8, the input data registers 41A and 41B store the written sub-scanning direction pixel numbers Nsw1 and Nsw2, respectively, and then output them to the selection circuit 36. The selection circuit 36 selects a smaller value from the written sub-scanning direction pixel numbers Nsw1 and Nsw2 and outputs the selected value to the comparison circuits 32 and 34 as a written sub-scanning direction pixel number Nsw.

  With this configuration, for example, even when image deformation processing is performed on a stereo image input from two cameras, the image deformation circuit 10 is matched with the light DMAC 3 attached to the camera with the slower progress. Can be controlled. Therefore, the frame delay can be made as small as possible without failure of the image conversion process.

  In the fourth embodiment described above, after the two written sub-scanning direction pixel numbers Nsw1 and Nsw2 are stored, the selection circuit 36 selects the sub-scanning direction pixel number having a smaller value. The present invention is not limited to this, and after storing a plurality of written sub-scanning direction pixel numbers, the selection circuit 36 may select a sub-scanning direction pixel number having a smaller value.

  In the above embodiment, the frame memory 5 is used. However, the present invention is not limited to this, and a predetermined image memory may be used.

  In the above embodiment, the AND gates 33 and 35 are provided. However, the present invention is not limited to this, and a predetermined logic circuit for generating the reference image read request signal may be provided.

  The above-described embodiment can be widely applied to electronic devices such as various video devices using a wide-angle lens such as a fisheye lens such as a digital camera, a video conference system, and a surveillance camera. In the electronic device, the frame delay from image input to image output can be reduced.

Summary of embodiments.
An image deformation circuit according to a first aspect of the present invention is an image deformation circuit that performs predetermined image deformation processing on input image data stored in an image memory to form output image data. The image transformation circuit divides the output image data into a plurality of output image areas, reads a part of the input image data necessary for forming each output image area as reference image data, writes the image data in an image memory, Image deformation processing is performed in units of output image areas. The image deformation circuit includes a calculation circuit, a first comparison circuit, and a logic circuit. The calculation circuit calculates the number of pixels in the sub-scanning direction of the reference image in the entire input image data of the reference image data from the control information for reading the reference image data corresponding to each output image area from the image memory. The first comparison circuit compares the number of pixels in the sub-scanning direction of the reference image calculated by the calculation circuit with the number of pixels in the written sub-scanning direction in which the input image data has been written in the image memory. When the first comparison circuit determines that the number of pixels in the sub-scanning direction of the reference image is larger than the number of pixels in the written sub-scanning direction, the logic circuit reads the reference image read request signal to the image memory. Activate the pending signal to negate.

  An image deformation circuit according to a second aspect of the present invention is the image deformation circuit according to the first aspect, wherein the number of pixels in the sub-scanning direction of the entire input image data is compared with the number of pixels in the written sub-scanning direction. A comparison circuit is further provided. The logic circuit deactivates the pending signal when the second comparison circuit determines that the number of pixels in the written sub-scanning direction is equal to or greater than the number of pixels in the sub-scanning direction of the entire input image data. .

The image deformation circuit according to a third aspect of the present invention is the image deformation circuit according to the first or second aspect, wherein the output image area is written when the image data after the image deformation process is written into the image memory. The first head address is determined by the following equation.
Start address of the write destination of each output image area = (offset address of the entire corrected image data)
+ (Number of main scanning pixels of output image area) x (number of current areas -1)
+ (Number of pixels in the output image area) x (current area number / number of areas in the main scanning direction in the output image data)
Then, the image data after the image transformation process is written into the image memory using the determined start address of the write destination of each output image area.

An image deformation circuit according to a fourth aspect of the present invention is the image deformation circuit according to the second aspect, further comprising a storage circuit and a selection circuit. The storage circuit stores a plurality of written sub-scanning direction pixel numbers in which the input image data has been written to the image memory. The selection circuit selects a minimum value from the stored plurality of written sub-scanning direction pixel numbers and outputs the selected value as the written sub-scanning direction pixel number to the first and second comparison circuits.

An image deformation method according to a fifth aspect of the present invention is an image deformation method for an image deformation circuit that performs predetermined image deformation processing on input image data stored in an image memory to form output image data. . The image transformation circuit divides the output image data into a plurality of output image areas, reads a part of the input image data necessary to form each output image area as reference image data, writes the image data in an image memory, Image deformation processing is performed in units of output image areas. The image deformation method includes the following steps. From the control information for reading out the reference image data corresponding to each output image area from the image memory, the number of reference image sub-scanning pixels in the entire input image data of the reference image data is calculated. The first comparison circuit compares the calculated number of pixels in the sub-scanning direction of the reference image with the number of pixels in the written sub-scanning direction in which the input image data has been written to the image memory. When it is determined that the number of pixels in the sub-scanning direction of the reference image is larger than the number of pixels in the written sub-scanning direction, a pending signal for negating a read request signal for the reference image to the image memory is activated.

The image deformation method according to a sixth aspect of the present invention is the image deformation method according to the fifth aspect, wherein the second comparison circuit determines the number of pixels in the sub-scanning direction of the entire input image data as the written sub-scanning direction. Comparing with the number of pixels. Further, the image transformation method further includes a step of deactivating the pending signal when the number of pixels in the written sub-scanning direction is determined to be equal to or greater than the number of pixels in the sub-scanning direction of the entire input image data. Including.

The image deformation method according to a seventh aspect of the present invention is the image deformation method according to the fifth or sixth aspect, wherein each of the output image areas when the image data after the image deformation process is written into the image memory. Is determined by the following equation.
Start address of the write destination of each output image area = (offset address of the entire corrected image data)
+ (Number of main scanning pixels of output image area) x (number of current areas -1)
+ (Number of pixels in the output image area) x (current area number / number of areas in the main scanning direction in the output image data)
Then, the image data after the image transformation process is written into the image memory using the determined start address of the write destination of each output image area.

An image deformation method according to an eighth aspect of the present invention includes the step of storing a plurality of written sub-scanning direction pixel numbers in which the input image data has been written to the image memory in the image deformation method according to the sixth aspect. . Further, the image transformation method selects a minimum value from the stored plurality of written sub-scanning direction pixel numbers as the written sub-scanning direction pixel number to the first and second comparison circuits. A step of outputting.

  An electronic apparatus according to a ninth aspect of the present invention includes the image deformation circuit according to the first to fourth aspects.

  According to the first and fifth aspects, it is possible to simultaneously start the processing of the write DMAC at the preceding stage of the image deformation circuit and the processing of the image deformation circuit. In addition, if the reference image data required by the image transformation circuit is not written to the image memory, the processing of the arbitrary image transformation circuit is temporarily stopped, so that the frame delay can be made as small as possible without failure of the image conversion processing. It becomes possible to do.

  According to the second and sixth aspects, in the first and fifth aspects, even when the calculation result of the number of pixels in the sub-scanning direction exceeds the number of pixels in the sub-scanning direction of the entire input image data. Thus, the frame delay can be made as small as possible without failure of the image conversion process.

  According to the third and seventh aspects, in the first, second, fifth, and sixth aspects, it is possible to process sequentially from the output image area in which the value of the number of pixels in the sub-scanning direction is small. Therefore, the frame delay can be further reduced as compared with the first, second, fifth, and sixth aspects.

  According to the fourth and eighth aspects, even when image transformation processing is performed on input images captured from a plurality of cameras, such as stereo images, the light DMAC associated with the camera with the slower progress is used. In addition, the image deformation circuit can be controlled. Therefore, the frame delay can be made as small as possible without failure of the image conversion process.

1 ... Sensor interface circuit,
2. Image signal processor (ISP),
3. Write direct memory access controller (write DMAC)
4 ... Interconnect circuit,
5 ... Frame memory,
6 ... Read Direct Memory Access Controller (Read DMAC)
7 ... CPU,
10: Image transformation circuit,
11 ... correction arithmetic circuit,
12, 12A, 12B ... sequencer circuit 13 ... interface circuit,
14: Setting register circuit,
15: Register control circuit,
21A, 21B, 22A, 22B, 23A. 23B ... SRAM circuit,
31 ... Necessary sub-scanning direction pixel number calculating circuit,
32. Comparison circuit,
33 ... Andgate,
34. Comparison circuit,
35 ... Andgate,
36 ... selection circuit,
41, 41A, 41B, 42 to 45... Registers.

Japanese Patent No. 4243767 Japanese Patent No. 4268206 Japanese Patent No. 4097130

Claims (9)

An image deformation circuit that performs predetermined image deformation processing on input image data stored in an image memory to form output image data,
The output image data is divided into a plurality of output image areas, and a part of the input image data necessary to form each output image area is read as reference image data and written to the image memory, and images are output in units of output image areas. In an image deformation circuit that performs deformation processing,
A calculation circuit for calculating the number of pixels in the sub-scanning direction of the reference image in the entire input image data of the reference image data from the control information for reading the reference image data corresponding to each output image area from the image memory;
A first comparison circuit that compares the number of pixels in the sub-scanning direction of the reference image calculated by the calculation circuit with the number of pixels in the written sub-scanning direction in which the input image data has been written to the image memory;
When the first comparison circuit determines that the number of pixels in the sub-scanning direction of the reference image is larger than the number of pixels in the written sub-scanning direction, it negates a read request signal for the reference image to the image memory. An image transformation circuit comprising a logic circuit for activating a pending signal. A second comparison circuit that compares the number of pixels in the sub-scanning direction of the entire input image data with the number of pixels in the written sub-scanning direction;
The logic circuit deactivates the pending signal when the second comparison circuit determines that the number of pixels in the written sub-scanning direction is equal to or greater than the number of pixels in the sub-scanning direction of the entire input image data. The image deformation circuit according to claim 1. When writing the image data after the image transformation process to the image memory, the start address of the write destination of each output image area is determined by the following equation:
Start address of the write destination of each output image area = (offset address of the entire corrected image data)
+ (Number of main scanning pixels in the output image area) x (number of current areas -1)
+ (Number of pixels in the output image area) x (current area number / number of areas in the main scanning direction in the output image data)
3. The image transformation circuit according to claim 1, wherein the image data after the image transformation processing is written into the image memory using the determined start address of each output image area. A storage circuit for storing a plurality of written sub-scanning direction pixel numbers in which the input image data has been written to the image memory;
A selection circuit that selects a minimum value from the stored plurality of written sub-scanning direction pixel numbers and outputs the selected value to the first and second comparison circuits as the written sub-scanning direction pixel number; 3. The image deformation circuit according to claim 2, wherein An image deformation method for an image deformation circuit for performing predetermined image deformation processing on input image data stored in an image memory to form output image data,
The image transformation circuit divides the output image data into a plurality of output image areas, reads a part of the input image data necessary to form each output image area as reference image data, writes the image data in an image memory, Perform image transformation processing in units of output image areas,
The above image transformation method is
Calculating the number of pixels in the sub-scanning direction of the reference image in the entire input image data of the reference image data from the control information for reading the reference image data corresponding to each output image area from the image memory;
A first comparison circuit comparing the calculated number of pixels in the sub-scanning direction of the reference image with the number of pixels in the written sub-scanning direction in which the input image data has been written to the image memory;
A step of activating a pending signal for negating a read request signal for a reference image to the image memory when it is determined that the number of pixels in the sub-scanning direction of the reference image is larger than the number of pixels in the written sub-scanning direction An image transformation method characterized by comprising: A second comparison circuit comparing the number of pixels in the sub-scanning direction of the entire input image data with the number of pixels in the written sub-scanning direction;
And a step of deactivating the pending signal when it is determined that the number of pixels in the written sub-scanning direction is equal to or greater than the number of pixels in the sub-scanning direction of the entire input image data. 6. The image deformation method according to 5. When writing the image data after the image transformation process to the image memory, the start address of the write destination of each output image area is determined by the following equation:
Start address of the write destination of each output image area = (offset address of the entire corrected image data)
+ (Number of main scanning pixels of output image area) x (number of current areas -1)
+ (Number of pixels in the output image area) x (current area number / number of areas in the main scanning direction in the output image data)
7. The image transformation method according to claim 5, further comprising a step of writing the image data after the image transformation process into the image memory using the determined start address of each output image area. . Storing a plurality of written sub-scanning direction pixel numbers in which the input image data has been written to the image memory;
A step of selecting a minimum value from the stored plurality of written sub-scanning direction pixel numbers and outputting the selected value as the written sub-scanning direction pixel number to the first and second comparison circuits. The image transformation method according to claim 6 .   An electronic apparatus comprising the image deformation circuit according to claim 1.

Images