JP2017220180A - Image processing system, image processing device, and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a function of taking image data from an external video apparatus and applying image processing to it in a simple configuration, in addition to a function normally disposed in a graphic device.SOLUTION: An FIFO (4) for holding imaging data input from an image sensor (3) is connected to a memory bus (8b). An FPGA (5) detects a rising edge or falling edge of a vertical synchronous signal of the image sensor (3), and, after a prescribed period determined so that the capacity of the FIFO (4) during the input of one frame of imaging data from the image sensor (3) to the FIFO (4) is kept in the state lower than an upper limit and higher than a lower limit, starts an output of the imaging data from the FIFO (4) to the memory bus (8b).SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing system that performs image processing on image data output to a memory bus.

  Conventionally, techniques for capturing image data captured by a camera and displaying the image of the image data have been proposed (Patent Documents 1 and 2).

JP 2007-293457 A (published on November 8, 2007) Japanese Patent No. 5627993 (issued on November 19, 2014)

  Many commercially available graphic devices have a configuration for performing image processing on image data read from a memory via a memory bus. In such a graphic device, when image data captured by the camera is further captured and image processing is performed, it is necessary to newly provide an input system for capturing the image data separately from the memory bus.

  Therefore, the present inventors diligently or commonly used the function normally provided in the graphic device to capture the image data captured by the camera, and as a result of intensive investigations to achieve a simple configuration, The inventors have completed the present invention.

  Although Patent Document 1 discloses a technique for performing image processing on image data read from a video memory, it does not disclose any of the above-described diversion or sharing configuration. Also, Patent Document 2 discloses an image processing technique for combining a player's image captured by an infrared camera with a graphic pattern stored in advance in a memory. The configuration according to is not disclosed at all.

  An object of the present invention is to realize an image processing system and the like having a function of fetching image data from an external video device and performing image processing in addition to a function normally provided in a graphic device with a simple configuration.

  In order to solve the above problems, an image processing system according to an aspect of the present invention includes an image processing unit that performs image processing on image data read from the memory via a memory bus connected to the memory. An image processing system for holding video data input from the camera in synchronization with a vertical synchronization signal of the camera, a holding unit connected to at least a part of the memory bus, and the holding unit to the memory bus An output control unit that controls output of video data to the camera, and the output control unit detects one rising or falling edge of the vertical synchronization signal and then outputs one frame of the video data from the camera During the input to the holding unit, the output is opened after a predetermined period of time determined to keep the capacity of the holding unit below the upper limit value and above the lower limit value. Is allowed, the image processing unit performs image processing on image data output to the memory bus from the holding portion.

  According to the above configuration, the video data input from the camera is held by the holding unit, and the video data read from the holding unit is output to the image processing unit via the memory bus.

  Since the video data is output from the holding unit to the memory bus after a predetermined period from the start of video data input to the holding unit in synchronization with the rising or falling edge of the vertical synchronization signal of the camera, At the time of output, a certain amount of video data is held in the holding unit. Therefore, when one frame of the video data is input from the camera to the holding unit, it is held from the camera by setting a predetermined period in advance so that the capacity of the holding unit is kept below the upper limit value and larger than the lower limit value. The output of the video data from the holding unit to the memory bus can be terminated in accordance with the time when the input of the video data to the unit is completed.

  Therefore, a holding unit can be connected to the memory bus that was used to read image data from the memory, and the video data of the camera can be read via the memory bus, so a simple configuration using the conventional configuration Thus, it is possible to realize a function of capturing image data from a camera and performing image processing.

  The image processing system (particularly, the image processing unit) according to an aspect of the present invention may be realized by a computer. In this case, the image processing is performed by causing the computer to operate as each unit included in the image processing system. An image processing system control program for realizing the system by a computer and a computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention. Furthermore, the image processing system according to an aspect of the present invention may be realized as an integrated circuit (IC chip). In this case, a chip including the integrated circuit is also included in the scope of the present invention.

  According to the present invention, it is possible to realize an image processing system or the like having a function of fetching image data from an external video device and performing image processing in addition to a function normally provided in a graphic device with a simple configuration.

1 is a block diagram illustrating a schematic configuration of an image processing system according to an embodiment of the present invention. It is a timing chart of DMA transfer by FPGA included in the image processing system. It is the schematic which shows the scanning period of the imaging data for 1 frame imaged with the image sensor connected to the said image processing system. It is explanatory drawing explaining an example of the change of the capacity | capacitance of FIFO contained in the said image processing system. It is a timing chart which shows the processing content of the image processing by the image processing part contained in the above-mentioned image processing system. It is a figure which shows the flip-flop with which the said FPGA is provided.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing a schematic configuration of an image processing system 1 according to an embodiment of the present invention. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals, and redundant description is omitted.

  The image processing system 1 includes an image processing unit 2 and a FIFO (First In First Out) 4 (holding unit) that holds imaging data (video data) that is a still image or a moving image captured by an image sensor 3 (camera). An FPGA (Field Programmable Gate Array) 5 (output control unit) for controlling various exchanges between the FIFO 4 and the image processing unit 2, and a CGROM (Character Generator Read Only Memory) 6. The CGROM 6 stores in advance compressed or uncompressed image data for a display target such as various sprites, backgrounds, and fonts, a compressed or uncompressed α blend value used for α blending, and the like. In FIG. 1, a display device 7 that displays image data subjected to image processing by the image processing unit 2 is also illustrated. Note that the image processing unit 2 and the FPGA 5 are also members constituting the image processing apparatus.

  The image processing unit 2 reads image data from the FIFO 4 and / or reads image data from the CGROM 6 to generate image data to be displayed on the display device 7. The image processing unit 2 includes a control unit 21, a memory interface (I / F) 22, an image data buffer 23, an imaging data buffer 24, a drawing unit 25, an output unit 26, and a video output interface (I / F). ) 27, GPIO (General Purpose Input Output) 28, and I2C (Inter Integrated Circuit) 29.

  The control unit 21 controls the entire image processing unit 2.

  The memory I / F 22 is connected to the CGROM 6 via the memory bus 8 (particularly, the first memory bus 8a that is a part of the memory bus 8). The memory I / F 22 receives the image data supplied from the CGROM 6 via the first memory bus 8 a and supplies it to the image data buffer 23. If the image data supplied from the CGROM 6 is compressed image data, it is decoded by a decoding unit (not shown) and supplied to the image data buffer 23.

  Here, the memory I / F 22 originally stores information necessary for accessing the image data stored in the CGROM 6 such as an address signal and information necessary for reading the image data such as a data signal. Via the CGROM 6.

  The memory bus 8 is a memory bus connected to a NAND flash (registered trademark) or a NOR flash of a graphic chip. The memory bus 8 has a specification for capturing data at high speed, and a high-speed camera having a high frame rate can be connected.

  The inventors pay attention to an input system in which the memory I / F 22 captures image data from the CGROM 6 via the memory bus 8 and diverts the input system to capture of image data captured by the image sensor 3. And sharing the image data stored in the CGROM 6 and the captured image data from the image sensor 3, and as a result, the present invention has been completed.

  In the following, in the present embodiment, a configuration in which the memory bus 8 and the memory I / F 22 are shared for capturing image data from the CGROM 6 and capturing image data from the image sensor 3 will be described.

  In addition to being connected to the CGROM 6, the memory I / F 22 is further connected to the FIFO 4 via the memory bus 8 (particularly, the second memory bus 8 b that is another part of the memory bus 8). . As a result, the memory I / F 22 receives the imaging data captured by the image sensor 3 and held in the FIFO 4 via the second memory bus 8b, and supplies it to the imaging data buffer 24.

  Here, as described above, the memory I / F 22 is originally configured with a circuit so that information exchange for capturing image data from the CGROM 6 can be performed with the read-only CGROM 6. is there. Therefore, the memory I / F 22 does not have a circuit configuration that assumes the exchange of various information between the FIFO 4 newly connected to the memory bus 8 and the image sensor 3 that supplies imaging data to the FIFO 4.

  Therefore, in the image processing system 1, the FPGA 5 is provided between the FIFO 4 and the memory I / F 22 so that the memory I / F 22 can capture the image data held in the FIFO 4 via the memory bus 8. Provided. The FPGA 5 controls various exchanges between the FIFO 4 and the memory I / F 22 and between the FIFO 4 and the image processing unit 2. The memory I / F 22 can receive imaging data from the FIFO 4 by transmitting and receiving various types of information by the FPGA 5, and the image processing unit 2 performs image processing on the received imaging data and reads the image data read from the CGROM 6. At the same time, it can be displayed on the display device 7.

  Returning to the description of the image processing unit 2 again, the image data buffer 23 stores the image data supplied from the memory I / F 22. The image data is written in the image data buffer 23 in units of frames of the display device 7 and read by the drawing unit 25 for drawing.

  On the other hand, the imaging data buffer 24 stores imaging data supplied from the memory I / F 22. In the present embodiment, the imaging data buffer 24 is assumed to perform triple buffering, and includes a first buffer 24a (storage unit), a second buffer 24b (storage unit), and a third buffer 24c (storage unit). It is configured. The first buffer 24a, the second buffer 24b, and the third buffer 24c are arranged in a predetermined order (for example, first buffer 24a → second buffer 24b → third buffer 24c → first buffer 24a →...) Imaging data is written from the FIFO 4 in units of frames of the sensor 3, and the imaging data is read out by the drawing unit 25 for drawing in the above-described predetermined order.

  For example, when imaging data is written from the FIFO 4 in units of frames of the image sensor 3 in the order of the first buffer 24a → second buffer 24b → third buffer 24c → first buffer 24a →. At the stage where the imaging data is written in, the image processing by the drawing unit 25 is performed on the imaging data of the previous frame written in the second buffer 24b, and the two already written in the third buffer 24c. The imaging data of the previous frame is output to the output unit 26.

  In this embodiment, the imaging data buffer 24 performs triple buffering. However, the number of buffers included in the imaging data buffer 24 is not limited, and may be two or four or more. There may be.

  The drawing unit 25 reads image data from the image data buffer 23 frame by frame, and generates drawing data obtained by performing image processing such as enlargement / reduction, deformation, and color processing on the read image data. The drawing unit 25 outputs the generated drawing data to the output unit 26. In addition, the drawing unit 25 reads the imaging data for each frame from the imaging data buffer 24, generates drawing data obtained by performing the above-described image processing on the read imaging data, and outputs the drawing data to the output unit 26.

  The output unit 26 supplies the drawing data input from the drawing unit 25 to the display device 7 via the video output I / F 27 and causes the display device 7 to display the drawing data. The output unit 26 has a frame buffer in which drawing data can be written. The frame buffer performs double buffering, and the two buffers are alternately written in units of frames, one of which draws drawing data for drawing and the other reads drawing data for display. The video output I / F 27 outputs the drawing data as digital RGB.

  The GPIO 28 and the I2C 29 are input / output interfaces for connecting the image processing unit 2 to external devices (in this embodiment, the FPGA 5 and the image sensor 3). The image processing unit 2 (in particular, the control unit 21) and the FPGA 5 are connected via the GPIO 28, and various information can be exchanged. In addition, the image processing unit 2 (particularly, the control unit 21) and the image sensor 3 can exchange various kinds of information via an SCCB (Serial Camera Control Bus) connected to the I2C 29. For example, the control unit 21 performs various settings such as a display space and a resolution for a register provided in the image sensor 3.

  The FIFO 4 holds image data captured by the image sensor 3. The FIFO 4 is connected to the memory bus 8 (particularly, the second memory bus 8b). The imaging data held in the FIFO 4 is supplied to the memory I / F 22 via the memory bus 8. The FIFO 4 is a first-in first-out memory that holds imaging data sent from the image sensor 3 and outputs the held imaging data to the memory I / F 22 in the order of sending. That is, the FIFO 4 is a first-in first-out read / write memory that is used for temporary storage of imaging data, can be accessed at high speed, and does not require address setting.

  The FPGA 5 is connected to the image sensor 3, the FIFO 4, the memory I / F 22, and the GPIO 28. The FPGA 5 mediates transmission / reception between the image sensor 3 and the FIFO 4 and the image processing unit 2. The FPGA 5 can be easily set by the user for a method of transmission / reception between the image sensor 3 and the FIFO 4 and the image processing unit 2 (particularly, the memory I / F 22).

  Specifically, the FPGA 5 instructs the FIFO 4 to directly transfer the imaging data from the FIFO 4 to the memory I / F 22 (that is, DMA (Direct Memory Access) transfer) in accordance with the instruction from the image processing unit 2. Hereinafter, the above-described DMA transfer will be described with reference to FIG.

  FIG. 2 is a timing chart of DMA transfer by the FPGA 5. FIG. 2 shows an example in which the image sensor 3 outputs 30 captured images per second. For this reason, a falling edge period or a rising edge of a vertical synchronization signal (hereinafter, referred to as “VSYNC”) of the image sensor 3. The edge period (for example, the period from time t2 to time t7) is about 1/30 second (33 ms). The input clock frequency to the image sensor 3 is 25 MHz (VGA resolution). In the following, with respect to the falling edge of VSYNC, in one falling edge period of VSYNC (for example, a period from time t1 to time t5), one frame (one frame) of imaging data is transferred from the image sensor 3 to the FIFO4. To be output.

  When VSYNC rises at time t2, output of image data captured by the image sensor 3 to the FIFO 4 is started. On the other hand, the DMA transfer of the imaging data from the FIFO 4 to the memory I / F 22 is on standby until time t4. Therefore, during the period from time t2 to time t4, the imaging data output from the image sensor 3 to the FIFO 4 is accumulated in the FIFO 4.

  At time t1, when VSYNC falls, the FPGA 5 raises VSH in synchronization with the falling edge. VSH is generated by the FPGA 5 and output to the image processing unit 2 in order to notify the image processing unit 2 of the rise or fall of VSYNC.

  During the period from time t1 to t4 (hereinafter referred to as “waiting period W”), the DMA transfer is waited for without starting, and the DMA transfer is started at time t4. The standby period W (here, 8 ms) is measured by a timer provided in the image processing unit 2, for example. The control unit 21 starts the measurement by the above-described timer by detecting the HSH at the H level, and instructs the FPGA 5 to start DMA transfer when the standby period W elapses (that is, at time t4). The subsequent transfer of the imaging data is executed by direct communication between the FIFO 4 and the memory I / F 22 controlled by the FPGA 5. Note that the measurement of the waiting period W may be performed using a counter provided in the FPGA 5.

  In the period from time t4 to time t6, the image data captured by the image sensor 3 is output to the FIFO 4, and the image data is output from the FIFO 4 to the memory I / F 22. As described above, since the FIFO 4 is a first-in first-out method, imaging data is output from the FIFO 4 in the order input from the image sensor 3 during the period from time t4 to time t6.

  After the output of imaging data from the image sensor 3 to the FIFO 4 is completed at time t5, the FIFO 4 becomes empty (lower limit) at time t6. That is, one frame of imaging data is output from the FIFO 4 to the memory I / F 22. The DMA transfer is finished as soon as possible.

  In this way, the imaging data for one frame output from the FIFO 4 to the memory I / F 22 is written into the imaging data buffer 24 (here, the first buffer 24a) by the memory I / F 22.

  At time t7, output of imaging data for the next frame to the FIFO 4 is started, and thereafter, processing similar to that described above is performed, and imaging data is written to the second buffer 24b.

  Here, a method for calculating the standby period W will be described. FIG. 3 is a schematic diagram illustrating a scan period of image data for one frame imaged by the image sensor 3. As shown in FIG. 3, in a scan period per frame (horizontal scan period × vertical scan period), an effective period for scanning an area in which imaging data is valid (horizontal effective pixel number × vertical effective pixel number); There is actually an invalid period during which imaging data is not output.

  If the input speed of the imaging data from the image sensor 3 to the FIFO 4 and the output speed from the FIFO 4 to the memory I / F 22 are substantially equal, the invalid period is set as the standby period W as it is. When the input of the imaging data for one frame from the sensor 3 to the FIFO 4 ends, the output of the imaging data for one frame from the FIFO 4 to the memory I / F 22 ends.

  On the other hand, if the data rate of output from the FIFO 4 to the memory I / F 22 is faster than the data rate of input from the image sensor 3 to the FIFO 4, input of imaging data for one frame from the image sensor 3 to the FIFO 4 is performed. It is possible for the FIFO 4 to become empty before it is finished. In such a case, an appropriate margin period is added to the above-described invalid period according to the difference between the data speed of the output from the FIFO 4 to the memory I / F 22 and the data speed of the input from the image sensor 3 to the FIFO 4. The waiting period W may be adjusted as described above.

  Of course, the data rate of input from the image sensor 3 to the FIFO 4, the capacity of the FIFO 4, and the data rate of output from the FIFO 4 to the memory I / F 22 can be adjusted, respectively. It is possible to prevent the capacity from falling below the lower limit (underflow) and the capacity of the FIFO 4 from exceeding the upper limit (overflow).

  Hereinafter, an example of calculating the standby period W (8 ms) will be described using specific numerical values. In the scan period shown in FIG. 3, the number of vertical scans is 510, the number of horizontal scans is 784, the vertical effective pixels are 480, and the horizontal effective pixels are 640. As described above, one frame period is 33 ms.

  (1) The ratio between the scanning period and the effective period is calculated.

(480 × 640) ÷ (510 × 784) = 0.6833
(2) The effective period for one frame period is calculated.
33 × 0.7683 = 25.39 [ms]
(3) The invalid period is calculated by subtracting the valid period calculated in (2) from one frame period.
33-25.39 = 7.61 [ms]
(4) The waiting period W is determined by adding the above-described margin period to the invalid period calculated in (3) above.
7.61 + 0.39 = 0.00 [ms]
Note that the margin period needs to be set within the range from the input start time to the FIFO 4 from the image sensor 3 to the output start time of imaging data in the above-described effective period. When a margin period exceeding this range is set, at the start of input of imaging data for the next frame, a part of the imaging data for the previous frame remains from the FIFO 4 and the imaging data for the previous frame is output. This is because there is a risk of being incomplete.

  Next, the end of the DMA transfer will be described with reference to FIG. FIG. 4 is an explanatory diagram illustrating an example of a change in the capacity of the FIFO 4 when DMA transfer is performed. The “upper limit value” shown in FIG. 4 means the upper limit value of the capacity of the FIFO 4. Here, a case where the data rate of output from the FIFO 4 to the memory I / F 22 is faster than the data rate of input from the image sensor 3 to the FIFO 4 will be described as an example.

  As shown in FIG. 4, when VSYNC of the image sensor 3 rises, input of imaging data from the image sensor 3 to the FIFO 4 starts. However, actually, the output starts after a delay period (a period indicated by A in FIG. 4) due to the specification of the image sensor 3 has elapsed (here, the output starts at time T2).

  Along with the input of imaging data from the image sensor 3 to the FIFO 4, the amount of imaging data stored in the FIFO 4 (hereinafter referred to as “FIFO capacity”) increases. This increase continues until time T3 (that is, until the standby period W elapses).

  At time T3, when output from the FIFO 4 to the memory I / F 22 starts, the FIFO capacity decreases accordingly. This is because the data rate of output from the FIFO 4 to the memory I / F 22 is faster than the data rate of input from the image sensor 3 to the FIFO 4. This descent continues until the input from the image sensor 3 to the FIFO 4 ends at time T4, that is, until the input of imaging data for one frame is completed.

  When the input from the image sensor 3 to the FIFO 4 is completed at time T4, only the output from the FIFO 4 to the memory I / F 22 is performed. As a result, the rate of decrease of the FIFO capacity increases, and at time T5, the FIFO capacity is empty. Become. That is, the output from the FIFO 4 to the memory I / F 22 is completed.

  Here, in order to start the DMA transfer, as described above, an instruction from the image processing unit 2 to the FIFO 4 via the FPGA 5 is required, whereas an instruction for ending the DMA transfer is unnecessary. It should be noted. This is because, after the standby time W is appropriately provided, after the input of the imaging data from the image sensor 3 to the FIFO 4 is started, the output of the imaging data for one frame from the FIFO 4 to the memory I / F 22 is completed ( That is, the FIFO capacity becomes empty only after the time T5), and the DMA transfer automatically ends at that time.

  Further, the time when the FIFO capacity becomes maximum is the time when output from the FIFO 4 to the memory I / F 22 is started (time T3). Therefore, assuming the amount of data stored in the FIFO 4 at the start of output, and using a FIFO memory having an upper limit value exceeding the data amount for the FIFO 4, the FIFO capacity is limited to the upper limit of the FIFO 4 during DMA transfer. The value is never exceeded. Therefore, it should be noted that it is not necessary to monitor the FIFO capacity of the FIFO 4 at the time of DMA transfer, and a configuration for monitoring the FIFO capacity becomes unnecessary.

  The example shown in FIG. 4 is an example of a change in the capacity of the FIFO 4 when the data rate of output from the FIFO 4 to the memory I / F 22 is faster than the data rate of input from the image sensor 3 to the FIFO 4. However, even when the data rate of the output from the FIFO 4 to the memory I / F 22 is the same as or slower than the data rate of the input from the image sensor 3 to the FIFO 4, the data rate of the input from the image sensor 3 to the FIFO 4 Needless to say, the FIFO 4 underflow and overflow can be prevented by adjusting the capacity of the FIFO 4 and the data rate of the output from the FIFO 4 to the memory I / F 22, respectively.

  When the data rate of output from the FIFO 4 to the memory I / F 22 is the same as or slower than the data rate of input from the image sensor 3 to the FIFO 4, the FIFO capacity is determined until the input from the image sensor 3 to the FIFO 4 is completed. It rises or is almost constant. For this reason, it is only necessary to adjust the FIFO capacity to be less than the upper limit value of the FIFO 4 at time T4 shown in FIG.

  As described above, when one frame of the imaging data is being input from the image sensor 3 to the FIFO 4, the capacity of the FIFO 4 is kept below the upper limit value and larger than the lower limit value.

  Finally, image processing by the image processing unit 2 will be briefly described. FIG. 5 is a timing chart showing the processing contents of image processing performed by the image processing unit 2 on the image data written in the image data buffer 23 and the imaging data written in the imaging data buffer 24. 5 shows an example in which the image processing unit 2 outputs 60 drawing data per second to the display device 7. For this reason, the vertical blanking period signal (hereinafter, “VBLANK”) of the image processing unit 2 is used. The falling edge period or the rising edge period (for example, the period from time t101 to time t102) is approximately 1/60 second (16 ms). In the following, it is assumed that drawing data for one sheet (one frame) is output from the image processing unit 2 to the display device 7 during the falling edge period of VBLANK with reference to the falling edge of VBLANK.

  When VBLANK falls at time t101, the display frame buffer of the output unit 26 is cleared (S11). The imaging data written in the imaging data buffer 24 is converted into ARGB format (S12), and the image data written in the image data buffer 23 and the imaging data written in the imaging data buffer 24 are processed. Then, address conversion is performed by generating address data indicating a display position on the display screen of the display device 7 (S13). Then, filter processing such as edge enhancement is performed on the image data written in the image data buffer 23 and the imaging data written in the imaging data buffer 24 (S14).

  Here, the control unit 21 of the image processing unit 2 confirms the level of VSH output from the FPGA 5 at each of the processing completion points in steps S11 to S14 (C11 to C14). Instructs the FPGA 5 to return the VSH to the L level. Note that after the processing in step S14 is completed, the H level of the VSH is polled until the H level VSH can be detected (C14).

  As described above, the VSH is generated by the FPGA 5 and output to the image processing unit 2 in order to notify the image processing unit 2 of the rise or fall of VSYNC of the image sensor 3. The reason for this is that the image processing unit 2 does not have a mechanism for directly receiving VSYNC from the image sensor 3, and the FPGA 5 outputs VSH to the image processing unit 2 as an alternative to VSYNC.

  In addition, the structure which produces | generates VSH is realizable with the flip-flop shown in FIG. 6, for example. The VSYNC of the image sensor 3 is input to the input terminal S, and an instruction (here, an H level signal) from the control unit 21 based on the VSH H level detection is input to the input terminal R. In the example of FIG. 6, when H is set to the input terminal S due to the rise of VSYNC, H is set to the output terminal Q, and the output terminal Q maintains H until H is set to the input terminal R. It will be.

  Returning to FIG. 5 again, the image processing by the image processing unit 2 will be described.

  In the period from time t101 to time t103, the drawing unit 25 outputs the drawing data on which the image processing of steps S11 to S14 described above has been performed to the output unit 26. The output unit 26 performs double buffering as described above. In the period from time t101 to time t103, the drawing data subjected to the above-described image processing is written in one of the two buffers.

  However, the falling edge period or rising edge period of VSYNC of the image sensor 3 is about 1/30 second (33 ms), while the falling edge period or rising edge period of VBLANK of the image processing unit 2 is about 1/60 second. (16 ms). That is, the period required for the image processing unit 2 to draw one frame is approximately half of the period required for the image sensor 3 to capture one frame.

  For this reason, the image processing unit 2 switches between a buffer in which drawing data is written and a buffer in which drawing data is read out on condition that VBLANK of the display device 7 is detected twice.

  For example, when the first VBLANK is detected at time t102 and the second VBLANK is detected at time t103, drawing data subjected to image processing in the period from time t103 to time t105 is written in the other buffer. Will be. Further, the buffer in which the drawing data subjected to the image processing is written in the period from the time t101 to the time t103 is read, and the read drawing data is displayed on the display device 7. However, in each of the period from time t101 to time t102 and the period from time t102 to time t103, the same drawing data read from the same buffer (here, the image in the period from time t101 to time t103) The processed drawing data) is displayed.

  The image processing system 1 (particularly the image processing unit 2) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or by software using a CPU (Central Processing Unit). It may be realized.

  In the latter case, the image processing system 1 includes a CPU that executes instructions of a program that is software that implements each function, and a ROM (Read Only Memory) in which the program and various data are recorded so as to be readable by a computer (or CPU). Alternatively, a storage device (these are referred to as “recording media”), a RAM (Random Access Memory) that expands the program, and the like are provided. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope shown in the claims, and the present invention is also applied to an embodiment obtained by appropriately combining technical means disclosed in the embodiment. It is included in the technical scope of the invention.

  Further, the present invention is suitable for FA devices, automobiles, image sensors, game machines and the like.

  DESCRIPTION OF SYMBOLS 1 Image processing system, 2 Image processing part, 3 Image sensor (camera), 4 FIFO (holding part), 5 FPGA (output control part), 6 CGROM (memory), 7 Display apparatus, 8 Memory bus, 8a 1st memory Bus, 8b Second memory bus, 22 Memory I / F, 23 Image data buffer, 24 Imaging data buffer, 24a First buffer (storage unit), 24b Second buffer (storage unit), 24c Third buffer (storage unit) 25 Drawing unit 26 Output unit 27 Video output I / F 28 GPIO 29 I2C

Claims (7)

An image processing system including an image processing unit that performs image processing on image data read from the memory via a memory bus connected to a memory,
A holding unit connected to at least a part of the memory bus for holding video data input from the camera in synchronization with a vertical synchronization signal of the camera;
An output control unit that controls output of video data from the holding unit to the memory bus;
The output control unit detects the rising or falling edge of the vertical synchronization signal, and then inputs one frame of the video data from the camera to the holding unit, so that the capacity of the holding unit is less than the upper limit value. And after a predetermined period of time determined to maintain a state greater than the lower limit, start the output,
The image processing system, wherein the image processing unit performs image processing on video data output from the holding unit to the memory bus. The scan period of the video data per frame is composed of a valid period for scanning an area in which the video data is valid and an invalid period in which the video data is not output.
The image processing system according to claim 1, wherein the predetermined period is longer than the invalid period. The output control unit generates a control signal indicating the rising or falling edge of the vertical synchronization signal in synchronization with the rising or falling edge of the vertical synchronization signal, and sends the generated control signal to the image processing unit. Output,
The image processing according to claim 1, wherein the image processing unit instructs the output control unit to output video data from the holding unit to the memory bus based on the control signal. system.   The image processing system according to claim 1, wherein the holding unit performs the output in the order of the input. Further comprising at least two storage units for sequentially storing video data read from the holding unit via the memory bus in a predetermined order in units of frames;
The image processing system according to claim 1, wherein the image processing unit performs image processing on the stored video data while switching the storage unit in units of frames. An image processing unit that performs image processing on image data read from the memory via a memory bus connected to the memory;
An output control unit for controlling video data output from the holding unit connected to at least a part of the memory bus to hold video data input from the camera in synchronization with a vertical synchronization signal of the camera. And
The output control unit detects the rising or falling edge of the vertical synchronization signal, and then inputs one frame of the video data from the camera to the holding unit, so that the capacity of the holding unit is less than the upper limit value. And after a predetermined period of time determined to maintain a state greater than the lower limit, start the output,
The image processing apparatus, wherein the image processing unit performs image processing on video data output from the holding unit to the memory bus. An image processing method for performing image processing on image data read from the memory via a memory bus connected to the memory,
A holding step for holding in the holding unit video data input from the camera in synchronization with a vertical synchronization signal of the camera;
A state in which the capacity of the holding unit is less than the upper limit value and larger than the lower limit value while one frame of the video data is being input from the camera to the holding unit after detecting the rising or falling edge of the vertical synchronization signal An output step of outputting the held video data to the memory bus after a predetermined period determined to maintain
And a processing step of performing image processing on the video data output to the memory bus.

Images