JP7419773B2 - Image processing device - Google Patents

Description

The present invention relates to an image processing device.

Patent Document 1 describes a first-in/first-out type memory provided between an image input means or an image recording means and an image compression/decompression means, and a method for accessing the memory immediately before overflow or underflow of the memory occurs. and a control means that responds to the output of the error detection means to disable processing from the middle of the line being processed, and returns to normal processing using the end of the line as a delimiter after the error is cleared. An image compression/expansion device is disclosed.

Patent Document 2 describes an image processing system that temporarily stores input image data and sequentially transfers read image data through a buffer means that outputs the stored image data only when an output request is received. The reading device includes an overflow detection means for detecting an overflow of the buffer means, and an interpolation means for interpolating the target image data from neighboring image data, and the reader has an overflow detection means for detecting an overflow of the buffer means, and an interpolation means for interpolating the target image data from neighboring image data. An image reading device is disclosed that is characterized in that image data corresponding to data is created and supplemented by an interpolation means.

Patent Document 3 discloses that a film image of a photographic film is scanned by a CCD line sensor, the image data read from the CCD line sensor is transferred to an external image receiving device via a line buffer, and the data receiving side becomes busy and the data is If the state where image data cannot be transferred continues, the line buffer will overflow, so the overflow detection unit detects the overflow. When an overflow is detected, writing of image data to the line buffer is interrupted, and a rescan is executed. , an image reading device is disclosed in which a communication control unit detects image data that matches the image data at the time of overflow detection during rescanning, and restarts writing of image data to the line buffer from the time of this detection.

Japanese Patent Application Publication No. 06-054202 Japanese Patent Application Publication No. 2001-024886 Japanese Patent Application Publication No. 09-252391

In an image processing apparatus using a data capture function using a FIFO memory, the present invention provides an image processing apparatus that uses image data stored in a line buffer to perform image processing when an overflow occurs in the FIFO memory. However, it is an object of the present invention to provide an image processing device in which the waiting time of subsequent image processing is suppressed.

An image processing device according to a first aspect includes a processor and a control unit, and the control unit uses the image data sent from a FIFO memory that temporarily stores image data obtained by reading an image in pixel units to the image processing unit. When an overflow in the FIFO memory is detected, input of the image data to the image processing section is stopped, and the image data is stored in a line buffer that stores the image data in predetermined units. The image processing is performed using the image data that is currently available.

In the image processing device according to a second aspect, in the image processing device according to the first aspect, the control unit stops temporary storage of the image data in the FIFO memory when an overflow in the FIFO memory is detected. , to stop inputting the image data to the image processing section.

In the image processing device according to a third aspect, in the image processing device according to the second aspect, the image is read line by line, the predetermined unit is the line unit, and the control unit reads the FIFO memory. Temporary storage of the image data is stopped for one line, and after the stopping for one line is completed, the image data of the predetermined unit input to the image processing unit and the line buffer are stored. The image processing is executed using the stored image data.

In the image processing device according to a fourth aspect, in the image processing device according to the third aspect, the processor processes the image data stored in the line buffer within a stop period of the FIFO memory for one line. If the process is not completed, error handling is performed.

In the image processing device according to a fifth aspect, in the image processing device according to any one of the first to fourth aspects, the line buffer stores the image data for a plurality of lines, and the control unit The image processing is executed using the pixel values of each of the plurality of lines continuous in the scanning direction.

An image processing device according to a sixth aspect is the image processing device according to any one of the first to fifth aspects, wherein the image data processed by the image processing section is transferred to a storage device using a direct memory access method. The image processing apparatus further includes a transfer section, and the transfer method between the FIFO memory and the image processing section and between the image processing section and the transfer section is a handshake method.

According to the first aspect, in an image processing apparatus using a data capture function using a FIFO memory, overflow in the FIFO memory is reduced compared to a case where image processing is not continued using image data stored in a line buffer. Even when this occurs, it is possible to provide an image processing apparatus in which the waiting time of subsequent image processing is suppressed.

According to the second aspect, when an overflow in the FIFO memory is detected, image processing is performed by stopping temporary storage of image data in the FIFO memory, compared to directly stopping input of image data to the image processing unit. This has the effect that the input of image data to the section can also be stopped.

According to the third aspect, after the temporary storage of two or more lines of image data has been stopped in the FIFO memory, the predetermined unit of image data input to the image processing section and the image data stored in the line buffer are stored in the line buffer. Compared to the case where image processing is performed using existing image data, it is possible to suppress a decrease in the accuracy of image processing in the image processing unit.

According to the fourth aspect, compared to the case where error processing is executed when the processing of image data stored in the line buffer is not completed within a period exceeding the stop period of the FIFO memory for one line, This has the effect that it is possible to quickly shift to error processing.

According to the fifth aspect, compared to the case where the image processing unit executes image processing other than image processing using the pixel values of each of a plurality of lines continuous in the sub-scanning direction, the present configuration can be used for fringe processing, for example. It has the effect that it can be applied.

According to the sixth aspect, even if the communication between the units connected to the subsequent stage of the FIFO memory is performed using a communication method with an established procedure that takes more time, the waiting time of the subsequent image processing can be reduced. This has the effect of being able to suppress

1 is a block diagram showing an example of the configuration of an image processing device according to an embodiment. FIG. 2 is a block diagram illustrating an overflow mode of the image processing device according to the embodiment. 5 is a time chart illustrating the operation of each mode of the image processing device according to the embodiment. 3 is a flowchart showing the flow of overflow processing of the image processing apparatus according to the embodiment.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. In this embodiment, an image processing apparatus according to the present invention is used to describe a read image in which an image reading unit (scanner unit) reads and acquires an image, and performs predetermined image processing on image data transmitted to the image processing apparatus. An example of a form applied to a processing device will be described.

FIG. 1 is a block diagram showing an image processing apparatus 10 according to the present embodiment together with a scanner section 30 connected to the outside of the image processing apparatus 10. As shown in FIG. 1, the image processing device 10 includes an image processing section 11, a line buffer 12, a register 13, and FIFO (First In First Out) memories 14-1, 14-2, and 14-3 (in FIG. , "CAP FIFO" (hereinafter collectively referred to as "FIFO memory 14"), LSYNC pattern detection section 15, separation section 16 (in FIG. 1, "MUX"), input interface 17 (in FIG. 1, (denoted as "INF"), an analog-to-digital conversion circuit 18 (denoted as "A/D" in FIG. 1), a tagger section 19, a DMA (Direct Memory Access) section 21, and a control section 20. It is configured.

The input interface 17 is an interface that receives image data as analog data DA sent from the scanner section 30. Details of the input interface 17 will be described later.

The analog-to-digital conversion circuit 18 is a circuit that converts analog data DA into digital data DD. Analog image data from the scanner section 30 is converted into digital image data and sent to the subsequent separation section 16.

The separation unit 16 separates pixel data to be sent to the FIFO memory 14 and LSYNC pattern data to be sent to the LSYNC pattern detection unit 15 from the digital data DD.

The FIFO memory 14 is a memory that temporarily stores (captures) pixel data sent from the separation unit 16 in units of pixels. Each of the FIFO memories 14-1, 14-2, and 14-3 has a capacity of several pixels. In this embodiment, a configuration using three FIFO memories 14 will be described as an example, but the number of FIFO memories 14 is not particularly limited, and for example, a configuration using four or more FIFO memories may be used.

The LSYNC pattern detection section 15 is a section that detects a line synchronization pattern (LSYNC pattern) included in the digital data DD. The scanner unit 30 according to the present embodiment reads an image line by line, and inserts a synchronization signal for identifying each line as an LSYNC pattern between the image data of each line. When the LSYNC pattern detection section 15 detects an LSYNC pattern, it notifies the image processing section 11 of the detection. Note that detection of the LSYNC pattern may be notified via the control unit 20.

The tagger unit 19 is a part that assigns tags (identifiers) from 0 to 8 to input pixels.

The image processing section 11 is a section that performs predetermined image processing on image data sent from the scanner section 30. Although the content of the image processing performed by the image processing unit 11 is not particularly limited, in this embodiment, fringe processing is performed as an example. “Fringe processing” refers to processing for correcting deviations in R, G, and B scan positions that occur when the scanner unit 30 scans an image due to scanning by the scanner. The correction is performed by setting coefficients for each of R, G, and B for the pixel values of three lines of pixels that are continuous in the sub-scanning direction. A register may be provided to store this coefficient. The image processing unit 11 typically performs the fringe processing using three lines of image data.

The register 13 is a storage unit for parameters generated during image processing in the image processing unit 11, such as the above-mentioned coefficients.

The line buffer 12 is a memory that stores a plurality of lines of image data transmitted from the scanner section 30 in units of lines. In this embodiment, the capacity of the line buffer 12 is set to two lines. However, the present invention is not limited to this, and the capacity of the line buffer 12 may be for three lines or more.

The DMA unit 21 is a control unit for outputting the image data processed by the image processing device 10 to an external storage device or the like using a direct memory access method, and is composed of, for example, a DMA. The direct memory access method refers to directly transferring data between memories or between a memory and an I/O (Input/Output) device without going through a CPU.
Note that the "DMA section 21" is an example of the "transfer section" according to the present invention.

The control unit 20 is a unit that centrally controls the entire image processing apparatus 10, and is configured to include a CPU, ROM, RAM, etc. (not shown). The control unit 20 also controls overflow processing executed in the image processing device 10. Details of the overflow process will be described later. Note that the above-mentioned "CPU" is an example of a "processor" according to the present invention.

A DDR (Double Data Rate) device 22 shown in FIG. 1 is provided outside the image processing apparatus 10, and outputs output data (image data) processed by the image processing section 11 and output-controlled by the DMA section 21 to a subsequent stage. This is, for example, a DDR type memory that performs relay for transmission to the image processing circuit 23 and the like. The DDR method is a method that uses both the rising and falling edges of a clock signal that controls operations to speed up processing. The DDR device 22 may be accessed not only by the image processing apparatus 10 but also by other devices, and in that case, a waiting list occurs.

On the other hand, the scanner section 30 includes a CIS (Contact Image Sensor) 31 and an output interface 32 (denoted as "INF" in FIG. 1).

The CIS 31 is a contact type image reading device that reads an image of a document or the like placed on the scanner section 30 and outputs the image data obtained by reading as analog data DA. The output interface 32 is an interface for transmitting analog data DA sent from the CIS 31 to the input interface 17 of the image processing device 10 paired with the output interface 32. Although there are no particular limitations on the methods of the output interface 32 and the input interface 17, for example, an LVDS (Low Voltage Differential Signaling) interface or the like can be used.

Next, the overall operation of the image processing device 10 will be explained. First, the clock system is divided into a CLK1 system from the scanner section 30 to the FIFO memory 14, and a CLK2 system from the FIFO memory 14 to the DMA section 21. The CLK1 system is a clock system operated by a clock signal CLK1 generated by the scanner section 30, and image data acquired by the scanner section 30 is transmitted to the image processing device 10 at a constant speed using the clock signal CLK1. There are no restrictions on data transmission at this time, and data is transmitted in a flow-by-flow manner.

On the other hand, the CLK2 system is a clock system operated by the clock signal CLK2 generated by the image processing device 10, and the image data transmitted by the CLK1 system is captured by the FIFO memory 14, and the image data transmitted by the CLK1 system is captured by the image processing unit in the subsequent stage. 11, and the image data processed by the image processing unit 11 is further DMA-transferred to the DDR device 22. While the CLK1 system uses a flow-through method, the CLK2 system uses a handshake method to transmit image data. The "handshake method" is a method in which data is processed after establishing and synchronizing transmission procedures between circuits. While the handshake method is a method that enables more reliable data transmission, a standby state may occur if the transmitted data is congested. In this embodiment, an example will be described in which transfer from the FIFO memory 14 is performed using a handshake method; however, the present invention is not limited to this, and other transfer methods may be used.

As described above, the CLK2 system of the image processing device 10 may perform handshake transfer, and depending on the usage status of the band of the DDR device 22, a blockage may occur in the transfer by the DMA section 21. If the transfer by the DMA section 21 becomes clogged, the clog may propagate to the FIFO memory 14 as well. In this case, an error may occur in the image processing device 10 due to the overflow of the FIFO memory 14, and the image processing device 10 may be stopped and restart processing may be required. If such a situation occurs, for example, the start of processing in the subsequent image processing circuit 23 may be delayed, and waiting time may increase.

Therefore, in this embodiment, when an overflow occurs in the FIFO memory 14, input of image data to the image processing section 11 is stopped, and image processing is performed using the image data stored in the line buffer 12. We decided to continue. This suppresses the occurrence of an interruption period, so even if an overflow occurs in the FIFO memory 14 in an image processing apparatus using a data capture function using the FIFO memory 14, the waiting time for subsequent image processing is suppressed. An image processing device is provided.

Next, with reference to FIGS. 2 to 4, the content of the image processing performed by the image processing device 10 will be described in more detail. As described above, in the image processing device 10 according to the present embodiment, when an overflow occurs in the FIFO memory 14, predetermined processing different from normal image processing, that is, overflow processing is performed, and the image processing is performed. Avoiding interruptions.

Hereinafter, the mode in which normal image processing is performed in the image processing device 10 will be referred to as a "normal mode", and the mode in which overflow processing will be performed when an overflow occurs will be referred to as an "overflow mode". Detection of an overflow in the FIFO memory 14 and transition from the normal mode to the overflow mode are executed by a selector within the control unit 20.
In the normal mode, image processing in the image processing unit 11 is executed using three lines, whereas in the overflow mode, image processing is executed using two lines.

When the control section 20 detects an overflow of the FIFO memory 14 while operating in the normal mode, the control section 20 executes an overflow process according to the following procedure.
(Procedure 1) As shown in FIG. 2, the input to the image processing unit 11 is stopped, and the capture by the FIFO memory 14 is stopped until the next line.
(Step 2) Clear the FIFO memory 14 and do not accept input until the next line start timing. The start timing of the next line is determined by detecting the LSYNC pattern in the LSYNC pattern detection section 15.
(Step 3) The fringing process that is being executed on 3 lines (the current 1 line input to the image processing unit 11 and 2 lines stored in the line buffer 12) in the normal mode is saved in the line buffer 12. Execute with only 2 lines. At this time, it is necessary when transitioning from fringe processing using 3 lines (hereinafter sometimes referred to as "3-line processing") to fringe processing using 2 lines (hereinafter sometimes referred to as "2-line processing"). The parameters for fringe processing for two-line processing may be stored in the register 13. The parameters in this case are coefficients for each of R, G, and B in the fringe processing.

Through the above procedure, image processing in the image processing unit 11 can be extended to the inactive period between lines of image data sent from the scanner unit 30, so that output from the image processing device 10 can be continued. . At this time, if the processing in step 3 is not completed by the start of the next line, conventional error handling is performed.
(Step 4) Return to normal mode at the start of the next line. The start timing of the next line is determined by detecting the LSYNC pattern in the LSYNC pattern detection section 15.

The operation in each mode will be described in more detail with reference to FIG. 3. FIG. 3 is a time chart showing the operation of each mode. FIG. 3(a) is a time chart of normal mode, and FIG. 3(b) is a time chart of overflow mode in case of normal termination. FIG. 3(c) shows a time chart of the overflow mode in the case of error termination.

In the normal mode shown in FIG. 3A, LSYNC is detected by the LSYNC pattern detection unit 15 at time t1, and the processing of one line is completed at time t2 (the set number of pixels has been completed). The period T2 from time t1 to t2 is a valid period during which the n-th line is being processed, and the FIFO memory 14 is operating normally. The fringe processing at this time is 3-line processing, and the n, (n-1), and (n-2)th pixels are used. The period from time t2 to time t3 when the next LSYNC is detected (period T3) is an invalid period during which the next line of image data is waited for input. During the invalid period, the image processing unit 11 does not process anything in principle. Incidentally, the period until LSYNC is detected at time t1 is also an invalid period of period T1. When LSYNC is detected at time t3, the valid period (period T4) of the (n+1)th line starts, and the image processing unit 11 uses the (n+1), n, and (n-1)th pixels to Perform fringe processing.

Next, with reference to FIG. 3(b), overflow processing in the case of normal termination will be described. This process corresponds to the case where the process of (procedure 3) is completed before the start of the next line in the above procedure. In FIG. 3B, after an invalid period of period T5, LSYNC is detected at time t4, and the valid period of the nth line is started, but an overflow of the FIFO memory 14 occurs at time t5. Therefore, the 3-line processing ends in a period T6 from time t4 to t5, and overflow processing starts from time t5. That is, from time t5, two-line processing is executed using the (n-1) and (n-2)-th pixels without using the n-th pixel. Since this 2-line processing ends at time t6, the period T7 from time t5 to time t6 is the period of 2-line processing. That is, in period T7, processing for one line is completed and the set number of pixels is completed.

Here, the valid period in the normal mode ends at time t* (corresponding to time t2 in FIG. 3(a)). That is, in overflow processing, the valid period is extended to the invalid period in the normal mode, if necessary. Therefore, the invalid period T8 from time t6 to t7 in the overflow mode is shorter than the invalid period in the normal mode (corresponding to period T3 in FIG. 3(a)). Thereafter, 3-line processing is started from time t7 (period T9). The 3-line processing in period T9 originally uses the (n+1), n, and (n-1)th pixels (see FIG. 3(a)), but in this mode, the nth pixel is Pixels cannot be captured. Therefore, as shown in FIG. 3(b), the nth pixel is substituted (interpolated) with the (n-1)th pixel.

Next, with reference to FIG. 3(c), overflow processing in the case of error termination will be described. This process corresponds to the case where the process (procedure 3) is not completed by the start of the next line in the above procedure. In FIG. 3C, LSYNC is detected at time t8 after the invalid period T10 has elapsed, and an overflow of the FIFO 14 occurs at time t9. That is, the period T11 from time t8 to t9 is a valid period for executing the 3-line process. From time t9, the process shifts to overflow processing of two-line processing using the (n-1) and (n-2)th pixels (period T12), and LSYNC is detected at time t10. However, in this example, the 2-line processing could not be completed between time t9 and t10, that is, before the next LSYNC was detected, and the output for 1 line could not be completed in time (the set number of pixels was not completed). Met). In this case, the error stop process is started when LSYNC is detected at time t10 (period T13). In the error stop processing, for example, image processing in the image processing section 11 is stopped. Furthermore, for example, restart processing may be executed.

Next, the flow of overflow processing executed in the image processing apparatus 10 according to the present embodiment will be described with reference to the flowchart shown in FIG. 4. FIG. 4 is a flowchart showing the processing flow of the overflow processing program executed in the image processing device 10. This overflow processing program is stored in a storage unit such as a ROM (not shown) of the control unit 20 of the image processing device 10, and the CPU reads the overflow processing program from the storage unit such as the ROM, expands it to a RAM, etc., and executes it. . Further, in this embodiment, it is assumed that the scanner unit 30 has already started scanning and has started transmitting image data to the image processing device 10.

First, in step S100, reception of image data transmitted from the scanner unit 30 is started.

In step S101, it is determined whether the LSYNC pattern detection section 15 has detected LSYNC (whether a detection signal has been transmitted from the LSYNC pattern detection section 15). In step S101, the process waits until LSYNC is detected, and when LSYNC is detected, the process moves to step S102.

In step S102, the FIFO memory 14 starts capturing.

In step S103, it is determined whether an overflow has occurred in the FIFO memory 14. If the determination is affirmative, the process moves to step S106, and if the determination is negative, the process moves to step S104.

In step S104, it is determined whether processing of one line of image data has been completed. If the determination is negative, the process returns to step S103 and continues to detect overflow of the FIFO memory 14. On the other hand, if the determination is affirmative, the process moves to step S105.

In step S105, it is determined whether processing of all lines has been completed. If the determination is negative, the process returns to step S101 to continue detecting LSYNC. On the other hand, if the determination is affirmative, this overflow processing program is terminated. Here, in this embodiment, as an example, the overflow process is executed on a page-by-page basis, so all lines mean all the lines within one page. Note that in this embodiment, a mode in which the overflow process is executed on a page-by-page basis will be described as an example, but the present invention is not limited to this, and may be executed on a job (processing) basis, for example.

In step S106, in response to the fact that the FIFO memory 14 has overflowed, the process shifts to overflow mode. That is, the input to the image processing unit 11 is stopped, and the capture operation of the FIFO memory 14 is stopped.

In step S107, processing parameters for image processing in the image processing section 11 are changed.
That is, parameters such as coefficients are changed in accordance with the change from 3-line processing to 2-line processing. At this time, parameters for 2-line processing may be stored in the register 13.

In step S108, the input of the image processing unit 11 is changed to the image data stored in the line buffer.

In step S109, it is determined whether LSYNC is detected. If the determination is affirmative, the process moves to step S112, whereas if the determination is negative, the process moves to step S110.

In step S110, it is determined whether one line has ended or not. If the determination is negative, the process returns to step S109 and continues to detect LSYNC, while if the determination is positive, the process moves to step S111. An affirmative determination in step S110 corresponds to the normal termination overflow mode shown in FIG. 3(b), and a negative determination corresponds to the error termination overflow mode shown in FIG. 3(c).

In step S111, the processing in the image processing section 11 is returned to normal processing. That is, the mode is changed to return to the normal mode and perform 3-line processing. After that, the process moves to step S105.

In step S112, in response to the fact that two line processing could not be completed during one line, error processing is executed, and in subsequent step S113, restart processing is executed and the present overflow processing program is ended. Note that the restart processing in step S113 may be performed as necessary, and the present overflow processing program may be terminated simply by executing the error processing in step S112.

In the above embodiment, the line buffer 12 has a capacity for two lines. However, the present invention is not limited to this. It may also be configured to have a capacity for one line, or generally N (an integer of 3 or more) lines. When the line buffer 12 has a capacity for N lines, overflow processing can be performed in a period of (N-1) lines.

Further, in the above embodiment, fringe processing has been described as an example of image processing in the image processing apparatus 10, but the present invention is not limited to this, and may be applied to other image processing using the line buffer 12. At this time, the effects of this embodiment are more effective if image processing is performed on a plurality of lines that are continuous in the sub-scanning direction.

Further, in the embodiment described above, the line buffer 12 has been described as an example of the storage unit connected to the image processing unit 11, but the invention is not limited to this, and image data is generally stored in predetermined units (for example, page units). It may also be a storage unit that stores .

In the above embodiments, the processor refers to a processor in a broad sense, and includes a general-purpose processor (e.g., CPU: Central Processing Unit, etc.), a dedicated processor (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, etc.) FPGA: Field Programmable Gate Array, programmable logic device, etc.) Furthermore, the operations of the processor in the above embodiments may not only be performed by one processor, but also performed by a plurality of processors located at physically separate locations. Further, the order of each operation of the processor is not limited to the order described in the above embodiments, and may be changed as appropriate.

10 Image processing device 11 Image processing section 12 Line buffer 13 Registers 14, 14-1, 14-2, 14-3 FIFO memory 15 LSYNC pattern detection section 16 Separation section 17 Input interface 18 Analog-to-digital conversion circuit 19 Tagger section 20 Control section 21 DMA section 22 DDR device 23 Image processing circuit 30 Scanner section 31 CIS
32 Output interface CLK1, CLK2 Clock signal DA Analog data DD Digital data T1 to T13 Period t1 to t11 Time

Claims (6)

Equipped with a processor and a control unit,
The control unit causes the image processing unit to perform image processing using the image data sent from the FIFO memory that temporarily stores image data obtained by reading the image in pixel units, and when an overflow in the FIFO memory is detected. , stopping the input of the image data to the image processing section, and using the image data stored in a line buffer connected to the image processing section and storing the image data in predetermined units. An image processing device that continues the image processing. 2. The control unit, when detecting an overflow in the FIFO memory, stops temporary storage of the image data in the FIFO memory and stops inputting the image data to the image processing unit. image processing device. The image is read line by line,
the predetermined unit is the line unit,
The control unit stops temporary storage of the image data in the FIFO memory for one line, and
After one line of stopping is completed, the image processing is executed using the image data of the predetermined unit inputted to the image processing unit and the image data stored in the line buffer. The image processing device according to claim 2. The image processing device according to claim 3, wherein the processor executes error processing if processing of the image data stored in the line buffer is not completed within a stop period of the FIFO memory for one line. . The line buffer stores the image data for multiple lines,
The image processing device according to any one of claims 1 to 4, wherein the control unit executes the image processing using pixel values of each of the plurality of lines continuous in the sub-scanning direction. further comprising a transfer unit that transfers the image data processed by the image processing unit to a storage device using a direct memory access method,
The image processing according to any one of claims 1 to 5, wherein a transfer method between the FIFO memory and the image processing unit and between the image processing unit and the transfer unit is a handshake method. Device.