JP5511577B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly, to an improvement of an image processing apparatus provided with image storage means composed of three or more frame buffers.

  In general, a moving image is composed of a large number of frame images, and an image processing apparatus that inputs and outputs moving images is provided with a frame buffer for temporarily storing frame images. In such an image processing apparatus, for example, when the input cycle is shorter than the output cycle or when the input cycle is not synchronized with each other, a phenomenon called tearing occurs in which the frame image being read is updated and a disturbed image is read. It is known.

  For example, when a captured moving image is displayed in almost real time (preview display), the captured image is repeatedly written to the frame buffer and the display image is repeatedly read from the frame buffer. At this time, when writing and reading are performed on the same frame buffer, a frame image in which only a part is updated may be read.

  Conventionally, a method for avoiding such tearing has been proposed (for example, Patent Document 1). The image processing apparatus described in Patent Document 1 includes three frame buffers and prevents tearing by temporarily stopping writing to the frame buffer when a frame drop occurs. However, this method has a problem that the latest frame image cannot be read out immediately after returning from the frame drop because tearing is prevented by stopping writing to the frame buffer when the frame drop occurs. It was.

  For example, there may be a case where a frame drop is intentionally generated and a still image is temporarily displayed when a preview of a captured moving image is displayed. In such an imaging apparatus, the latest frame image cannot be displayed immediately after returning from the frame drop, and the latest frame image is displayed after the old frame image is displayed once, which is discontinuous and unnatural. There was a problem that display was performed.

JP2007-298760

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing apparatus capable of preventing tearing and displaying the latest frame image when returning from a dropped frame. .

An image processing apparatus according to a first aspect of the present invention comprises an image storage means composed of three or more frame buffers each capable of storing one frame image, an image writing means for periodically writing the frame image to the image storage means, Image reading means for periodically reading the frame image from the image storage means, previous write destination information indicating the frame buffer that has been written last , and writing completed immediately before writing related to the previous write destination information Based on the previous write destination information and the access destination storage means for holding the previous write destination information indicating the frame buffer and the read destination information indicating the frame buffer being read, the next read destination is obtained. Based on the read destination selection means for selecting the frame buffer and updating the read destination information, the previous write destination information and the read destination information And a writing destination selection means for selecting the frame buffer to be the next writing destination, and the image writing means writes the frame image to the frame buffer selected by the writing destination selection means. the image reading means, to read out the frame image from the frame buffer to the read destination selecting means has selected, the write destination selection means compares the last write destination information and the read destination information, If they match, select the frame buffer excluding the frame buffer indicated by the previous write destination information and the previous write destination information, and if they do not match, the previous write destination information and The frame buffer excluding the frame buffer indicated by the read destination information is selected .

  With such a configuration, a frame buffer to be the next read destination is selected based on the previous write destination information, and a frame buffer to be the next write destination is selected based on the previous write destination information and the read destination information. You can choose. For this reason, it is possible to prevent tearing from occurring because the writing destination of the image writing means and the reading destination of the image reading means match. In particular, by preventing the occurrence of tearing without stopping writing when a frame drop occurs, it is possible to prevent an old frame image from being read when returning from a frame drop.

Also, with such a configuration, the latest frame image can always be read except during a period when frame dropping occurs.

Further, with such a configuration, tearing can be prevented from occurring using the three frame buffers, and the latest frame image can be read when returning from a dropped frame. That is, it can be realized without increasing the cost.

In the image processing apparatus according to the second aspect of the invention, in addition to the above configuration, the image reading unit reads the frame image without synchronizing with the writing of the frame image by the image writing unit, and selects the writing destination. The means reads the read destination information after a predetermined delay time has elapsed after the update of the previous write destination information, and the delay time is read by the read destination selection means. And it is comprised so that it may become longer than the time which update of the said reading destination information.
An image processing apparatus according to a third aspect of the present invention comprises an image storage means composed of three or more frame buffers each capable of storing one frame image, an image writing means for periodically writing the frame image to the image storage means, Holds image reading means for periodically reading out the frame image from the image storage means, previous write destination information indicating the frame buffer that has been written last, and read destination information indicating the frame buffer being read. Based on the previous write destination information, a read destination selection means for selecting the frame buffer to be the next read destination and updating the read destination information, the previous write destination information, Write destination selection means for selecting the frame buffer to be the next write destination based on the read destination information, and the image writing means The frame image is written into the frame buffer selected by the selection unit, the image reading unit reads the frame image from the frame buffer selected by the reading destination selection unit, and the image reading unit The frame image is read out without being synchronized with the writing of the frame image by the image writing unit, and the writing destination selecting unit is configured to update the previous writing destination information after a predetermined delay time has elapsed. The reading destination information is read, and the delay time is configured to be longer than the time required for reading the previous writing destination information and updating the reading destination information by the reading destination selection means.

  With such a configuration, the writing of the frame image by the image writing unit and the reading of the frame image by the image reading unit are asynchronous with each other, and the writing destination selection unit and the reading destination selection unit operate asynchronously with each other. Even in this case, tearing can be prevented, and old frame images can be prevented from being read when returning from dropped frames.

In addition to the above configuration, the image processing apparatus according to a fourth aspect of the present invention periodically acquires a frame image with a blank period interposed between the frame images, and generates a synchronization signal indicating the output timing of the frame image. Image acquisition means for periodically writing the frame image acquired by the image acquisition means to the image storage means, and the write destination selection means for outputting output timing of the frame image. The previous write destination information is updated in synchronization with the above, and the read destination information is read out in synchronization with the output start timing of the frame image. With such a configuration, the image processing apparatus can be realized with a simple configuration by using the blank period of the image acquisition means, and the cost can be suppressed.

In addition to the above configuration, the image processing apparatus according to the fifth aspect of the present invention is configured such that one of the writing period and the reading period of the frame image is not an integral multiple of the other. The present invention is particularly suitable for such an image processing apparatus.

  In the image processing apparatus according to the present invention, the reading destination selection unit selects a frame buffer to be the next reading destination based on the previous writing destination information, and the writing destination selection unit selects the previous writing destination information and the reading. Based on the destination information, the frame buffer to be the next writing destination is selected. For this reason, it is possible to prevent tearing from occurring because the writing destination of the image writing means and the reading destination of the image reading means match. In particular, tearing can be prevented without stopping writing when a frame drop occurs, and an old frame image can be prevented from being read when returning from a frame drop.

1 is a block diagram showing an example of the configuration of an image processing apparatus according to the present invention, and an imaging apparatus 100 is shown as an example of an image processing apparatus. FIG. 2 is a block diagram illustrating an example of a detailed configuration of an image processing unit 12 in FIG. 1. It is the flowchart which showed an example of the reading process of a frame image. It is the flowchart which showed an example of the write-in process of a frame image. 5 is a chart showing an example of a writing destination determination method by a writing destination selection unit 211; It is explanatory drawing which showed an example of the operation state of the image process part 12 at the time of a frame drop | offset in time series. It is explanatory drawing which showed the operation state when the frame drop generate | occur | produces in the conventional image processing apparatus in time series. It is explanatory drawing which showed the mode when the writing destination and reading destination of a frame image were changed simultaneously. It is explanatory drawing which showed operation | movement when the change of a reading destination was simultaneous with the completion | finish timing of a frame image. The operation in the case where the reading destination is changed at the same time as the start timing of the frame image is shown. 3 is a timing chart showing an example of the operation of the image processing unit 12 in FIG. 2. 12 is a timing chart following FIG. 11. It is explanatory drawing which showed the operation state in case a imaging | photography period is shorter than a display period in time series.

  FIG. 1 is a block diagram illustrating a configuration example of an image processing apparatus according to the present invention, and an imaging apparatus 100 is illustrated as an example of the image processing apparatus. The imaging apparatus 100 includes a display control unit 10, a camera unit 11, an image processing unit 12, and a display unit 13, and can display a captured moving image in real time while capturing a moving image.

  The camera unit 11 is an image pickup device including image acquisition means for acquiring moving image data by shooting, for example, a CCD (Charge Coupled Device), and the camera synchronization signal Csync and the shot image data Cdata are set for each predetermined shooting cycle. Is output. The captured image data Cdata is moving image data captured by the camera unit 11 and includes a large number of frame images generated in time series. The camera synchronization signal Csync is a signal indicating the output start timing and output end timing of each frame image.

  The display unit 13 is a display device that includes a display means for displaying a moving image, for example, an LCD (Liquid Crystal Display) panel, and outputs a display synchronization signal Dsync for each predetermined display cycle. Data Ddata is input. The display image data Ddata is moving image data displayed by the display unit 13 and includes a large number of frame images displayed in time series. The display synchronization signal Dsync is a signal indicating the input start timing of each frame image.

  The image processing unit 12 is an image processing unit that generates display image data Ddata based on the captured image data Cdata. That is, a frame image synchronized with the camera synchronization signal Csync is acquired from the camera unit 11, and a frame image synchronized with the display synchronization signal Dsync is generated and output to the display unit 13. The image processing unit 12 includes a plurality of frame buffers for temporarily storing frame images, writes the frame image input from the camera unit 11 to one of the frame buffers, and each time this writing is completed, Output CE. Further, the frame buffer for reading the frame image is switched based on the display switching request DR.

  The display control unit 10 is a display control unit that generates a display switching request DR based on the write end interrupt CE from the image processing unit 12, for example, a microprocessor. The display switching request DR is a signal for requesting an update of the frame image output from the image processing unit 12 to the display unit 13, and when the display switching request DR is not input, the image processing unit 12 outputs the same frame image. Keep doing. That is, in the imaging apparatus 100, in addition to the frame dropping that occurs inevitably, an intentional frame dropping can be generated by software control.

  Here, frame dropping is a phenomenon in which some frame images are missing from moving image data, and usually occurs when the shooting cycle is shorter than the display cycle. Here, it means a phenomenon in which there exists a frame image that is input to the image processing unit 12 but not output from the image processing unit 12. The image processing unit 12 can intentionally generate a frame drop using the display switching request DR even if the shooting period is equal to or longer than the display period.

  FIG. 2 is a block diagram showing an example of a detailed configuration of the image processing unit 12 of FIG. The image processing unit 12 includes an input control unit 21, an image storage unit 22, an output control unit 23, and an access destination storage unit 24.

<Image storage unit 22>
The image storage unit 22 includes three frame buffers 22a to 22c, and each frame buffer 22a to 22c can store one frame image. The frame image is written by the input control unit 21 and read by the output control unit 23.

<Access destination storage unit 24>
The access destination storage unit 24 is a storage unit that holds the frame buffers 22a to 22c selected as access destinations for the input control unit 21 and the output control unit 23, and includes the write destination information WS, the previous write destination information WS1, and the previous time. Write destination information WS2 and read destination information RS are stored. The write destination information WS is data indicating a frame buffer (hereinafter, write destination) in which a frame image is being written. The previous write destination information WS1 is data indicating the write destination information WS before the update, that is, the frame buffer (hereinafter referred to as the previous write destination) in which the writing of the frame image is finished last. The previous write destination information WS2 is data indicating the previous write destination information WS1 before update, that is, the frame buffer (hereinafter referred to as the previous write destination) for which writing has been completed immediately before writing related to the write destination information WS. . The read destination information RS indicates a frame buffer (hereinafter referred to as a read destination) that is reading a frame image.

<Input control unit 21>
The input control unit 21 is an input control unit of the image storage unit 22 that writes the frame image input from the camera unit 11 to the image storage unit 22, and includes a writing destination selection unit 211 and an image writing unit 212. The input control unit 21 selects any one of the frame buffers 22a to 22c as a frame image writing destination, and writes the frame image to the selected frame buffer.

  The write destination selection unit 211 selects a frame buffer as a write destination for each frame image. The selection of the frame buffer is started based on the camera synchronization signal Csync, and the frame buffer to be selected is determined based on the previous write destination information WS1, the previous and second write destination information WS2, and the read destination information RS. Specifically, the previous writing destination and the reading destination are compared, and if the two do not match, the frame buffer excluding the previous writing destination and the reading destination is selected as a new writing destination. On the other hand, if they match, the frame buffer excluding the previous writing destination and the previous writing destination is selected as a new writing destination.

  Further, the write destination selection unit 211 updates the write destination information WS based on the selection result, and also updates the previous write destination information WS1 and the previous write destination information WS2. Furthermore, the writing destination selection unit 211 generates a writing end interrupt CE and outputs it to the display control unit 10 every time writing of one frame image is completed.

  The image writing unit 212 writes the frame image input from the camera unit 11 to the frame buffer selected as the writing destination. Note that the start and end of frame image writing are determined based on the camera synchronization signal Csync.

<Output control unit 23>
The output control unit 23 is an output control unit of the image storage unit 22 that reads a frame image from the image storage unit 22 and outputs the frame image to the display unit 13. The output control unit 23 includes a reading destination selection unit 231 and an image reading unit 232. The output control unit 23 selects any one of the frame buffers 22a to 22c as a frame image reading destination, and updates the reading destination information RS based on the selection result. Also, a frame image is read from the selected frame buffer.

  Based on the display switching request DR, the reading destination selection unit 231 selects a frame buffer that is a reading destination. The selection of the reading destination is started in synchronization with the display synchronization signal Dsync, and the frame buffer to be selected is determined based on the previous writing destination information WS1. Specifically, the previous writing destination is selected as a new reading destination. Further, the read destination information RS is updated, and the selection result is reflected in the read destination information RS.

  The image reading unit 232 reads the frame image from the frame buffer selected as the reading destination and outputs it to the display unit 13. Note that the start and end of reading of the frame image are determined based on the display synchronization signal Dsync.

  Steps S101 to S104 in FIG. 3 are a flowchart illustrating an example of a frame image reading process. This flowchart is started when the display synchronization signal Dsync indicating the start of input of the frame image is input, and is executed by the output control unit 23.

  First, it is determined whether or not there is a display switching request DR from the display control unit 10 (step S101). If the display switching request DR has not been received, the process proceeds to step S104. On the other hand, when the display switching request DR is received, the reading destination selection unit 231 selects the previous writing destination as a new reading destination (step S102). That is, the frame buffer indicated by the previous write destination information WS1 is selected as a new read destination. Next, based on the selection result of step S102, the reading destination selection unit 231 updates the reading destination information RS (step S103). That is, the content of the previous write destination information WS1 is copied to the read destination information RS.

  Next, the image reading unit 232 reads the frame image from the frame buffer that is the reading destination, and the read frame image is output to the display unit 13 (step S104). That is, if the display switching request DR has not been received in step S101, the same frame image as that in the previous display cycle is read, whereas if the display switching request DR has been received, a new frame image is displayed. Read out. When the reading of one frame image is completed, the reading process in FIG. 3 ends.

  Steps S201 to S208 in FIG. 4 are flowcharts showing an example of a frame image writing process. This flowchart is started when the camera synchronization signal Csync indicating the start of frame image output is input, and is executed by the input control unit 21.

  First, the write destination selection unit 211 compares the previous write destination and the read destination (step S201). That is, it is determined whether or not the read-out frame buffer matches the frame buffer that has finished writing last.

  If they do not match, the write destination selection unit 211 selects a frame buffer other than the previous write destination and read destination as a new write destination (steps S202 and S203). In order to avoid the occurrence of tearing, the frame buffer being read cannot be selected as a new writing destination. In addition, since the previous writing destination is the next reading destination, the previous writing destination cannot be selected as a new writing destination in order to avoid occurrence of tearing. For this reason, in this embodiment, when the two do not match, the frame buffer excluding the previous write destination and the read destination is selected as a new write destination.

  On the other hand, if the two match in step S202, the write destination selection unit 211 selects a frame buffer other than the previous write destination and the previous write destination as a new write destination (step S204). When the read destination matches the previous write destination, in order to avoid the occurrence of tearing, a destination other than the read destination (previous write destination) may be selected as the next write destination. Here, in order to select the frame buffer with the oldest completion of writing as a new writing destination, a frame buffer other than the previous writing destination and the previous writing destination is selected as a new writing destination. In this case, all the frame buffers 22a to 22c can be sequentially selected as the write destination.

  Next, based on the selection results of steps S203 and S204, the write destination selection unit 211 updates the write destination information WS (step S205). That is, the newly selected write destination is written in the write destination information WS. Thereafter, the frame image input from the camera unit 11 is written into the frame buffer selected as the writing destination by the image writing unit 212 (step S206).

  After completing the writing of the frame image, the writing destination selecting unit 211 updates the previous writing destination information WS1 and the previous writing destination information WS2 (step S207). That is, the content of the previous write destination information WS1 is copied to the write destination information WS2 two times before and the content of the write destination information WS is copied to the previous write destination information WS1. As will be described later, it is desirable to provide a time difference between the update of the write destination information WS and the previous write destination information WS1.

  When the update of the previous write destination information WS1 and the previous write destination information WS2 is completed, a write end interrupt CE is generated by the write destination selection unit 211 and output to the display control unit 10 (step S208). The writing process 4 ends.

  FIG. 5 is a chart showing an example of a writing destination determination method by the writing destination selection unit 211. The previous writing destination, the previous writing destination, the reading destination, and the next writing selected based on these writing destinations. The combination with the destination is shown. In the figure, A to C mean frame buffers 22a to 22c, respectively, and "*" means any frame buffer 22a to 22c. According to this chart, the next frame is written so that the write frame buffer excluding the previous write destination and display destination becomes the write destination, and as much as possible all the frame buffers 22a to 22c become the write destinations in order. You can see that the destination is selected.

  6A to 6F are explanatory diagrams showing an example of an operation state of the image processing unit 12 when a frame drop occurs, in a time series. The frame buffers 22a to 22c and the write destination information WS are illustrated in FIG. The contents of the previous write destination information WS1 and the read destination information RS are shown. Note that FP1 to FP6 in the figure are frame images sequentially input from the camera unit 11.

  (A) in the figure shows a state in a normal time when no frame drop occurs. Frame images FP1 and FP2 are stored in the frame buffers 22a and 22b, respectively, the read destination information RS indicates the frame buffer 22a, and the write destination information WS indicates the frame buffer 22b.

  (B) to (d) in the figure show a state in which frame dropping occurs. The reading destination is fixed to the frame buffer 22a, and the frame image FP1 is repeatedly read. On the other hand, the write destination information WS alternately points to the frame buffers 22b and 22c other than the read destination, and new frame images FP3 to FP5 are sequentially written to all writable frame buffers 22b and 22c. Yes. In the state (d), the frame buffer 22a is the read destination, the frame buffer 22c is the previous write destination, and the frame buffer 22c is the write destination.

  (E) and (f) in the figure show the state after returning from a dropped frame. In the state of (d), when the reading destination information RS is updated, it recovers from frame dropping. At this time, first, the previous write destination information WS1 is copied to the read destination information RS, and reading of the frame image FP4 from the frame buffer 22b that has become the read destination is started. (E) in the drawing shows the state at this time.

  Thereafter, the next writing destination is selected, and the writing destination information WS and the previous writing destination information WS1 are updated. As a result, the frame buffer 22a becomes the writing destination and the frame buffer 22c becomes the previous writing destination, and writing of the frame image FP6 to the frame buffer 22a is started. When the read destination information RS is updated in this state, the frame buffer 22c becomes the read destination, and the frame image FP5 is read. (F) in the figure shows the state at this time.

  In FIG. 6, the frame image FP1 is repeatedly read during the frame drop period, but immediately after returning from the frame drop, the frame image FP4 is read. That is, immediately after returning from the frame image, the latest frame image at that time is read out. As a result, reading is performed in the order of the frame images FP1, FP4, FP5, and FP6.

  FIG. 7 is an explanatory diagram showing, in chronological order, operation states when frame dropping occurs in a conventional image processing apparatus, and the conditions and description method are the same as those in FIG. In the conventional image processing apparatus, tearing is prevented from occurring by stopping the writing of the frame image while the frame dropping occurs. For this reason, the frame images FP4 and FP5 are not written to the frame buffer (see (c) and (d) in the figure). As a result, immediately after returning from the dropped frame, the frame image FP3 held in the frame buffer 22c is read (see (e) in the figure). Thereafter, the reading destination is fixed to the frame buffer 22c until the writing of the frame image FP6 to the frame buffer 22a is completed (see (f) in the figure). As a result, reading is performed in the order of the frame images FP1, FP3, and FP6.

  In FIG. 7, the point that the frame image FP1 is repeatedly read during the frame drop period is the same as in FIG. 6, but the old frame image FP3 is read immediately after returning from the frame drop. That is, immediately after returning from the frame image, the frame image PF3 is read instead of the latest frame image FP4 at that time, and then the frame image FP6 is read. That is, after returning from frame dropping, an old frame image is displayed and discontinuous moving image data is displayed.

  On the other hand, in the image processing unit 12 according to the present invention shown in FIG. 6, the frame images FP1, FP4, FP5, and FP6 are read in order, and the latest frame image is displayed after returning from the dropped frame. In addition, continuous moving image data can be displayed.

  FIG. 8 is an explanatory diagram showing a state where the writing destination and reading destination of the frame image are changed simultaneously. In the imaging apparatus 100 of FIG. 2, when the camera unit 11 and the display unit 13 are operating asynchronously, tearing occurs by simultaneously changing the frame image writing destination and reading destination in the image processing unit 12. there's a possibility that. FIG. 8 is a diagram for explaining the situation when such tearing occurs.

  In this figure, the captured image data Cdata, the previous writing destination information WS1, and the reading destination information RS are shown in time series. Symbols a to c attached to the respective frame images constituting the captured image data Cdata mean the frame buffers 22a to 22c that are the writing destinations of the frame images. Similarly, symbols a to c attached to the previous write destination information WS1 and the read destination information RS also mean the frame buffers 22a to 22c pointed to.

  When the frame image writing destination is “b”, the previous writing destination is “a”, and the reading destination is “c”, if the writing destination and the reading destination of the frame image are changed at the same time, The destination is either “a” or “c”, and the read destination after the change is either “a” or “b”. For this reason, the write destination and the read destination after the change are both “a”, and tearing may occur.

  The previous writing destination information WS1 is updated from “a” to “b” in synchronization with the writing end timing of the frame image. At this time, the read destination selection unit 231 refers to the previous write destination information WS1 to determine the next read destination, and therefore, if “a” before the change is read, the read destination information RS is updated to “a”. Is done. On the other hand, if the write destination selection unit 211 that determines the next write destination reads “b” after the change from the previous write destination information WS1, and reads “c” before the change from the read destination information RS, The previous write destination information WS1 is also updated to “a”. As a result, the write destination and the read destination are both the same frame buffer 22a, and tearing occurs.

  9 and 10 are explanatory diagrams of a method for preventing the occurrence of tearing by using the blank period of the captured image data Cdata. Here, it is assumed that the captured image data Cdata alternately repeats a data period TD in which one frame image is output and a fixed blank period TB in which no frame image is output. Using this blank period TB, by providing a time lag between the update timing of the previous write destination and the write destination determination timing, tearing that occurs due to the change timing of the write destination and the read destination is prevented. doing.

  Here, the previous writing destination information WS1 is updated in synchronization with the writing end timing of the frame image, while the writing destination is determined in synchronization with the writing start timing of the frame image. That is, a delay time corresponding to the blank period TB is provided between the update of the previous write destination and the update of the write destination. Due to the existence of this delay time, tearing is prevented from occurring due to the write destination and the read destination being the same frame buffer at the update timing.

  FIG. 9 shows an operation in the case where the reading destination is changed at the same time as the end timing of the frame image. When the frame image writing destination is “b”, the previous writing destination is “a”, and the reading destination is “c”, the previous writing destination and the reading destination are simultaneously changed in synchronization with the end timing of the frame image. Then, the read destination after the change is either “a” or “b”. However, the writing destination is updated in synchronization with the start timing of the next frame image. For this reason, the updated write destination is always “b”. That is, until the next writing destination is selected, the reading destination is fixed to either “a” or “b”, and is guaranteed not to be “c”. Accordingly, as in the case of FIG. 8, the new writing destination is also “a”, and tearing does not occur.

  FIG. 10 shows an operation when the change of the reading destination is simultaneous with the start timing of the frame image. In a state where the writing destination of the frame image is “b”, the previous writing destination is “a”, and the reading destination is “c”, the previous writing destination is updated in synchronization with the end timing of the frame image. When the read destination is updated in synchronization with the next start timing, the previous write destination and read destination after the update are both “b”. On the other hand, the updated write destination is “a” or “c”.

  That is, since there is a time lag corresponding to the blank period TB from the update of the previous write destination to the update of the write destination and the read destination, the previous write destination is uniquely determined when the read destination is changed. " On the other hand, since the writing destination is selected based on either “c” or “b” which is the reading destination before and after the change, the updated writing destination is “a” or “c”. Therefore, the updated read destination and write destination do not become the same frame buffer, and tearing does not occur.

  In short, the blank period TB is used as the delay time if the previous writing destination is updated in synchronization with the end timing of the frame image and the writing destination is updated in synchronization with the next start timing of the frame image. It is possible to prevent tearing from occurring due to the update timing of the reading destination.

  11 and 12 are timing charts showing an example of the operation of the image processing unit 12 in FIG. 2, and FIG. 12 is a continuation of FIG. The camera synchronization signal Csync is at a high level during the output period of the captured image data Cdata, and is at a low level during the non-output period. “A” to “c” shown in each frame image of the photographed image data Cdata mean frame buffers 22 a to 22 c serving as a writing destination of the frame image, that is, writing destination information WS.

  The writing destination information WS is updated in synchronization with the writing start timing of the captured image data Cdata, that is, the rising edge of the camera synchronization signal Csync. The previous writing destination information WS1 is updated in synchronization with the writing end timing of the captured image data Cdata, that is, the falling edge of the camera synchronization signal Csync. Similarly, the write destination information WS2 is also updated in synchronization with the write end timing of the captured image data Cdata, but is omitted in the drawing. The read destination information RS is updated in synchronization with the fall of the display switching request DR.

  The display synchronization signal Dsync is at a high level during the input period of the display image data Ddata and is at a low level during the non-input period. However, in the figure, the non-input period is omitted and only the falling timing is shown. The writing end interrupt CE is generated at the end of writing of the captured image data Cdata and is output to the display control unit 10. The display switching request DR is generated by the display control unit 10 based on the write end interrupt CE. That is, the display switching request DR rises after the writing end interrupt CE is output. Thereafter, it falls in synchronization with the display synchronization signal Dsync, and the readout destination information RS is updated in synchronization with this fall.

  When the camera synchronization signal Csync falls at time t1, the writing of the frame image to the writing destination “a” by the image writing unit 212 is completed. At this time, the write destination selection unit 211 updates the previous write destination information WS1 from “c” to “a”, the write destination information WS2 is updated to “c”, and the write end interrupt CE is generated.

  When the camera synchronization signal Csync rises at time t2, the write destination selection unit 211 selects a new write destination. Here, since the previous write destination is “a” and the read destination is “c”, “b” is selected as the new write destination, and the write destination information WS is updated.

  At time t3, the display switching request DR falls in synchronization with the display synchronization signal Dsync, and a new reading destination is selected. Here, since the previous writing destination is “a”, “a” is selected as a new reading destination, and the reading destination information RS is updated. The display switching request DR is activated by the display control unit 10 that has received the write end interrupt CE.

  At time t4, the display switching request DR falls and “a” is selected as a new reading destination. At this time, the write destination “c” is not designated as the read destination, and frame dropping has occurred.

  At times t5 and t6, the write end interrupt CE is generated. However, since the write end interrupt CE at time t6 is output before the display control request DR corresponding to the write end interrupt CE at time t5 is generated by the display control unit 10, the writing at time t5 is performed. The interruption end interrupt is discarded at time t6, and the reading destination corresponding to the writing end interruption is not switched.

  At time t7, the camera synchronization signal Csync and the display switching request DR fall simultaneously, and the previous write destination is updated from “b” to “c”. At the same time, the read destination is changed to “b” or “c”. Updated.

  At time t8, a new writing destination is selected in synchronization with the rising edge of the camera synchronization signal Csync. Since the time t8 is after the elapse of the blank period TB from the time t7, the update of the read destination information RS to “b” or “c” is confirmed at the time t8, and “c” of the previous write destination is determined. The update to is also confirmed. Therefore, “a” is selected as the new writing destination. At this time, frame dropping for two frames has occurred, but the frame buffer 22a holding the latest frame image is designated as the reading destination. At time t7, the camera synchronization signal Csync and the display switching request DR are simultaneously lowered, but tearing does not occur.

  In order to prevent tearing, the length of the blank period TB can be a problem. Specifically, the blank period TB is longer than the time required for the read destination selection unit 231 to read the previous write destination information WS1 to select a new read destination and update the read destination information RS. It needs to be a long period. In order to absorb the asynchronousness of the operation clocks of the write destination selection unit 211 and the read destination selection unit 231, there is a delay of several clocks for synchronization when referring to the mutual information. There is no problem if the blank period TB of the camera unit 11 is made longer than the sum of the operation clocks of the unit 211 and the reading destination selection unit 231 by several clocks.

  When the camera synchronization signal Csync falls at time t9, the previous write destination information WS1 and the previous write destination information WS2 are updated. As a result of this update, the writing destination is “b”, the previous writing destination is “b”, the writing destination is “c”, and the reading destination is “a”.

  At time t10, since the display switching request DR falls simultaneously with the rise of the camera synchronization signal Csync, the write destination becomes “c” or “a” at the same time as the read destination is updated to “b”. Updated to

  Since time t10 is after the elapse of the blank period TB from time t9, at time t10, the previous write destination information WS1 is changed to “b” and the write destination information WS2 is changed to “c” two times before. The update has been confirmed. On the other hand, when the write destination selection unit 211 refers to the read destination information RS to select a new write destination, either “a” before update or “b” after update is read. Therefore, “c” or “a” is selected as the new writing destination, and the writing destination information WS is updated. At this time, since the new reading destination is “b”, no tearing occurs regardless of whether the new writing destination is “c” or “a”.

  FIG. 13 is an explanatory diagram showing, in chronological order, operation states when the shooting cycle is shorter than the display cycle. Since the illustrated imaging cycle is longer than the display cycle, frame dropping inevitably occurs. However, the image processing apparatus according to the embodiment of the present invention can display the latest frame image immediately after returning from a frame drop even when a frame drop occurs.

  In addition, the illustrated imaging cycle and display cycle are not in a relationship in which one of them is an integral multiple of the other, and the change timing of the camera synchronization signal Csync may coincide with the falling timing of the display synchronization signal Dsync. However, in the image processing apparatus according to the embodiment of the present invention, tearing does not occur even when the camera unit 11 and the display unit 13 operate asynchronously with each other.

  In the above embodiment, when the write destination selection unit 211 matches the previous write destination and the read destination, the frame buffer other than the frame buffer indicated by the previous write destination information WS1 and the previous write destination information WS2 is used. An example of selecting is described. Such a configuration is a preferred embodiment of the present invention in that frame images can be sequentially written to all writable frame buffers during the occurrence of frame dropping. It is not limited only to such a structure.

  That is, when the previous writing destination and the reading destination match, any of the frame buffers other than the frame buffer indicated by the previous writing destination information WS1 may be selected as the writing destination. Even with such a configuration, it is possible to prevent an old frame image from being read immediately after returning from a dropped frame while preventing tearing. In other words, the frame buffer excluding the previous writing destination and the reading destination can be always selected as the next writing destination without comparing the previous writing destination and the reading destination.

  In the above-described embodiment, the example in which the image storage unit 22 includes the three frame buffers 22a to 22c has been described. Such a configuration is a preferred embodiment of the present invention in that the cost can be suppressed, but the present invention is not limited to the above configuration. That is, even when the image storage unit 22 includes four or more frame buffers 22a to 22c, it is possible to prevent an old frame image from being read immediately after returning from a frame drop while preventing occurrence of tearing. it can. In this case, any one of the frame buffers satisfying the above selection conditions may be selected as the write destination information WS, the previous write destination information WS1, the previous write destination information WS2, and the read destination information RS.

CE Write end interrupt Cdata Captured image data Csync Camera synchronization signal Ddata Display image data Dsync Display synchronization signal DR Display switching request FP1 to FP7 Frame image RS Read destination information TB Blank period TD Data period WS Write destination information WS1 Previous write Destination information WS2 Previous writing destination information t1 to t10 Time 10 Display control unit 11 Camera unit 12 Image processing unit 13 Display unit 21 Input control unit 211 Write destination selection unit 212 Image writing unit 22 Image storage units 22a to 22c Frame buffer 23 Output control unit 231 Reading destination selection unit 232 Image reading unit 24 Access destination storage unit 100 Imaging device

Claims (5)

Image storage means comprising three or more frame buffers each capable of storing one frame image;
Image writing means for periodically writing frame images to the image storage means;
Image reading means for periodically reading the frame image from the image storage means;
Last write destination information indicating the frame buffer last written, last write destination information indicating the frame buffer written immediately before writing related to the previous write destination information, and the reading being read Access destination storage means for holding read destination information indicating a frame buffer;
Based on the previous writing destination information, a reading destination selecting means for selecting the frame buffer to be a next reading destination and updating the reading destination information;
Write destination selection means for selecting the frame buffer to be the next write destination based on the previous write destination information and the read destination information;
The image writing means writes the frame image to the frame buffer selected by the writing destination selection means,
The image reading unit may read out the frame image from the frame buffer to the read destination selecting means has selected,
The write destination selection means compares the previous write destination information and the read destination information,
If both match, select the frame buffer excluding the frame buffer indicated by the previous write destination information and the previous write destination information,
An image processing apparatus , wherein if the two do not match, the frame buffer excluding the frame buffer indicated by the previous write destination information and the read destination information is selected . The image reading means reads the frame image without synchronizing with the writing of the frame image by the image writing means,
The write destination selection means reads the read destination information after a predetermined delay time has elapsed after the update of the previous write destination information,
The image processing apparatus according to claim 1 , wherein the delay time is longer than a time required for reading the previous write destination information and updating the read destination information by the read destination selection unit. Image storage means comprising three or more frame buffers each capable of storing one frame image;
Image writing means for periodically writing frame images to the image storage means;
Image reading means for periodically reading the frame image from the image storage means;
Access destination storage means for holding previous write destination information indicating the frame buffer that has been written last, and read destination information indicating the frame buffer being read;
Based on the previous writing destination information, a reading destination selecting means for selecting the frame buffer to be a next reading destination and updating the reading destination information;
Write destination selection means for selecting the frame buffer to be the next write destination based on the previous write destination information and the read destination information;
The image writing means writes the frame image to the frame buffer selected by the writing destination selection means,
The image reading means reads the frame image from the frame buffer selected by the reading destination selection means,
Further, the image reading means reads the frame image without synchronizing with the writing of the frame image by the image writing means,
The write destination selection means reads the read destination information after a predetermined delay time has elapsed after the update of the previous write destination information,
The image processing apparatus according to claim 1, wherein the delay time is longer than a time required for reading the previous write destination information and updating the read destination information by the read destination selection unit . The image acquisition means for periodically acquiring the frame image with a blank period interposed between the frame images and generating a synchronization signal indicating the output timing of the frame image,
The image writing means periodically writes the frame image acquired by the image acquisition means to the image storage means,
The writing destination selecting means updates the previous writing destination information in synchronization with the output end timing of the frame image and reads out the reading destination information in synchronization with the output start timing of the frame image. The image processing apparatus according to claim 2, wherein: 4. The image processing apparatus according to claim 2 , wherein one of the writing period and the reading period of the frame image is not an integral multiple of the other.

Images