JP2016039517A - Image processing method, imaging device, image processing apparatus, and imaging display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To shorten the time lag from imaging to display.SOLUTION: An image processing apparatus includes an image signal generation unit for generating an image signal indicating an image to be displayed by each line of a display unit based on an imaging signal outputted from an imaging section, and outputting it to the display unit, and a timing control unit for controlling the image signal output timing of the image signal generation unit. Assuming the time from output of the imaging signal to generation of the image signal is an image processing time, and the time from frame start of the imaging signal to the frame start of the image signal is a phase difference, the timing control unit controls the image signal generation unit to output an image signal so that the frame rate of the display unit becomes the maximum frame rate that can be displayed in the display unit, when the phase difference is longer than the image processing time, and controls the image signal generation unit to output an image signal so that the frame rate of the display unit becomes the frame rate of the imaging unit, after the phase difference becomes shorter than the image processing time.SELECTED DRAWING: Figure 12

Description

  The present invention relates to an image processing method, an imaging device, an image processing device, and an imaging display device.

  In a so-called mirrorless single-lens digital camera, a liquid crystal provided on the rear surface of the housing in real time according to an image signal captured by an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. A so-called live view operation displayed on an electronic viewfinder (hereinafter referred to as EVF [Electronic View Finder]) attached to the upper part of the panel or casing can check the image of the subject (for example, Patent Document 1).

JP 2014-11729 A

  However, in this live view operation, there is a significant delay from when the subject is imaged by the image sensor to when the image is displayed on the viewfinder or the like. For this reason, it is difficult to make the camera follow the moving subject. Furthermore, when an instruction to capture a still image is given based on the displayed subject image, a timing shift occurs between the displayed subject image and the actually captured still image, and particularly fast-moving subjects. In this case, it is difficult to capture an intended still image.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to reduce the delay time from imaging to display.

  In order to solve the above problems, an image processing method according to the present invention provides a display unit capable of displaying an image at a higher frame rate than the imaging unit based on an imaging signal output by an imaging unit that images a subject. An image processing method for generating an image signal indicating an image to be displayed on each line, and outputting the generated image signal to a display unit capable of displaying an image at a frame rate higher than that of the imaging unit. The time from when the imaging signal is output until the image signal is generated is defined as the image processing time, and the time from the start of the frame of the imaging signal to the start of the frame of the image signal output to the display unit Is the phase difference, the frame rate of the imaging unit is the first frame rate, and the highest frame rate that can be displayed on the display unit is the second frame rate, the phase difference is If the recording time is longer than the image processing time, the phase difference is gradually reduced by outputting the image signal to the display unit so that the frame rate of the display unit becomes the second frame rate. However, after the image processing time has elapsed, the image signal is output to the display unit so that the frame rate of the display unit becomes the first frame rate.

According to the present invention, when the phase difference that is the time from the start of the frame of the imaging signal to the start of the frame of the image signal is longer than the image processing time, the second frame that has a higher frame rate than the imaging unit. By outputting the image signal at the rate, the phase difference is reduced by a period corresponding to the difference between the frame period of the display unit and the frame period of the imaging unit. After the phase difference becomes equal to or less than the image processing time, the image signal is synchronized with the imaging signal by setting the frame rate of the display unit to the first frame rate that is the frame rate of the imaging unit. Output.
Therefore, according to the present invention, when the phase difference is larger than the image processing time, the phase difference can be shortened step by step until the image processing time becomes less than the image processing time, and the phase difference converges to the image processing time or less. After that, since the phase difference can be maintained, the delay time from imaging to display can be shortened as compared with the case where the phase difference is not shortened step by step until the phase difference becomes equal to or shorter than the image processing time.
Thereby, for example, when there is a change in the image processing time due to a change in the image processing method, when the image processing time varies for each line, when the display unit is replaced with a different frame rate, or When replacing the imaging unit with a different frame rate, some or all of the phase difference between the imaging unit and the display unit, the frame rate of the imaging unit, and the maximum frame rate that can be displayed on the display unit change. Even in this case, the phase difference can be automatically converged to a length equal to or shorter than the image processing time.
When the image processing time varies for each line of the display unit, for example, the maximum time among the varying times may be regarded as the image processing time.

In the image processing method described above, when the phase difference is longer than the image processing time, after the image signal is generated, the image processing apparatus waits until an image indicated by the generated image signal can be displayed on the display unit. Then, the generated image signal may be output to the display unit.
According to this aspect, since the display unit waits until the image indicated by the image signal can be displayed, the image signal is output so that the frame rate of the display unit becomes the second frame rate. The phase difference of the part can be shortened step by step.

In the image processing method described above, the display unit operates in synchronization with a horizontal synchronization pulse output at a constant period, and outputs the generated image signal to the display unit in synchronization with the horizontal synchronization pulse. When the image signal generation time for generating the image signal is after the displayable time at which the image indicated by the image signal can be displayed on the display unit after the phase difference becomes equal to or less than the image processing time. The image signal may be output to the display unit in synchronization with a horizontal synchronization pulse output first after the image signal generation time.
According to this aspect, the interval from the generation of the image signal to the output can be made equal to or less than the horizontal scanning period determined by the horizontal synchronization pulse. That is, the output timing of the image signal of each line can be controlled with the accuracy of the horizontal scanning period determined by the horizontal synchronization pulse. For this reason, even when the image processing time required for generating the image signal varies from line to line, the image signal of each line is output to the display unit at a timing corresponding to the image processing time for each line. Is possible.

In the image processing method described above, the display unit operates in synchronization with a horizontal synchronization pulse, and the generated image signal is output to the display unit in synchronization with the horizontal synchronization pulse, and the phase difference is When the image signal generation time for generating the image signal is after the displayable time at which the image indicated by the image signal can be displayed on the display unit after the image processing time or less, the image signal generation time Until the output of the horizontal sync pulse is stopped, the output of the image signal is stopped, and after the generation time of the image signal, the horizontal sync pulse is output and synchronized with the output horizontal sync pulse. Then, the image signal may be output.
According to this aspect, the time length of the horizontal scanning period determined by the horizontal synchronization pulse is determined according to the timing at which the image signal of each line can be output. For this reason, even when the image processing time required for generating the image signal varies from line to line, the image signal of each line is output to the display unit at a timing corresponding to the image processing time for each line. Is possible.

In addition, the imaging apparatus according to the present invention includes each line of an imaging unit that images a subject and outputs an imaging signal, and a display unit that can display an image at a higher frame rate than the imaging unit based on the imaging signal. Generating an image signal indicating an image to be displayed, outputting the generated image signal to the display unit, and a timing control unit for controlling the timing at which the image signal generation unit outputs the image signal The time from when the imaging unit outputs the imaging signal until the image signal generation unit generates the image signal is defined as an image processing time, and from the start of the frame of the imaging signal, the display The time until the start of the frame of the image signal to be output to the unit is the phase difference, the frame rate of the imaging unit is the first frame rate, and the highest frame rate that can be displayed on the display unit Is the second frame rate, the timing control unit generates the image signal so that the frame rate of the display unit becomes the second frame rate when the phase difference is longer than the image processing time. A first timing control for gradually decreasing the phase difference by outputting the image signal to a unit; and after the phase difference becomes equal to or less than the image processing time, the frame rate of the display unit The second timing control for causing the image signal generation unit to output the image signal can be executed so as to achieve one frame rate.
According to the present invention, when the phase difference is larger than the image processing time, the phase difference can be shortened step by step until it becomes equal to or less than the image processing time. The phase difference can be maintained.

In addition, the image processing apparatus according to the present invention is configured to display an image to be displayed on each line of the display unit capable of displaying an image at a frame rate higher than that of the imaging unit based on an imaging signal output by the imaging unit that images the subject An image signal generating unit that generates an image signal indicating the image signal and outputting the generated image signal to the display unit; and a timing control unit that controls a timing at which the image signal generating unit outputs the image signal. The time from when the imaging unit outputs the imaging signal to when the image signal generation unit generates the image signal is defined as an image processing time, and is output to the display unit from the start of the frame of the imaging signal. The time until the start of the frame of the image signal is the phase difference, the frame rate of the imaging unit is the first frame rate, and the highest frame rate that can be displayed on the display unit is the second frame rate. When the phase difference is longer than the image processing time, the timing control unit sends the image signal generation unit with the image signal generation unit so that the frame rate of the display unit becomes the second frame rate. A first timing control for gradually reducing the phase difference by outputting a signal; and after the phase difference becomes equal to or less than the image processing time, the frame rate of the display unit is changed to the first frame rate. As described above, the second timing control for causing the image signal generation unit to output the image signal can be executed.
According to the present invention, when the phase difference is larger than the image processing time, the phase difference can be shortened step by step until it becomes equal to or less than the image processing time. The phase difference can be maintained.

  Moreover, the imaging display apparatus which concerns on this invention is provided with the display part mentioned above and the imaging device mentioned above, It is characterized by the above-mentioned.

1 is a block diagram illustrating a configuration of an imaging display device 1 according to a first embodiment of the present invention. It is explanatory drawing for demonstrating the relationship between the effective image sensor area | region AS and the display area AD. 3 is a timing chart for explaining the operation of the imaging display device 1. 3 is a timing chart for explaining the operation of the imaging display device 1. It is explanatory drawing for demonstrating display area AD. 3 is a timing chart for explaining the operation of the imaging display device 1. 3 is a timing chart for explaining the operation of the imaging display device 1. 4 is a block diagram illustrating a configuration of a display unit 40. FIG. 3 is a block diagram illustrating a configuration of an image processing unit 21. FIG. It is explanatory drawing for demonstrating a thinning process. It is explanatory drawing for demonstrating a distortion correction process. 3 is a timing chart for explaining the operation of the imaging display device 1. 3 is a timing chart for explaining the operation of the imaging display device 1. 3 is a timing chart for explaining the operation of the imaging display device 1. 3 is a timing chart for explaining the operation of the imaging display device 1. It is a timing chart for demonstrating operation | movement of the imaging display apparatus which concerns on contrast. It is a timing chart for demonstrating operation | movement of the imaging display apparatus which concerns on 2nd Embodiment of this invention. It is a timing chart for demonstrating operation | movement of the imaging display apparatus which concerns on 2nd Embodiment. It is a timing chart for demonstrating operation | movement of the imaging display apparatus which concerns on 2nd Embodiment.

  DESCRIPTION OF EMBODIMENTS Embodiments for carrying out the present invention will be described with reference to the drawings. However, in each figure, the size and scale of each part are appropriately changed from the actual ones. Further, since the embodiments described below are preferable specific examples of the present invention, various technically preferable limitations are attached thereto. However, the scope of the present invention is particularly limited in the following description. Unless otherwise stated, the present invention is not limited to these forms.

<A. First Embodiment>
The imaging display device 1 according to the first embodiment of the present invention will be described below.

<1. Configuration of Imaging Display Device>
FIG. 1 is a block diagram illustrating functions of the imaging display device 1.
As shown in FIG. 1, the imaging display device 1 captures an image of a subject, outputs data obtained by imaging as an imaging signal DS, and performs image processing on the imaging signal DS to generate an image signal D. An image processing circuit 100 that generates an image, a display unit 40 that displays an image according to the image signal D, an operation unit 60 for inputting a setting change, an imaging instruction, and the like of the imaging display device 1, and an imaging display device 1 and a CPU 50 for controlling the entire operation.

The imaging display device 1 is a so-called mirrorless digital camera, and displays an image related to a subject imaged by the imaging unit 10 on the display unit 40 in almost real time by an operation of the operation unit 60 by a user of the imaging display device 1. The operation in the live view mode and the operation in the imaging mode in which the image related to the subject imaged by the imaging unit 10 is stored as a still image in the still image storage memory can be selected.
The imaging unit 10, the image processing circuit 100, the operation unit 60, and the CPU 50 excluding the display unit 40 from the imaging display device 1 are examples of “imaging device”.

  The imaging unit 10 scans signals from an imaging optical system 11 that forms an image of a subject and light receiving elements (imaging devices) arranged in a matrix in a line-sequential manner to obtain an imaging signal DS corresponding to the subject image. An image sensor 12 for outputting and a timing generator 13 for outputting various timing signals to the image sensor 12 are provided.

As shown in FIG. 2A, in the effective image sensor area AS, the image sensor 12 includes a plurality of light receiving elements in the X axis direction (horizontal direction) in the QS row and the Y axis direction (vertical direction) intersecting the X axis direction. ) Are arranged in a matrix so as to form PS rows (PS lines) (PS and QS are natural numbers of 2 or more). In other words, the effective image sensor area AS is configured such that lines composed of QS light receiving elements arranged in the X-axis direction are arranged in PS rows in the Y-axis direction. Each light receiving element generates a pixel data signal Sig according to the detected light amount.
In addition, a color filter of any one of red, green, and blue is provided so as to correspond to each light receiving element on a one-to-one basis. Hereinafter, a set of one light receiving element and a color filter provided corresponding to the one light receiving element may be referred to as a pixel of the image sensor 12.

  When the imaging display device 1 operates in the imaging mode, the image sensor 12 converts all of the pixel data signal Sig output from the light receiving elements of the PS row × QS column provided in the effective image sensor area AS to the still image imaging data. Can be output as When the still image capturing data is output, the image processing unit 21 described later performs various image processing such as filter processing on the still image capturing data to generate still image data, and the generated still image The image data is stored in a memory for storing still images.

On the other hand, when the imaging display device 1 operates in the live view mode, the image sensor 12 performs a thinning process on the pixel data signal Sig output from the light receiving element of PS row × QS column, thereby obtaining data of the pixel data signal Sig. The size is reduced and output as an imaging signal DS corresponding to pixels of P rows × Q columns as shown in FIG. 2B (P is a natural number satisfying 2 ≦ P ≦ PS. Q is 2 ≦ Q) ≦ natural number satisfying QS).
Note that the image sensor 12 may include pixels in an area other than the effective image sensor area AS, but description of the pixels in the effective image sensor area AS is omitted in this specification for the sake of simplicity.

The timing generator 13 generates an imaging vertical synchronization signal SVsync, an imaging horizontal synchronization signal SHsync, and an imaging dot clock signal SCLK, and outputs the generated signals to the image sensor 12 and the image processing circuit 100.
FIG. 3 is a timing chart for explaining the imaging vertical synchronization signal SVsync, imaging horizontal synchronization signal SHsync, and imaging dot clock signal SCLK generated by the timing generator 13. The imaging vertical synchronization signal SVsync defines a vertical scanning period Fs (that is, a frame period of the imaging unit 10) for reading the pixel data signal Sig from the light receiving elements in the entire effective image sensor area AS (for the PS line) of the image sensor 12. Signal. In addition, the frame rate of the imaging unit 10, that is, “1 / Fs” may be referred to as “first frame rate”. The imaging horizontal synchronization signal SHsync is a signal that defines a horizontal scanning period Hs for reading the pixel data signal Sig from the light receiving elements for one line in the effective image sensor area AS. The imaging dot clock signal SCLK is a signal that defines the timing for reading the pixel data signal Sig from the light receiving elements for one pixel in the effective image sensor area AS.
The time length of the vertical scanning period Fs is constant (fixed length), and the time length of the horizontal scanning period Hs is also constant (fixed length). The vertical scanning period Fs includes a plurality of horizontal scanning periods Hs.

Returning to FIG.
The display unit 40 is an EVF (Electronic View Finder) for displaying an image showing a subject to be imaged and allowing the user of the imaging display device 1 to grasp the state of the subject, and an image generated by the image processing circuit 100. A liquid crystal panel 42 for displaying an image corresponding to the signal D and an EVF controller 41 for controlling the operation of the liquid crystal panel 42 are provided.

As shown in FIG. 2C, the liquid crystal panel 42 is arranged in a matrix in the display area AD so that a plurality of pixels are arranged in N columns in the X-axis direction and M rows (M lines) in the Y-axis direction. (M is a natural number satisfying 2 ≦ M ≦ P. N is a natural number satisfying 2 ≦ N ≦ Q). In other words, the display area AD is configured such that M lines of N pixels arranged in the X-axis direction are arranged in M rows in the Y-axis direction. These pixels of M rows × N columns include a pixel for displaying red, a pixel for displaying green, and a pixel for displaying blue. An image corresponding to the image signal D generated by the image processing circuit 100 is displayed in the display area AD.
As described above, M ≦ P and N ≦ Q. That is, the number of effective pixels provided in the effective image sensor area AS may differ from the number of pixels provided in the display area AD.
2A to 2C are respectively a coordinate system fixed to the effective image sensor area AS, a conceptual coordinate system for representing the imaging signal DS, and a display area AD. This is a fixed coordinate system, and the directions of the axes of these three coordinate systems may be different from each other.

  The time length for which the display area AD of the display unit 40 can display an image for one screen is shorter than the time length for which the imaging unit 10 can capture an image for one screen. For this reason, when an image is to be displayed at a cycle that the liquid crystal panel 42 can display, the output of the imaging signal DS from the imaging unit 10 cannot catch up. Therefore, in the imaging display device 1 according to the present embodiment, the display speed of the display unit 40 is adjusted from the imaging unit 10 by adjusting the output timing of the image signal D from the image processing circuit 100 by the image processing circuit 100. The output cycle of the imaging signal DS is followed.

As shown in FIG. 1, the image processing circuit 100 generates an image signal D based on the imaging signal DS, and outputs the generated image signal D to the display unit 40. And a timing control unit 30 that controls the timing at which the unit 20 outputs the image signal D.
The image signal generation unit 20 performs image processing on the imaging signal DS to generate the image signal D, and a VRAM / line buffer that temporarily stores the image signal D generated by the image processing unit 21 22 (hereinafter referred to as “line buffer 22”), and an image signal output unit 23 that acquires the image signal D from the line buffer 22 and outputs the acquired image signal D to the display unit 40.

The image signal D is a signal that defines an image (gradation) to be displayed by each of the M rows × N columns of pixels provided in the display area AD of the liquid crystal panel 42. Hereinafter, among the image signals D indicating the image to be displayed in the display area AD, the image signal D for one line indicating the image to be displayed in the m-th line of the display area AD is the image signal D [m]. (M is a natural number satisfying 1 ≦ m ≦ M).
The image processing unit 21 generates an image signal D indicating an image to be displayed in the display area AD of the liquid crystal panel 42 for each image to be displayed with pixels for one line. More specifically, the image processing unit 21 generates the image signal D in the order of the image signal D [1], the image signal D [2], ..., the image signal D [M]. Details of the image processing executed by the image processing unit 21 will be described later.
When the image processing unit 21 generates the image signal D [m], the image processing unit 21 stores the generated image signal D [m] in the line buffer 22 and completes the storage of the image signal D [m] in the line buffer 22. A write completion signal PtA indicating that this has occurred is output to the timing control unit 30.

  In the present embodiment, the write completion signal PtA is a signal indicating the line number m corresponding to the image signal D for which writing to the line buffer 22 by the image processing unit 21 has been completed. Hereinafter, the line number m indicated by the write completion signal PtA is particularly referred to as “line number ma” (ma is a natural number satisfying 1 ≦ ma ≦ M). The line number ma, which is the value indicated by the write completion signal PtA, is an example of “image processing line information” indicating a line for which image processing has been completed.

  The write completion signal PtA is not limited to a signal indicating a line number, and has a pulse waveform that rises to a high level when the generation of the image signal D [m] by the image processing unit 21 is completed. The binary signal may be included. When the write completion signal PtA is a binary signal, the timing controller 30 counts the number of pulses included in the write completion signal PtA after the display of one screen is started, for example. The line number of the image signal D [m] that has been generated by the processing unit 21 may be calculated. In this case, the pulse waveform (or the number of pulse waveforms) included in the write completion signal PtA corresponds to “image processing line information”.

The image signal output unit 23 reads the image signal D for each line from the line buffer 22 under the control of the timing control unit 30, and outputs the read image signal D [m] for one line to the display unit 40. Output.
Hereinafter, when it is necessary to distinguish for convenience of explanation, the image signal D generated by the image processing unit 21 and stored in the line buffer 22 will be referred to as an image signal DGA, and the image signal output unit 23 will be referred to as the line buffer 22. The image signal D acquired from the above and output to the display unit 40 is referred to as an image signal DGB. Of the image signals DGA stored in the line buffer 22 by the image processing unit 21, the image signal DGA indicating an image to be displayed on the m-th line in the display area AD is referred to as an image signal DGA [m]. Among the image signals DGB output from the output unit 23 to the display unit 40, the image signal DGB indicating an image to be displayed on the m-th line of the display area AD is referred to as an image signal DGB [m].

When the process of reading the image signal DGB [m] from the line buffer 22 and outputting the image signal DGB [m] to the display unit 40 is completed, the image signal output unit 23 displays the image signal DGB [m]. An output completion signal PtB indicating that the output to is completed is output to the timing controller 30. In the present embodiment, the output completion signal PtB is a signal indicating the line number m corresponding to the image signal D that has been output to the display unit 40 by the image signal output unit 23. Hereinafter, the line number m indicated by the output completion signal PtB is particularly referred to as “line number mb” (mb is a natural number satisfying 1 ≦ mb ≦ M). The line number mb that is a value indicated by the output completion signal PtB is an example of “display output line information” that indicates a line of the image signal D [m] that has been output to the display unit 40.
Although details will be described later, the image signal output unit 23 may output an invalid signal Dmy to the display unit 40 instead of outputting the image signal DGB [m] (see FIG. 4). In this case, the image signal output unit 23 does not output the output completion signal PtB.

  The output completion signal PtB is not limited to a signal indicating a line number, and has a pulse waveform that rises to a high level when the output of the image signal D [m] by the image signal output unit 23 is completed. The binary signal may be included.

  As shown in FIG. 1, the timing control unit 30 generates an output control signal CTR based on the write completion signal PtA and the output completion signal PtB, and generates various timing signals to output an image signal. The timing generator 32 that controls the timing at which the unit 23 outputs the image signal DGB [m], and the parameter transmitter 33 that transmits the setting parameter PRM that defines the operation of the EVF controller 41 to the EVF controller 41 are provided.

Based on the write completion signal PtA and the output completion signal PtB, the output control unit 31 displays the image signal D [indicating the image that the image signal output unit 23 should display on the mth row of the display area AD on the display unit 40. m] (image signal DGB [m]) is determined whether it is ready to output, and an output control signal CTR indicating the determination result is generated.
Here, “the preparation for outputting the image signal D [m] (image signal DGB [m]) is completed” means that the following first condition and second condition are satisfied.
(First Condition) The image processing unit 21 has completed the image processing of the m-th row image signal D [m] (image signal DGA [m]).
(Second Condition) The image signal output unit 23 has completed the output of the image signal D [m-1] (image signal DGB [m-1]) in the (m-1) th row.

The first condition is satisfied when the line number ma indicated by the write completion signal PtA is greater than or equal to the line number m, that is, when “m ≦ ma” is satisfied. The second condition is satisfied when the line number mb indicated by the output completion signal PtB satisfies “mb = m−1” (strictly, when “m = 1”, “mb = M” is satisfied). Satisfied when filling).
In this specification, the line of the display area AD that displays the image indicated by the image signal D [m] that is to be determined by the output control unit 31 may be referred to as a “display target line”.

Strictly speaking, in order for the image signal output unit 23 to output the image signal DGB [m], the following third condition needs to be satisfied.
(Third Condition) The timing at which the image signal output unit 23 outputs the image signal D [m] (image signal DGB [m]) in the m-th row is a period during which the display area AD can display an image (described later in FIG. 4). To be included in the horizontal effective data period DHI).
However, when the first condition and the second condition described above are satisfied, the third condition is inevitably caused by the timing generator 32 controlling the output timing of the image signal DGB [m] from the image signal output unit 23. Satisfied. For this reason, in this embodiment, the third condition is not considered in the determination by the output control unit 31.

  The output control unit 31 can determine whether or not “preparation for outputting the image signal D [m] (image signal DGB [m]) has been completed” according to, for example, the following two modes.

As a first mode for determining whether or not “preparation for outputting image signal D [m] (image signal DGB [m]) is completed”, it is determined whether or not the first condition is satisfied (first This is a mode in which the output control unit 31 directly performs two determinations, i.e., determination 1 and determination whether the second condition is satisfied (second determination).
Specifically, when the image processing unit 21 outputs the write completion signal PtA, the output control unit 31 determines whether or not the line number ma indicated by the write completion signal PtA satisfies “m ≦ ma” ( When the image signal output unit 23 outputs the output completion signal PtB, it is determined whether or not the line number mb indicated by the output completion signal PtB satisfies “mb = m−1” (first determination is executed). 2), and when both the determination result of the first determination and the determination result of the second determination are affirmative, it is determined that “preparation for outputting the image signal D [m] has been completed” To do.
In this case, the output control unit 31 functions as a “processing status determination unit” that determines whether or not an image signal corresponding to an image to be displayed on the display target line has been generated by executing the first determination. By performing the second determination, it functions as a “display determination unit” that determines whether or not an image can be displayed on the display target line.
The output control unit 31 determines that both the determination result of the first determination and the determination result of the second determination are negative when the determination result of the first determination or the determination result of the second determination is negative. The first determination and the second determination are repeated until affirmative. Specifically, for example, the output control unit 31 outputs the write completion signal PtA every time the write completion signal PtA is output from the image processing unit 21 until both determination results of the first determination and the second determination are positive. The second determination may be performed every time the output completion signal PtB is output from the image signal output unit 23. Further, for example, the output control unit 31 performs the first determination and the second determination at a period of a horizontal scanning period Hd described later until both determination results of the first determination and the second determination become affirmative. It may be repeated. When both the determination result of the first determination and the determination result of the second determination are affirmative, the output control signal CTR is set to a value indicating that the determination result is affirmative.

Next, as a second mode of determining whether or not “preparation for outputting the image signal D [m] (image signal DGB [m]) is completed”, the output control unit 31 determines whether the previous determination (image signal The first condition is determined at the timing when the image signal output unit 23 outputs the output completion signal PtB after the result of the determination of whether or not preparation for outputting D [m−1] is completed is affirmative. Is a mode in which the determination (first determination) of whether or not is satisfied is executed.
In this aspect, when the determination result of the first determination is negative, the output control unit 31 repeats the first determination until the determination result of the first determination becomes affirmative, and the determination result of the first determination is When the result is affirmative, the output control signal CTR is set to a value indicating that the determination result is affirmative. Specifically, for example, when the determination result of the first determination is negative at the timing when the output completion signal PtB is output, the output control unit 31 thereafter causes the image processing unit 21 to write the write completion signal PtA. Each time is output, it is determined whether or not the line number ma indicated by the write completion signal PtA satisfies “m ≦ ma”. When “m ≦ ma” is satisfied, the first condition is satisfied. What is necessary is just to judge.

As described above, the image processing unit 21 generates the image signal D [m] (image signal DGA [m]) in the order of the line numbers, and the image signal output unit 23 selects the image signal D [m] (the image signal in the order of the line numbers. DGB [m]) is output. In the present embodiment, the output of the image signal D [m-2] in the m-2nd row is completed, and the output control unit 31 is “ready for outputting the image signal D [m-1]”. After the determination, the image signal output unit 23 outputs the image signal D [m−1]. Therefore, the timing at which the output control unit 31 determines whether or not “preparation for outputting the image signal D [m] (image signal DGB [m]) has been completed” is determined from the image signal output unit 23 by the image signal D [ m-2] (image signal DGB [m-2]) is output, and the output control unit 31 sets “image signal D [m−1] (image signal DGB [m−1]”). ) Is ready to output ". That is, the output control unit 31 indicates the output completion signal PtB output by the image signal output unit 23 at the timing of performing the first determination as to whether or not “preparation for outputting the image signal D [m] has been completed”. The line number mb is “m−1”.
For this reason, in the second mode, the output control unit 31 considers that the second condition is satisfied by the output of the output completion signal PtB from the image signal output unit 23. Then, the output control unit 31 performs a determination (first determination) as to whether or not the first condition is satisfied at the timing when the output completion signal PtB is output from the image signal output unit 23, whereby “image It is determined whether or not the output preparation of the signal D [m] (image signal DGB [m]) has been completed.

  In the present embodiment, the following description will be given on the assumption that the second aspect of the two aspects described above is adopted.

The timing generator 32 generates a display vertical synchronization signal DVsync, a vertical valid data signal DVactive, a display horizontal synchronization signal DHsync, a display dot clock signal DCLK, and an enable signal DEnb, and outputs these generated signals to the image signal output unit 23. And output to the display unit 40.
FIG. 4 is a timing chart for explaining the display vertical synchronization signal DVsync, the vertical valid data signal DVactive, the display horizontal synchronization signal DHsync, the display dot clock signal DCLK, and the enable signal DEnb generated by the timing generator 32.

  As shown in FIGS. 4A and 4B, the display vertical synchronization signal DVsync is a vertical scanning period Fd for displaying an image with pixels in the entire display area AD (for M lines) of the liquid crystal panel 42 (that is, This is a signal that defines the frame period of the display unit 40. The display horizontal synchronization signal DHsync is a signal that defines a horizontal scanning period Hd for displaying an image with pixels for one line of the display area AD. The display dot clock signal DCLK is a signal that defines the timing for displaying an image on each pixel of the display area AD.

In the present embodiment, the horizontal scanning period Hd has a predetermined fixed time length. In the present embodiment, the vertical scanning period Fd is composed of a plurality of horizontal scanning periods Hd, and has a variable time length equal to or less than the time length of the vertical scanning period Fs. That is, the number of horizontal scanning periods Hd included in each vertical scanning period Fd is variable. In the example shown in FIG. 4A, among the plurality of vertical scanning periods Fd shown in FIG. 4A, the vertical scanning period Fd1, which is the first vertical scanning period Fd, is a vertical scanning period subsequent to the vertical scanning period Fd1. The vertical scanning period Fd2 is shorter than Fd2, and the vertical scanning period Fd3 following the vertical scanning period Fd2 is illustrated as an example.
Of the waveforms of the display vertical synchronization signal DVsync, a pulse-like waveform that defines the start and end timings of the vertical scanning period Fd is referred to as a vertical synchronization pulse PlsV. Of the waveforms of the display horizontal synchronization signal DHsync, a pulse-like waveform that defines the start and end timings of the horizontal scanning period Hd is referred to as a horizontal synchronization pulse PlsH.

As shown in FIG. 4B, the vertical scanning period Fd includes a vertical synchronization period DVp, a vertical back porch period DVb, a vertical valid data period DVI, and a vertical front porch period DVf.
The vertical synchronization period DVp is a period in which the display vertical synchronization signal DVsync is active (low level in this example), and has a predetermined time length that starts at the same time as the vertical scanning period Fd starts. It is a period to have. The vertical back porch period DVb is a period subsequent to the vertical synchronization period DVp and has a predetermined time length. The vertical valid data period DVI is a variable time period following the vertical back porch period DVb. In the vertical valid data period DVI, the image signal output unit 23 outputs the image signal DGB (image signals DGB [1] to DGB [M]). The vertical front porch period DVf is a period subsequent to the vertical effective data period DVI, and is a period having a predetermined time length that ends at the same time as the end of the vertical scanning period Fd.
The vertical valid data period DVI is a period from the start of the horizontal scanning period Hd in which the enable signal DEnb is first active in each vertical scanning period Fd to the end of the horizontal scanning period Hd in which the enable signal DEnb is active for the Mth time. (The case where the enable signal DEnb becomes active will be described later).
The vertical valid data period DVI may be determined based on, for example, a count value output by a counter (not shown) that counts the number of times the enable signal DEnb is active. However, in the present embodiment, for convenience of explanation, in each vertical scanning period Fd, from the start of the horizontal scanning period Hd in which the enable signal DEnb is first active to the end of the horizontal scanning period Hd in which the enable signal DEnb is active for the Mth time. The vertical effective data signal DVactive which is active (high level in the example of this figure) during this period is introduced. That is, in the present embodiment, a period in which the vertical valid data signal DVactive is active will be described as a vertical valid data period DVI. The vertical valid data signal DVactive is a signal introduced for convenience of explanation, and the output control unit 31 may not output the vertical valid data signal DVactive.

As shown in FIGS. 4C and 4D, the horizontal scanning period Hd includes a horizontal synchronization period DHp, a horizontal back porch period DHb, a horizontal effective data period DHI, and a horizontal front porch period DHf.
The horizontal synchronization period DHp is a period in which the display horizontal synchronization signal DHsync is active (low level in this example), and has a predetermined time length that starts at the same time as the horizontal scanning period Hd starts. It is a period to have. The horizontal back porch period DHb is a period subsequent to the horizontal synchronization period DHp, and is a period having a predetermined time length. The horizontal effective data period DHI is a period having a predetermined time length following the horizontal back porch period DHb. The horizontal front porch period DHf is a period following the horizontal effective data period DHI, and is a period having a predetermined time length that ends at the same time as the end of the horizontal scanning period Hd.

In the present embodiment, in the horizontal scanning period Hd, the effective horizontal scanning period Hd-A (see FIG. 4C) for the image signal output unit 23 to output the image signal D [m] and the image signal D [ There is an invalid horizontal scanning period Hd-D (see FIG. 4D) in which an invalid signal Dmy [m] is output instead of outputting m].
FIG. 4C illustrates a case where the horizontal scanning period Hd is the effective horizontal scanning period Hd-A. As shown in this figure, the enable signal DEnb is active (high level in the example of this figure) in the horizontal effective data period DHI when the horizontal scanning period Hd is the effective horizontal scanning period Hd-A. Then, in the horizontal effective data period DHI in which the enable signal DEnb is active, the image signal D [m] (image signal DGB [m]) is output from the image signal output unit 23. On the other hand, the enable signal DEnb is inactive during a period other than the horizontal effective data period DHI (horizontal synchronization period DHp, horizontal back porch period DHb, horizontal front porch period DHf) in the effective horizontal scanning period Hd-A. The image signal output unit 23 outputs the image signal D [m] (image signal DGB [m]) in a period other than the horizontal effective data period DHI in which the enable signal DEnb is inactive during the effective horizontal scanning period Hd-A. And the invalid line signal DGB-dmy is output.
The third condition described above is satisfied when the timing generator 32 activates the enable signal DEnb in the horizontal valid data period DHI. That is, the timing control unit 30 including the output control unit 31 and the timing generator 32 is an image signal D [m] (image) corresponding to the display target line at a timing when all of the first condition to the third condition described above are satisfied. The signal DGB [m]) is output from the image signal output unit 23.

FIG. 4D illustrates the case where the horizontal scanning period Hd is the invalid horizontal scanning period Hd-D. As shown in this figure, the enable signal DEnb is inactive in the horizontal effective data period DHI when the horizontal scanning period Hd is the invalid horizontal scanning period Hd-D. The image signal output unit 23 outputs the invalid signal Dmy instead of the image signal D [m] (image signal DGB [m]) in the horizontal valid data period DHI in the invalid horizontal scanning period Hd-D. . On the other hand, the enable signal DEnb is inactive during the invalid horizontal scanning period Hd-D other than the horizontal valid data period DHI (horizontal synchronization period DHp, horizontal back porch period DHb, horizontal front porch period DHf). The image signal output unit 23 stops the output of the image signal D [m] (image signal DGB [m]) in the invalid horizontal scanning period Hd-D other than the horizontal valid data period DHI, and the invalid line The signal DGB-dmy is output.
The timing generator 32 sets the horizontal scanning period Hd to either the effective horizontal scanning period Hd-A or the invalid horizontal scanning period Hd-D based on the output control signal CTR output from the output control unit 31, in other words. For example, it is determined whether or not to enable the enable signal DEnb in the horizontal valid data period DHI. The relationship among the types of the output control signal CTR, the enable signal DEnb, and the horizontal scanning period Hd will be described later.

FIG. 5 is an explanatory diagram for explaining the relationship between various signals generated by the timing generator 32 and image display timing in the display area AD of the liquid crystal panel 42.
As shown in this figure, in the M rows × N columns of pixels from the first row line to the Mth row line of the display area AD, the vertical effective data signal DVactive is active during the vertical scanning period Fd. In the vertical valid data period DVI, an image for one screen indicated by the image signals D [1] to D [M] is displayed.
Further, the N pixels constituting the m-th row line in the display area AD are in the horizontal effective data period DHI in which the enable signal DEnb is active in the horizontal scanning period Hd (effective horizontal scanning period Hd-A). The image indicated by the image signal D [m] is displayed.
The vertical valid data period DVI is extended by the number of invalid horizontal scanning periods Hd-D included in the vertical valid data period DVI. In this figure, the horizontal scanning period Hd included in the vertical valid data period DVI is It is assumed that all are effective horizontal scanning periods Hd-A.

FIG. 6 is an explanatory diagram for explaining the output control signal CTR and the enable signal DEnb.
As described above, when the output control unit 31 determines that the output preparation of the image signal D [m] is completed, that is, when the first condition and the second condition are satisfied, the determination result is output to the output control signal CTR. Sets a value indicating that is positive. Specifically, in the present embodiment, when the output control unit 31 determines that the output preparation of the image signal D [m] is completed, the output control unit 31 temporarily has a pulse-like waveform that rises to a high level. Set. As shown in FIG. 6, a waveform indicating a determination result indicating that preparation for outputting the image signal D [m] set to the output control signal CTR is completed is referred to as an output permission pulse PL [m].

As described above, the output control unit 31 according to the present embodiment considers that the second condition is satisfied when the output completion signal PtB is output from the image signal output unit 23. Then, the output control unit 31 determines whether or not the image processing of the image signal D [m] is completed when the output completion signal PtB is output (whether the first condition is satisfied) (first condition). It is determined whether or not preparation for outputting the image signal D [m] has been completed.
As illustrated in FIG. 6, when the output control unit 31 determines whether or not the output preparation of the image signal D [m] is completed, the output control unit 31 completes the image processing of the image signal D [m]. The timing at which it is determined that the first condition is satisfied (that is, the timing at which the result of the first determination is affirmative) is referred to as image processing determination time TA [m].
The timing at which the output completion signal PtB is supplied to the output control unit 31 (assuming that the second condition is satisfied) is referred to as a display preparation determination time TB [m].
In the following, for convenience of explanation, the time at which the generation of the image signal D [m] by the image processing unit 21 is actually completed is defined as an image signal generation time TC [m]. That is, the image signal generation time TC [m] is substantially the same time as the time when the image processing unit 21 outputs the write completion signal PtA. The image signal generation time TC [m] is an example of “image signal generation time”.

The display preparation determination time TB [m] is substantially the same as the time when the output of the image signal D [m-1] from the output control unit 31 is completed, and the image signal D [m-1] is output validly. This is substantially the same time as the end of the horizontal effective data period DHI of the horizontal scanning period Hd-A (referred to as effective horizontal scanning period Hd-A [m-1]). The time when the horizontal effective data period DHI of the first horizontal scanning period Hd [m] after the display preparation determination time TB [m] is started is an example of “displayable time”.
In this specification, “substantially the same time” refers to the same time when there is a time lag caused by transmission / reception of signals or a time lag caused by various processes, and these time lags are ignored. It is a concept that includes cases that can be done.

The image processing determination time TA [m] is generated when the generation of the image signal D [m] (image signal DGA [m]) is completed by the display preparation determination time TB [m], that is, the display preparation determination time TB. When the image signal generation time TC [m] has passed by [m] (referred to as Case-1), the display preparation determination time TB [m] is substantially the same time.
In the case of Case-1, after the display preparation determination time TB [m] has elapsed, the output control unit 31 writes the write completion signal supplied to the output control unit 31 by the display preparation determination time TB [m]. Since it is determined that the line number ma indicated by PtA satisfies “m ≦ ma” and the timing of the determination is the image processing determination time TA [m], the image processing determination time TA [m] and the display preparation determination There is a time lag between time TB [m], but in the following, both times are considered to be substantially the same for simplicity.

On the other hand, the image processing determination time TA [m] is determined when the generation of the image signal D [m] (image signal DGA [m]) is not completed by the display preparation determination time TB [m], that is, the display preparation determination. When the image signal generation time TC [m] has not arrived by the time TB [m] (referred to as Case-2), the time when the image processing unit 21 completes the generation of the image signal D [m], That is, it is substantially the same time as the image signal generation time TC [m].
In the case of Case-2, after the image processing unit 21 completes the generation of the image signal D [m] at the image signal generation time TC [m], the image processing unit 21 outputs the write completion signal PtA, Since the timing at which the output control unit 31 that has received the supply of the write completion signal PtA determines that “m ≦ ma” is satisfied is the image processing determination time TA [m], the image processing determination time TA [m] Although there is a time lag between the image signal generation time TC [m], in the following, both times are considered to be substantially the same for simplicity.

The output control unit 31 outputs an output permission pulse to the output control signal CTR at the later time of the image processing determination time TA [m] and the display preparation determination time TB [m], that is, the image processing determination time TA [m]. Set PL [m]. That is, the output permission pulse PL [m] is output when the first condition and the second condition for the image signal D [m] are satisfied. The timing generator 32 is first enabled after the output permission pulse PL [m] is output and when the third condition is satisfied, in other words, after the output permission pulse PL [m] is output. Control is performed so that the image signal D [m] is output from the image signal output unit 23 when the signal DEnb becomes active.
Hereinafter, for convenience of explanation, the time when all of the first condition to the third condition are satisfied for the image signal D [m] is referred to as an output condition satisfaction time TJ [m].

  In the present embodiment, the timing generator 32 determines the level of the internal processing signal IS to be used in the internal processing of the timing generator 32 based on the output control signal CTR. Then, the timing generator 32 determines the timing at which the enable signal DEnb is activated and the type of the horizontal scanning period Hd (effective horizontal scanning period Hd-A or invalid horizontal scanning period Hd-D) based on the internal processing signal IS. decide.

Specifically, as shown in FIG. 6, the timing generator 32 activates the internal processing signal IS when the output permission pulse PL [m] is set in the output control signal CTR (high level in this example). And
Next, when the internal processing signal Is is active at the timing when the horizontal scanning period Hd is started, the timing generator 32 determines the classification of the horizontal scanning period Hd as an effective horizontal scanning period Hd-A [m] (classification). And the enable signal DEnb is activated at the timing when the horizontal effective data period DHI of the effective horizontal scanning period Hd-A [m] is started. The timing when the enable signal DEnb becomes active corresponds to the output condition satisfaction time TJ [m].
Then, the timing generator 32 generates an internal signal at the timing when the horizontal effective data period DHI of the effective horizontal scanning period Hd-A [m] starts and the enable signal DEnb becomes active, that is, at the output condition satisfaction time TJ [m]. The processing signal IS is deactivated.

  On the other hand, when the internal processing signal IS is inactive at the timing when the horizontal scanning period Hd is started, the timing generator 32 determines (classifies) the type of the horizontal scanning period Hd as the invalid horizontal scanning period Hd-D, The enable signal DEnb is inactive during the invalid horizontal scanning period Hd-D.

Hereinafter, in the example shown in FIG. 6, the output control unit 31 determines whether or not the output preparation of the image signal D [2] is completed, and sets the output permission pulse PL [2] in the output control signal CTR. A case (corresponding to Case-1) will be described.
In the example shown in FIG. 6, the display preparation determination time TB [2] is the horizontal effective period of the horizontal scanning period Hd [1] (effective horizontal scanning period Hd-A [1]) in which the output of the image signal D [1] is completed. This is the end of the data period DHI. In this example, it is assumed that the image signal generation time TC [2] for completing the image processing of the image signal D [2] comes before the display preparation determination time TB [2]. Therefore, in this example, the image processing determination time TA [2] is substantially the same time as the display preparation determination time TB [2]. Therefore, the output control unit 31 outputs the output permission pulse PL [2] as the output control signal CTR at the end of the horizontal effective data period DHI of the horizontal scanning period Hd [1], that is, at the display preparation determination time TB [2]. Is output.
The timing generator 32 activates the internal processing signal Is at the timing when the output permission pulse PL [2] is output as the output control signal CTR, that is, at the timing when the horizontal valid data period DHI of the horizontal scanning period Hd [1] ends. To do. In this case, the internal processing signal Is is active even at the start of the horizontal scanning period Hd [2]. Therefore, the timing generator 32 sets the horizontal scanning period Hd [2] as the effective horizontal scanning period Hd-A [2], and activates the enable signal DEnb in the horizontal effective data period DHI of the horizontal scanning period Hd [2]. .
That is, the start time of the horizontal effective data period DHI of the horizontal scanning period Hd [2] is the output condition satisfaction time TJ [2] in which all of the first condition to the third condition for the image signal D [2] are satisfied. Become. Therefore, the image signal D [2] is output in the horizontal scanning period Hd [2]. The timing generator 32 deactivates the internal processing signal IS at the timing when the horizontal valid data period DHI of the horizontal scanning period Hd [2] is started.

Next, in the example shown in FIG. 6, the output control unit 31 determines whether or not the output preparation of the image signal D [3] is completed, and sets the output permission pulse PL [3] in the output control signal CTR. The case (corresponding to Case-2) will be described.
In the example shown in FIG. 6, the display preparation determination time TB [3] is the horizontal effective period of the horizontal scanning period Hd [2] (effective horizontal scanning period Hd-A [2]) in which the output of the image signal D [2] is completed. This is the end of the data period DHI. In this example, it is assumed that the image signal generation time TC [3] for completing the image processing of the image signal D [3] is later than the display preparation determination time TB [3]. Therefore, the image processing determination time TA [3] is later than the display preparation determination time TB [3]. In this example, it is assumed that the image signal generation time TC [3] is after the start of the horizontal scanning period Hd [3]. Therefore, the output control unit 31 outputs the output permission pulse PL [3] at a time after the start of the horizontal scanning period Hd [3].
As described above, the timing generator 32 deactivates the internal processing signal Is at the start of the horizontal effective data period DHI of the horizontal scanning period Hd [2]. For this reason, the internal processing signal IS is inactive at the start of the horizontal scanning period Hd [3]. Therefore, the timing generator 32 classifies the horizontal scanning period Hd [3] as the invalid horizontal scanning period Hd-D, and deactivates the enable signal DEnb in the horizontal valid data period DHI of the horizontal scanning period Hd [3]. In this case, the image signal output unit 23 outputs the invalid signal Dmy without outputting the image signal D [3] in the horizontal effective data period DHI of the horizontal scanning period Hd [3].
Thereafter, the timing generator 32 activates the internal processing signal IS at the timing when the output permission pulse PL [3] is output as the output control signal CTR. In this example, the timing at which the output permission pulse PL [3] is output is before the start of the horizontal scanning period Hd [4]. In this case, the internal processing signal IS is active even at the start of the horizontal scanning period Hd [4]. Therefore, the timing generator 32 sets the horizontal scanning period Hd [4] as the effective horizontal scanning period Hd-A [3], and activates the enable signal DEnb in the horizontal effective data period DHI of the horizontal scanning period Hd [4]. .
That is, the start time of the horizontal effective data period DHI of the horizontal scanning period Hd [4] is the output condition satisfaction time TJ [3] in which all of the first condition to the third condition for the image signal D [3] are satisfied. Become. For this reason, the image signal D [3] is output in the horizontal scanning period Hd [4].
In the example shown in this figure, the output control unit 31 determines whether or not the output preparation of the image signal D [1] is completed and sets the output permission pulse PL [1] in the output control signal CTR. When determining whether or not the output preparation of the image signal D [1] is completed and setting the output permission pulse PL [1] in the output control signal CTR, the case of Case-1 is assumed. Yes.

  As described above, in this embodiment, the output control unit 31 outputs the output permission pulse PL [m] when the first condition and the second condition are satisfied. Then, the image signal output unit 23 outputs the image signal D [m] in the first horizontal scanning period Hd after the output permission pulse PL [m] is output. Therefore, when the image signal D [m] cannot be output from the image signal output unit 23 in a certain horizontal scanning period Hd by image processing by the image processing unit 21, the image signal D [m] from the image signal output unit 23 Can be adjusted with the accuracy of the horizontal scanning period Hd.

In the example illustrated in FIG. 6, the timing generator 32 determines the type of the horizontal scanning period Hd at the timing at which the horizontal scanning period Hd is started. However, this is merely an example. From the start of the horizontal front porch period DHf of the horizontal scanning period Hd in which PL [m] is output to the end of the horizontal back porch period DHb of the first horizontal scanning period Hd after the output permission pulse PL [m] is output. It may be decided between.
In the example shown in FIG. 6, the timing at which the internal processing signal IS is deactivated is the timing at which the enable signal DEnb is activated. However, this is only an example, and the timing generator 32 deactivates the internal processing signal IS. The timing of activation may be any time as long as it is during the horizontal effective data period DHI from when the enable signal DEnb becomes active until it becomes inactive.

In the present embodiment, the timing generator 32 determines the waveform of the enable signal DEnb and the type of the horizontal scanning period Hd using the internal processing signal IS, but this is only an example, and the internal processing signal IS These may be determined based on the output control signal CTR without using the signal IS.
In the present embodiment, the output control signal CTR has a waveform including the output permission pulse PL [m]. This is an example, and the output control signal CTR is, for example, an internal processing signal shown in FIG. It may have an IS waveform. In this case, the timing generator 32 may supply the output control unit 31 with various signals such as the enable signal DEnb necessary for the output control unit 31 to determine the waveform of the output control signal CTR.

FIG. 7 is an explanatory diagram for explaining the relationship between the effective horizontal scanning period Hd-A, the invalid horizontal scanning period Hd-D, and the vertical scanning period Fd.
The vertical scanning period Fd is a period for outputting the image signals D [1] to D [M] corresponding to the M rows. For this reason, the timing generator 32 provides M effective horizontal scanning periods Hd-A in the vertical effective data period DVI of each vertical scanning period Fd.
On the other hand, the timing generator 32 according to the present embodiment classifies the horizontal scanning period Hd into either an effective horizontal scanning period Hd-A or an invalid horizontal scanning period Hd-D. Only when the horizontal scanning period Hd is the effective horizontal scanning period Hd-A, the image signal D [m] is output in the horizontal scanning period Hd.
For this reason, the timing generator 32 according to the present embodiment provides a time length corresponding to the invalid horizontal scanning period Hd-D when the invalid horizontal scanning period Hd-D is provided in the vertical valid data period DVI of the vertical scanning period Fd. Display vertical synchronizing signal DVsync and vertical effective data signal DVactive so that M effective horizontal scanning periods Hd-A are provided in vertical effective data period DVI of each vertical scanning period Fd. To do.

For example, when the timing generator 32 sets all the horizontal scanning periods Hd of the vertical effective data period DVI as the effective horizontal scanning period Hd-A as in the vertical scanning period Fd1 shown in FIG. The time length of the data period DVI is set to a time length M times the horizontal scanning period Hd.
On the other hand, the timing generator 32, when providing one or more invalid horizontal scanning periods Hd-D in the vertical effective data period DVI as in the vertical scanning period Fd2 shown in FIG. Is a time length obtained by adding the time length M times the horizontal scanning period Hd and the total time length of one or more invalid horizontal scanning periods Hd-D existing in the vertical effective data period DVI. .
That is, the timing generator 32 adjusts the time length of the vertical scanning period Fd in units of the horizontal scanning period Hd, so that the image signal output unit 23 in each vertical scanning period Fd causes the image signals D [1] to D [M ] Can be output.

Note that the time length of the vertical scanning period Fd when all the horizontal scanning periods Hd of the vertical effective data period DVI are effective horizontal scanning periods Hd-A as in the vertical scanning period Fd1 shown in FIG. This is referred to as the standard vertical scanning time Td. Further, the maximum frame rate that can be displayed on the display unit 40, that is, the frame rate when the time length of the vertical scanning period Fd is the standard vertical scanning time Td, is “1 / Td”. May be called. Of the timing control executed by the timing control unit 30, controlling the timing so that the image signal D is output at the second frame rate may be referred to as “first timing control”.
Further, as in the vertical scanning period Fd2 shown in FIG. 7B, when one or more invalid horizontal scanning periods Hd-D are provided in the vertical valid data period DVI, the one or more invalid horizontal scanning periods Hd. The total time length of -D is referred to as extended vertical scanning time Tex. That is, the time length of the vertical scanning period Fd when one or more invalid horizontal scanning periods Hd-D are provided in the vertical valid data period DVI is the sum of the standard vertical scanning time Td and the extended vertical scanning time Tex. Of the timing control executed by the timing control unit 30, the timing is controlled so that the image signal D is output in the vertical effective data period DVI in which one or more invalid horizontal scanning periods Hd-D are provided. , Sometimes referred to as “second timing control”. Although details will be described later, in the second timing control, the timing control unit 30 controls the timing so that the image signal D is output at the first frame rate.

Next, the display unit 40 will be described with reference to FIG.
FIG. 8 is a block diagram illustrating a configuration of the display unit 40. As described above, the display unit 40 includes the EVF controller 41 that controls the operation of the liquid crystal panel 42, and the liquid crystal panel 42 that displays an image according to the image signal D.
As described above, the liquid crystal panel 42 is provided with the display area AD for displaying an image corresponding to the image signal D. The display area AD is also received corresponding to the intersection of the M scanning lines extending in the X-axis direction, the N data lines extending in the Y-axis direction, and the scanning lines and data lines in FIG. And M rows × N columns of pixels. Further, the liquid crystal panel 42 can enlarge and observe a scanning line driving circuit 421 for selecting scanning lines, a data line driving circuit 422 for driving data lines, and an image displayed in the display area AD. An optical system (not shown).

The EVF controller 41 includes a data input unit 411 to which an image signal D (image signal DGB) is input from the image signal output unit 23, and the number of effective horizontal scanning periods Hd-A in each vertical effective data period DVI (the enable signal DEnb is A counter 412 that counts the number of times of activation), a timing generator 413 that generates various timing signals that define the driving timing of the liquid crystal panel 42, and an image signal D (image signal DGC) to the liquid crystal panel 42. A data output unit 414; and a register 415 for storing a setting parameter PRM that defines the operation of the EVF controller 41.
In the present embodiment, data transmission between the image processing circuit 100 (the image signal generation unit 20 and the timing control unit 30) and the EVF controller 41 is performed by a parallel interface (not shown).

  When the imaging display device 1 operates in the live view mode, for example, when the user of the imaging display device 1 selects the operation in the live view mode using the operation unit 60, the imaging display device 1 starts the operation in the live view mode. Prior to this, the parameter transmission unit 33 supplies the setting parameter PRM to the timing generation unit 413. Then, the timing generation unit 413 stores (sets) the transmitted setting parameter PRM in the register 415.

The setting parameter PRM set in the register 415 is a value that defines the operation of the EVF controller 41 for operating the EVF controller 41 in accordance with the specifications of the liquid crystal panel 42.
As the setting parameter PRM, for example, the time length of the horizontal scanning period Hd (or the clock number of the display dot clock signal DCLK included in the horizontal scanning period Hd. Hereinafter, the clock number of the display dot clock signal DCLK is simply referred to as “clock number”. ), The time length of the horizontal effective data period DHI (or the number of pixels in the X-axis direction (N) in the display area AD), the time length of the horizontal synchronization period DHp (or the number of clocks), the horizontal back porch period DHb Of the vertical effective data period DVI when the invalid horizontal scanning period Hd-D is not included in the time length (or the number of clocks) of the horizontal front porch period DHf (or the number of clocks) of the vertical effective data period DVI. Time length (or the number of lines in the Y-axis direction (M) in the display area AD. In other words, the effective water included in the vertical effective data period DVI. The number of scanning periods Hd-A), the time length (or number of clocks) of the vertical synchronization period DVp, the time length (or number of clocks) of the vertical back porch period DVb, and the time length (or clocks) of the vertical front porch period DVf. Number) and the like.
In the imaging display device 1 according to the present embodiment, the operation timing of the EVF controller 41 according to the specifications of the liquid crystal panel 42 can be set from the image processing circuit 100. Therefore, when changing the size of the liquid crystal panel 42, the frame rate, etc. Even when the specification of the liquid crystal panel 42 is changed, it is not necessary to change the EVF controller 41. For this reason, it becomes possible to improve the versatility of the system.

The data input unit 411 receives from the image processing circuit 100 the display dot clock signal DCLK, the output signal from the image signal output unit 23 including the image signal D (image signal DGB) and the invalid signal Dmy, the enable signal DEnb, Is supplied.
When the enable signal DEnb becomes active, the data input unit 411 supplies the image signal D [m for one line supplied from the image signal output unit 23 in synchronization with the display dot clock signal DCLK while the enable signal DEnb is active. ] And the captured image signal D [m] is output to the data output unit 414. On the other hand, when the enable signal DEnb is inactive, the data input unit 411 discards the invalid signal Dmy supplied from the image signal output unit 23 without taking it in.

The counter 412 is supplied with the enable signal DEnb and the display vertical synchronization signal DVsync from the image processing circuit 100.
The counter 412 counts rising edges of the enable signal DEnb, and outputs a count value Cnt indicating the count result to the timing generation unit 413. The counter 412 resets the count value Cnt to “0” when the display vertical synchronization signal DVsync becomes active and the vertical synchronization pulse PlsV is supplied as the display vertical synchronization signal DVsync. Therefore, the counter 412 can count the number of effective horizontal scanning periods Hd-A included in each vertical scanning period Fd. That is, when the data input unit 411 takes in the image signal D [m] that designates an image to be displayed on the m-th line, the count value Cnt is the line number corresponding to the image signal D [m] ( m).

The timing generation unit 413 is supplied with the display vertical synchronization signal DVsync, the display horizontal synchronization signal DHsync, and the setting parameter PRM from the image processing circuit 100, and is supplied with the count value Cnt from the counter 412.
As described above, when the setting parameter PRM is supplied from the image processing circuit 100, the timing generation unit 413 sets the supplied setting parameter PRM in the register 415.

When the count value Cnt indicates “m”, the timing generation unit 413 causes the scanning line driving circuit 421 to select the m-th line (scanning line) corresponding to the count value Cnt. Further, when the count value Cnt indicates “m”, the timing generation unit 413 outputs the image signal D [m] for one line captured by the data input unit 411 to the data output unit 414 as the image signal DGC. [m] is supplied to the data line driving circuit 422. In this case, the data line driving circuit 422 outputs the data output unit 414 to N pixels (m-th row line) provided corresponding to the m-th row scanning line selected by the scanning line driving circuit 421. The image signal DGC [m] supplied from is written through the data line. As a result, the image of the selected line is displayed in the display area AD. In the present embodiment, the image signals DGA and DGB are digital signals, but the image signal DGC may be a digital signal or an analog signal.
Thus, the EVF controller 41 displays the image indicated by the image signal D supplied from the image signal output unit 23 in the display area AD of the liquid crystal panel 42.

When the EVF controller 41 detects that the count value Cnt is equal to the number of lines “M” of the display area AD set in the register 415, the display horizontal synchronization signal DHsync is first supplied after the detection. At the timing, that is, the timing at which the vertical front porch period DVf is started, preparation for the modified frame processing is started. After the vertical front porch period DVf is started, the timing generation unit 413 outputs a modified frame processing start signal Cng for instructing the data input unit 411 and the data output unit 414 to execute the modified frame processing. To do.
Here, the modified frame process is a process for preparing to display an image in the next vertical scanning period Fd. For example, the data stored in the buffers provided in the data input unit 411 and the data output unit 414 is erased. Processing to be executed is included. The frame reform process is started after the start of the vertical front porch period DVf. Further, it is preferable that the modified frame process is completed before the end of the vertical front porch period DVf.

<2. Image processing>
Next, details of the image processing unit 21 and image processing executed by the image processing unit 21 will be described with reference to FIGS. 9 to 11.

FIG. 9 is a block diagram illustrating a configuration of the image processing unit 21.
As shown in this figure, the image processing unit 21 performs interpolation processing on the line buffer 211 that temporarily stores the imaging signal DS output from the image sensor 12 and the imaging signal DS stored in the line buffer 211. A pixel interpolation processing unit 212 to perform, a color reproduction processing unit 213 to perform color reproduction processing on the interpolated imaging signal DS, a filter processing unit 214 to perform filter processing on the color reproduced imaging signal DS, and a filter A gamma correction unit 215 that performs gamma correction on the processed image signal DS, a line buffer 216 that temporarily stores the gamma-corrected image signal DS, and an image signal DS stored in the line buffer 216 in the display area A resizing processing unit 217 that performs resizing processing for converting the image signal D into the number of pixels included in the AD.

FIG. 10 is an explanatory diagram for explaining the relationship between the pixel data signal Sig output from the light receiving element included in the image sensor 12 and the imaging signal DS. As described above, when the imaging display device 1 operates in the imaging mode, the image sensor 12 outputs the entire pixel data signal Sig shown in FIG. 10A as still image imaging data. On the other hand, when the imaging display device 1 operates in the live view mode, the image sensor 12 thins out the pixel data signal Sig and outputs the imaging signal DS shown in FIG.
In FIG. 10A, each of the squares with “R” indicates a pixel data signal Sig output from the light receiving element (for detecting red light) corresponding to the red pixel, and “G”. Each of the squares marked with indicates a pixel data signal Sig output from the light receiving element of the green pixel, and each of the squares marked with “B” indicates pixel data output from the light receiving element of the blue pixel Signal Sig is shown.
In FIG. 10B, each of the squares marked with “R” corresponds to the image signal D corresponding to the red pixel of the display area AD (designating the gradation to be displayed by the red pixel). An imaging signal DS for generation is shown, and each of the squares marked with “G” indicates an imaging signal DS for generating an image signal D corresponding to a green pixel of the display area AD, and “B”. Each of the squares marked with indicates an imaging signal DS for generating an image signal D corresponding to a blue pixel of the display area AD.
In the present embodiment, it is assumed that red, green, and blue pixels are arranged in a Bayer array in the effective image sensor area AS of the image sensor 12 and the display area AD of the liquid crystal panel 42.

  As described above, the image sensor 12 performs the thinning process on the pixel data signal Sig output from the light receiving element of PS row × QS column, thereby outputting the imaging signal DS corresponding to the pixel of P row × Q column. .

In the thinning process, the image sensor 12 thins the pixel data signal Sig output from the light receiving element of the PS row into a signal corresponding to the pixel of the P row in the Y-axis direction.
Specifically, the image sensor 12 selects a readout target line from the PS row, reads out the pixel data signal Sig output from the light receiving element located in the readout target line, and is positioned in a line other than the readout target line. By skipping the pixel data signal Sig output by the light receiving element, the thinning is performed in the Y-axis direction.

In the present embodiment, since the arrangement of the pixels provided in the effective image sensor area AS of the image sensor 12 is a Bayer arrangement, the readout target line is determined at a rate of one line per odd line.
When the pixels are arranged in a Bayer array, lines composed of red pixels and green pixels and lines composed of green pixels and blue pixels are alternately arranged in the Y-axis direction. For this reason, by setting the readout target line at a ratio of one line to the odd number lines, the pixel after readout is divided into a line composed of red pixels and green pixels and a line composed of green pixels and blue pixels in the Y-axis direction. It can arrange so that it may line up alternately, and it can prevent that the pixel of the same color adjoins.
In the example shown in FIG. 10A, the read target lines are defined at a rate of 1 line per 3 lines in the Y-axis direction. Specifically, among the lines 1 to 9, the line 1, the line 4, and the line 7 are read target lines. In this case, as shown in FIG. 10B, it is possible to obtain an imaging signal DS in which the pixel data signal Sig is thinned out to one third in the Y-axis direction.

Further, the image sensor 12 performs thinning in the X-axis direction as the thinning process, and uses the pixel data signal Sig output from the light receiving elements in the QS column as a signal (imaging signal DS) corresponding to the pixels in the Q column.
Specifically, the image sensor 12 groups the QS light receiving elements positioned on each readout target line so that a predetermined number of light receiving elements form one set, and the predetermined number of light elements constituting each set. By averaging the values indicated by the pixel data signals Sig output by the light receiving elements, the pixel data signals Sig output by the QS light receiving elements are reduced to “a predetermined number”.

When the pixels of the image sensor 12 are arranged in a Bayer array, since pixels of two colors are alternately arranged in each line, grouping is performed in which a predetermined number of light receiving elements are set as a set every other line in the X-axis direction. Do. Then, the values indicated by the pixel data signals Sig output from a predetermined number of light receiving elements constituting each set are arithmetically averaged.
In the example shown in FIG. 10A, in the X-axis direction, every other three light receiving elements are grouped as one set, and pixel data signals output by the three light receiving elements of the same color constituting each set. The value indicated by Sig is averaged. Specifically, for example, in line 1 and line 7, pixels that are output by three light receiving elements corresponding to the three pixels, with the red pixels in the first, third, and fifth columns as a set. The value of the data signal Sig is arithmetically averaged by the adder Ave1, and the obtained average value is set as the value of the imaging signal DS corresponding to the red pixel in the first column. Similarly, in line 1 and line 7, the value of the pixel data signal Sig output from the light receiving elements corresponding to the green pixels in the second, fourth, and sixth columns is arithmetically averaged by the adder Ave2. The average value is set as the value of the imaging signal DS corresponding to the green pixels in the second column. In this case, as shown in FIG. 10B, it is possible to obtain an imaging signal DS in which the pixel data signal Sig is resized (reduced) to one third in the X-axis direction.

FIG. 11 is an explanatory diagram for explaining the resizing process executed by the resizing processing unit 217.
As described above, the number of pixels of the image indicated by the imaging signal DS is different from the number of pixels of the image indicated by the image signal D (the number of pixels in the display area AD). Therefore, the resizing processing unit 217 executes resizing processing for converting the imaging signal DS into an image signal D corresponding to the number of pixels in the display area AD.

  Note that the image indicated by the imaging signal DS may have distortion due to the optical characteristics of the lens provided in the imaging optical system 11. Specifically, the image showing the imaging result when the subject is imaged is barrel-shaped aberration that expands outward compared to the image that should be originally displayed in the display area AD, or compared to the image that should be originally displayed. May have pincushion aberrations that shrink toward the inside. For this reason, the resizing processing unit 217 executes distortion correction processing for correcting distortion aberration such as barrel aberration and pincushion aberration in the resizing processing.

  Hereinafter, the distortion correction processing executed in the resizing processing will be described with reference to FIG. In FIG. 11, it is assumed that the number of lines of the image indicated by the imaging signal DS is 16 lines (P = 16) and the number of lines of the image indicated by the image signal D is 12 lines (M = 12). .

11C and 11D show an image indicated by the image signal D to be displayed in the display area AD when the subject is imaged, and FIG. 11A shows a case where barrel aberration occurs. Fig. 11B shows an image indicated by the image pickup signal DS when pincushion aberration occurs.
That is, FIG. 11A shows a case where the image to be displayed in the display area AD is a square SQ, but the imaging signal DS shows a closed curve CV1 in which the square SQ is expanded due to barrel aberration. FIG. 11B shows a case where the image signal DS shows a closed curve CV2 in which the quadrangle SQ contracts due to pincushion aberration, even though the image to be displayed in the display area AD is the quadrangle SQ.

When the barrel aberration as shown in FIG. 11A occurs, the resizing processing unit 217 displays the image indicated by the closed curve CV1 as shown in FIG. 11A in the distortion correction processing as shown in FIG. It correct | amends to the image shown by the square SQ as shown in. Similarly, when the barrel aberration as shown in FIG. 11B occurs, the resizing processing unit 217 displays the image indicated by the closed curve CV2 as shown in FIG. The image is corrected to a square SQ as shown in (D).
In these cases, the resizing processing unit 217 associates the pixel in the image before correction with the pixel in the image after correction, sets the pixel before correction corresponding to the pixel after correction as the central pixel, and the central pixel and the surrounding pixels. The gradation to be displayed for the corrected pixel is determined based on the gradation to be displayed at each pixel in the reference region including the peripheral pixels as pixels.

For example, when the resize processing unit 217 determines the gradation of the corrected pixel PxS1 illustrated in FIG. 11C or 11D, the uncorrected pixel PxC1 illustrated in FIG. 11A or 11B. Is defined as the central pixel. Then, the resizing processing unit 217 determines the gradation to be displayed on the pixel PxS1 based on the gradation to be displayed on each pixel in the reference area Area1 including the pixel PxC1 that is the central pixel.
Similarly, when the resize processing unit 217 determines the gradation of the corrected pixel PxS2 illustrated in FIG. 11C or 11D, the pixel before correction illustrated in FIG. 11A or 11B. PxC2 is defined as the central pixel. Then, the resizing processing unit 217 determines the gradation to be displayed by the pixel PxS2 based on the gradation to be displayed by each pixel in the reference area Area2 including the pixel PxC2 that is the central pixel.

  11C and 11D, pixels with dark hatching indicate corrected pixels located in the first row, the seventh row, and the twelfth row in the image signal D. FIG. The pixels with dark hatching in A) and (B) indicate the uncorrected pixel (center pixel) corresponding to each of the corrected pixels, and are lightly hatched in FIGS. 11A and 11B. The pixel indicates a peripheral pixel corresponding to each of the central pixels.

As is clear from the example shown in FIG. 11, the degree of image expansion when barrel aberration occurs varies depending on the position of the line on the screen, and the position in the vertical direction (Y-axis direction) is at the end. As it approaches, the degree of image expansion increases. The degree of image shrinkage when pincushion aberration occurs varies depending on the position of the line on the screen, and the degree of image shrinkage increases as the position in the vertical direction (Y-axis direction) approaches the end.
Therefore, the number of lines of the imaging signal DS required when the resizing processing unit 217 generates the image signal D [m] varies depending on the position of the line corresponding to the image signal D [m] (value of m). For this reason, the length of time required for the resizing processing by the resizing processing unit 217 varies depending on the position of the line.

Here, the imaging signal DS corresponding to the p-th line is represented as an imaging signal DS [p] (p is a natural number satisfying 1 ≦ p ≦ P).
At this time, for example, in the example shown in FIG. 11, in order for the resizing processing unit 217 to generate the image signal D [1] corresponding to the first line, it corresponds to the first to fifth lines. The imaging signals DS [1] to DS [5] to be used are required. On the other hand, in order for the resizing processing unit 217 to generate the image signal D [7] corresponding to the seventh line, the imaging signal DS [8] corresponding to the eighth to tenth lines. ~ DS [10] is required. That is, the time length required for the resizing processing unit 217 to generate the image signal D [1] is longer than the time length required to generate the image signal D [7].

Hereinafter, one or a plurality of lines of imaging signals DS [p] that are necessary for generating the image signal D [m] are collectively referred to as imaging signals DGS [m].
For example, in the example shown in FIG. 11, the imaging signal DGS [1] required for generating the image signal D [1] is the imaging signal DS for five lines of the imaging signals DS [1] to DS [5]. [p], and the imaging signal DGS [7] necessary for generating the image signal D [7] is the imaging signal DS [p] for three lines of the imaging signals DS [8] to DS [10]. The imaging signal DGS [12] required for generating the image signal D [12] is the imaging signal DS [p] for five lines of the imaging signals DS [12] to DS [16].

  When the resize processing is completed and the image signal D is generated for each line, the resize processing unit 217 stores the generated image signal D [m] (image signal DGA [m]) for one line in the line buffer 22. At the same time, a write completion signal PtA indicating that the storage of the image signal D [m] in the line buffer 22 is completed is output.

<3. Image signal output>
Next, the relationship between the output timing of the imaging signal DS [p] from the imaging unit 10 and the output timing of the image signal D [m] from the image signal generation unit 20 will be described.
FIG. 12 shows that the imaging unit 10 outputs the imaging signal DS (DS [1] to DS [P]) in each of the vertical scanning periods Fs1 to Fs3 among a plurality of continuous vertical scanning periods Fs (Fs0 to Fs3). The image processing unit 21 generates an image signal D (D [1] to D [M]), that is, an image signal DGA (DGA [1] to DGA [M]), based on the timing to perform and the imaging signal DS. The generated image signal DGA is stored in the line buffer 22 and the image signal output unit 23 in each of the vertical scanning periods Fd1 to Fd3 among the plurality of continuous vertical scanning periods Fd (Fd0 to Fd3) 4 is a timing chart schematically showing the relationship between the timing of obtaining the image signal D, that is, the image signal DGB (DGB [1] to DGB [M]) from the image signal D and outputting it to the display unit 40. In the vertical scanning period Fs, a period in which the imaging signal DS is output is referred to as “frame of the imaging signal DS”. In the vertical scanning period Fd, the vertical effective data period DVI in which the image signal D can be output is referred to as a “frame of the image signal D”. Then, as shown in FIG. 12, the time from the start of the frame of the imaging signal DS to the start of the image signal D is referred to as a phase difference PD.

In FIG. 12, for convenience of explanation, the imaging signal DS [p] output in the vertical scanning periods Fs0 to Fs3 may be expressed separately from the imaging signals DS0 [p] to DS3 [p], respectively.
Further, as described with reference to FIG. 11, from the viewpoint of generation of the image signals D [1] to D [M] in the image processing unit 21, the imaging signals DS [1] to DS [P output from the imaging unit 10 are used. ] Are imaging signals DGS [1] to DGS [M]. Hereinafter, for convenience of explanation, the imaging signals DGS [m] output in the vertical scanning periods Fs0 to Fs3 may be distinguished and expressed as imaging signals DGS0 [m] to DGS3 [m], respectively.
Similarly, image signals D [m] (DGA [m] and DGA [m]) generated based on the imaging signals DGS0 [m] to DGS3 [m] are converted into image signals D0 [m] to D3 [, respectively. m] (DGA0 [m] to DGA3 [m], DGB0 [m] to DGB3 [m]).

As described above, the imaging unit 10 sequentially outputs the imaging signals DS [1] to DS [P] for each imaging horizontal synchronization signal SHsync. Further, when the supply of the imaging signal DS [p] corresponding to the imaging signal DGS [m] is started, the image processing unit 21 starts image processing for generating the image signal DGA [m]. That is, the timing at which the image processing unit 21 starts image processing for generating the image signal DGA [m] varies depending on the line.
In FIG. 12, the timing at which the imaging unit 10 supplies the imaging signals DGS [1] to DGS [M] to the image processing unit 21 is represented by a line L1. That is, the line L1 represents the state (timing) in which the image processing unit 21 sequentially starts image processing for generating each of the image signals DGA [1] to DGA [M] for each line. .
A state (timing) in which generation of the image signals DGA [1] to DGA [M] by the image processing unit 21 is completed and these are sequentially stored in the line buffer 22 for each line is represented by a line L2. The image signal output unit 23 outputs the image signal DGB [m] after the generation of the image signal DGA [m] is completed. Therefore, the image signal DGB [m] is not output at a time before the time indicated by the line L2. The line L2 is a line connecting the image signal generation times TC [1] to TC [M] described in FIG.

Further, when the image signal output unit 23 supplies the image signals DGB [1] to DGB [M] at an ideal timing for the display unit 40, that is, the highest frame rate (vertical scanning that the display unit 40 can display). When the image signals DGB [1] to DGB [M] are supplied so as to display at a frame rate when the time length of the period Fd is the standard vertical scanning time Td), the image signal output unit 23 outputs the image signal DGB [ The timing at which 1] to DGB [M] are sequentially output for each line is represented by a line L3. That is, the line L3 is assumed when the image signal output unit 23 outputs the image signal DGB [m] for one line for each horizontal scanning period Hd in which the display unit 40 can display an image for one line (that is, (When the frame rate of the display unit 40 is the second frame rate), the display unit 40 displays the timing at which the images indicated by the image signals DGB [1] to DGB [M] are displayed in line order for each horizontal scanning period Hd. It has a slope that increases by one line every horizontal scanning period Hd. That is, the line L3 assumes that all of the horizontal scanning periods Hd included in the vertical valid data period DVI are valid horizontal scanning periods Hd-A, and the invalid horizontal scanning period Hd- Assuming that D is included, the output time of the image signal D [m] on the premise that the output of the image signal D [m-1] of the m-1st row is completed (satisfaction of the second condition) The display preparation determination time TB [m] shown does not necessarily match.
The image signal output unit 23 outputs the image signal DGB [m] when the display unit 40 can display. Therefore, the image signal DGB [m] is not output at a time before the time indicated by the line L3.

In FIG. 12, the time required for image processing for generating the image signal DGA [m] for each line is defined as an image processing time UA. Hereinafter, for convenience of explanation, the image processing times UA corresponding to the image signals DGA0 [m] to DGA3 [m] may be distinguished and expressed as image processing times UA0 to UA3.
In FIG. 12, the time from when the image signal DGA [m] is stored in the line buffer 22 until it is output to the display unit 40 by the image signal output unit 23 is defined as a standby time UB. Hereinafter, for convenience of explanation, the waiting time UB corresponding to each of the image signals DGB0 [m] to DGB3 [m] may be distinguished and expressed as the waiting times UB0 to UB3.

As described above, the imaging signals DS [1] to DS [P] and the imaging signals DGS [1] to DGS [M] do not correspond one-to-one, but the image signal D [m corresponding to each line. The start interval of image processing for generating] may vary. For this reason, the line L1 is usually not a straight line but a broken line, but in FIG. 12, it is drawn as a straight line for convenience of illustration. When the line L1 is a straight line (for example, when the start point and the end point of the line L1 are connected to a straight line), the inclination of the line L1 is determined according to the first frame rate that is the frame rate of the imaging unit 10. .
Further, as described above, the number of lines of the imaging signal DS [p] included in each of the imaging signals DGS [1] to DGS [M] may vary depending on the position of the line. That is, as described with reference to FIG. 11, the image processing unit 21 may generate the image signal DGA [m] based on the imaging signal DGS [m] composed of the imaging signals DS [p] for three lines. The image signal DGA [m] may be generated based on the imaging signal DGS [m] composed of the imaging signals DS [p] for five lines. In the latter case, the image processing time UA is compared to the former case. Becomes longer. That is, the image processing time UA for the image processing unit 21 to generate the image signals DGA [1] to DGA [M] usually varies depending on the line position. For this reason, the line L2 is usually not a straight line but a broken line, but in FIG. 12, it is drawn as a straight line for convenience of illustration.

As shown in FIG. 12, the image processing unit 21 uses the image signal D1 [m] (DGA1 [m]) based on the imaging signal DS1 [p] (DGS1 [m]) output from the imaging unit 10 in the vertical scanning period Fs1. The line L2 indicating the time at which the image is generated is longer than the line L3 indicating the earliest time at which the display unit 40 can display the image indicated by the image signal D1 [m] (DGA1 [m]) in the vertical scanning period Fd1. Leading. Such a state in which the line L2 is temporally ahead of the line L3 is referred to as a “first state”.
That is, the first state refers to an image indicated by the image signal D [m] on the display unit 40 when the image processing unit 21 generates the image signal D [m] based on the imaging signal DS [p]. It is in a state that it is not ready to display. Here, when the display unit 40 is not ready to display the image signal D [m], for example, when the image signal D1 [m] is generated, the display unit 40 precedes the image signal D1. For example, the image indicated by the image signal D0 to be displayed is being displayed, and the display unit 40 cannot display the image indicated by the image signal D1 [m].
That is, in the first state, even if the image processing unit 21 generates the image signal D [m], the display unit 40 side for displaying the image signal D [m] is not ready in time, so the display The display of the image in the unit 40 is in a state of being delayed because the display preparation on the display unit 40 side becomes a bottleneck. In other words, the first state is that the image signal D [m] can be displayed promptly without delay at the timing when the display unit 40 can display the image signal D [m]. It is.

By the way, the time (cycle) required to display one screen on the display unit 40 is shorter than the time (cycle) required to image one screen on the imaging unit 10. The display delay in which display preparation becomes a bottleneck is gradually reduced and eliminated.
In FIG. 12, for convenience of illustration, only one set of vertical scanning period of one vertical scanning period Fs (Fs1) and one vertical scanning period Fd (Fd1) is shown as the first state. There may be multiple sets of vertical scan periods. In this case, the phase difference PD1 in the first state (the phase difference PD in the first state is denoted by the symbol PD1 as shown in FIG. 12) corresponds to the difference between the vertical scanning period Fd and the vertical scanning period Fs. The time to do becomes shorter. In other words, for each set of vertical scanning periods, the distance between the line L3 and the line L2 is generally shortened by a time corresponding to the difference between the vertical scanning period Fd and the vertical scanning period Fs.
Before the display delay on which the display preparation on the display unit 40 side becomes a bottleneck is eliminated, the line L2 precedes the line L3 in time. On the other hand, after the display delay in which the display preparation becomes a bottleneck is eliminated, the line L3 precedes the line L2. That is, the line L2 and the line L3 cross each other at the timing when the display delay that causes the display preparation to become a bottleneck is eliminated.
As described above, the line L2 may not be a straight line but a broken line. In this case, the cross between the line L2 and the line L3 may occur a plurality of times.

In the example shown in FIG. 12, the image processing unit 21 uses the image signal D2 [m] (DGA2 [m]) based on the imaging signal DS2 [p] (DGS2 [m]) output by the imaging unit 10 in the vertical scanning period Fs2. The line L2 indicating the generation time of the crossing line intersects the line L3 indicating the earliest time at which the display unit 40 can display the image indicated by the image signal D2 [m] (DGA2 [m]) in the vertical scanning period Fd2. ing. Such a state where the line L2 and the line L3 cross each other is referred to as a “second state”. When the cross between the line L2 and the line L3 occurs a plurality of times, the state in which such a cross first occurs is referred to as a “second state”. The time at which the line L2 and the line L3 cross is referred to as time Tth. When the cross between the lines L2 and L3 occurs a plurality of times, the time when the cross first occurs is defined as time Tth.
That is, the second state refers to displaying an image indicated by the image signal D [m] on the display unit 40 when the image processing unit 21 generates the image signal D [m] based on the imaging signal DS [p]. When the display unit 40 can display the image indicated by the image signal D [m] from the state where it is not ready to perform (the state where the line L2 always precedes the line L3 in time), the image processing unit 21 indicates a transition to a state where there is a case where image processing for generating the image signal D [m] is not completed (a state where the line L3 may precede the line L2 in time). .
That is, in the second state, before the time Tth, the image signal D [m] is displayed without delay at the timing at which the display unit 40 can display the image indicated by the image signal D [m]. On the other hand, after the time Tth, the image signal D [m] is generated even at the time when the image signal D [m] can be displayed on the display unit 40 side. Since the image processing in the image processing unit 21 for doing so is not in time, the display of the image in the display unit 40 may be delayed due to the image processing of the image processing unit 21 becoming a bottleneck.
In this second state, the phase difference PD2 (the phase difference PD in the second state is denoted by the symbol PD2 as shown in FIG. 12) is shorter than the phase difference PD1 as shown in FIG.

  After time Tth, the timing generator 32 inserts the invalid horizontal scanning period Hd-D in the vertical valid data period DVI, and the output timing of the image signal D [m] from the image signal output unit 23 (display unit) The display timing of the image indicated by the image signal D [m] at 40 is adjusted. As a result, the display unit 40 waits for completion of image processing for generating the image signal D [m], and immediately after the image signal D [m] is generated (within the time equal to or less than the horizontal scanning period Hd). ), An image indicated by the image signal D [m] can be displayed. That is, after the time Tth, by adjusting the output timing of the image signal D [m] (DGB [m]) from the image signal output unit 23 by inserting the invalid horizontal scanning period Hd-D, the display unit 40 side Is made to follow the timing of completion of the image processing in the image processing unit 21 with the accuracy of the horizontal scanning period Hd.

As shown in FIG. 12, in the vertical scanning period Fs3, the line L3 indicating the earliest time at which the display unit 40 can display the image indicated by the image signal D3 [m] (DGA3 [m]) in the vertical scanning period Fd3. Based on the output imaging signal DS3 [p] (DGS3 [m]), the image processing unit 21 temporally precedes the line L2 indicating the time when the image signal D3 [m] (DGA3 [m]) is generated. ing. Such a state in which the line L3 is temporally ahead of the line L2 is referred to as a “third state”. As described above, the line L2 is not a straight line but a broken line, and the crossing of the line L2 and the line L3 may occur a plurality of times. In such a case, the state in the set of vertical scanning periods (Fs and Fd) started after time Tth is referred to as a third state.
That is, the third state refers to image processing for the image processing unit 21 to generate the image signal D [m] when the display unit 40 completes preparation for displaying the image indicated by the image signal D [m]. It is a state that the case where it is not completed has always occurred.
That is, in the third state, even when the preparation on the display unit 40 side for displaying the image signal D [m] is completed, the image processing in the image processing unit 21 for generating the image signal D [m] is performed. Since the situation is not always in time, the display of the image on the display unit 40 is delayed due to the image processing of the image processing unit 21 becoming a bottleneck.
In this third state, the phase difference PD3 (the phase difference PD in the third state is denoted by the symbol PD3 as shown in FIG. 12) is shorter than the phase difference PD2 as shown in FIG. . The phase difference PD1 in the first state is larger than the image processing time UA (more specifically, the maximum value of the image processing times UA [1] to UA [M]), and the phase difference in the third state. PD3 is shorter than the image processing time UA (more specifically, the maximum value of the image processing times UA [1] to UA [M]).

Even in the third state, the timing generator 32 adjusts the display timing of the image indicated by the image signal D [m] on the display unit 40 by inserting the invalid horizontal scanning period Hd-D into the vertical valid data period DVI. To do. Thereby, the display timing on the display unit 40 side can be made to follow the completion timing of the image processing in the image processing unit 21 with the accuracy of the horizontal scanning period Hd.
As described above, the image processing time UA varies from line to line. However, the fluctuation range is sufficiently smaller than the vertical scanning period Fs. Therefore, in a state where the output timing of the image signal D [m] (display timing on the display unit 40 side) is made to follow the completion timing of the image processing in the image processing unit 21, the imaging unit 10 captures the imaging signal DS3. And the time length of the period during which the image signal output unit 23 outputs the image signal DGB3 are substantially the same. In other words, in the third state, the timing control unit 30 outputs the image signal D [m] so that the frame rate of the display unit 40 becomes the first frame rate that is the frame rate of the imaging unit 10. Timing is adjusted (second timing control).
In FIG. 12, for convenience of illustration, only one set of vertical scanning period of one vertical scanning period Fs (Fs3) and one vertical scanning period Fd (Fd3) is shown as the third state. Actually, there are a plurality of sets of vertical scanning periods. In the third state, in each of a plurality of sets of vertical scanning periods, the time length during which the imaging unit 10 outputs the imaging signal DS3 and the period during which the image signal output unit 23 outputs the image signal DGB3. The timing at which the image signal D [m] is output is adjusted such that the time length of the image signal D [m] is substantially the same. That is, in the third state, the image signal D [m] is set so that the frame rate of the display unit 40 becomes the first frame rate that is the frame rate of the imaging unit 10 in each of a plurality of sets of vertical scanning periods. The output timing is adjusted. Therefore, in the third state, the phase difference PD3 has substantially the same time length in each of the plurality of sets of vertical scanning periods. ,

  Hereinafter, with reference to FIG. 12 and FIG. 13, the image signal DS1 (DGS1) output by the imaging unit 10 in the vertical scanning period Fs1 and the image signal output by the image signal generation unit 20 to the display unit 40 in the vertical scanning period Fd1. The first state will be described by taking the relationship with D1 (DGB1) as an example.

FIG. 13 is a timing for explaining the relationship between the imaging signal DS1 [p] (imaging signal DGS1 [m]) and the image signal D1 [m] (image signal DGA1 [m] and image signal DGB1 [m]). It is a chart.
In FIG. 13 and FIGS. 14 and 15 described later, for the sake of simplicity, the number of lines of the image indicated by the imaging signal DS is 5 lines (P = 5), and the number of lines of the image indicated by the image signal D is 4. Assume the case of line (M = 4). In the example shown in FIGS. 13 to 15, the imaging signal DGS [1] includes the imaging signals DS [1] and DS [2], and the imaging signal DGS [2] is the imaging signals DS [2] and DS. When [3] is included, the imaging signal DGS [3] includes the imaging signals DS [3] and DS [4], and the imaging signal DGS [4] includes the imaging signals DS [4] and DS [5] Is assumed. That is, in the example illustrated in FIGS. 13 to 15, the image signal D [1] is generated based on the imaging signals DS [1] and DS [2], and the image signal D [2] is generated based on the imaging signal DS [2]. ] And DS [3], the image signal D [3] is generated based on the imaging signals DS [3] and DS [4], and the image signal D [4] is generated based on the imaging signal DS [4]. ] And DS [5] are assumed. In the examples shown in FIGS. 13 to 15, it is assumed that the line L2 and the line L3 are crossed only once.

As illustrated in FIG. 13, when the imaging unit 10 starts outputting an imaging signal DGS1 [m] composed of the imaging signals DS1 [m] and DS1 [m + 1], the image processing unit 21 Generation of the image signal DGA1 [m] is started based on DGS1 [m]. Then, the image processing unit 21 completes the generation of the image signal DGA1 [m] after the image processing time UA1 [m] has elapsed from the start of the image processing, and stores it in the line buffer 22.
On the other hand, the example shown in FIG. 13 illustrates the first state described above, and the line L2 precedes the line L3 in time. That is, in the example shown in FIG. 13, at the timing when the generation of the image signal DGA1 [m] by the image processing unit 21 is completed, the display unit 40 is not ready to display the image indicated by the image signal DGB1 [m]. In other words, at the timing when the generation of the image signal DGA1 [m] by the image processing unit 21 is completed, the output permission pulse PL [m] is not output from the output control unit 31.
Therefore, the image signal output unit 23 outputs the image signal DGB1 [m] for the waiting time UB1 [m] until the first horizontal scanning period Hd1 [m] after the output permission pulse PL [m] is output. After that, the image signal DGB1 [m] is output in the horizontal scanning period Hd1 [m].

  The first state illustrated in FIG. 13 is a case where the display preparation by the display unit 40 is not in time until the image processing by the image processing unit 21 is completed. In other words, the generation of the image signal DGA1 [m] by the image processing unit 21 is completed before the horizontal scanning period Hd1 [m] is started, and the image signal DGB1 [m] can be output from the image signal output unit 23. It is in a state. Therefore, in the first state illustrated in FIG. 13, all the horizontal scanning periods Hd included in the vertical effective data period DVI of the vertical scanning period Fd1 are effective horizontal scanning periods Hd-A. That is, in the first state, the time length of the vertical scanning period Fd becomes the standard vertical scanning time Td.

As described above, in the first state illustrated in FIG. 13, the image processing for generating the image signal D1 is completed with sufficient margin, but the display preparation on the display unit 40 side becomes a bottleneck. Thus, the display on the display unit 40 is delayed.
Therefore, the delay time ΔT1 from when the imaging unit 10 outputs the imaging signal DS1 to when the display unit 40 displays the image indicated by the image signal D1 is the time required for image processing in the image signal generation unit 20 (image processing). Time UA) and the time (waiting time UB) for waiting for display preparation in the display unit 40 after completion of image processing.

Next, referring to FIG. 12 and FIG. 14, an image signal DS2 (DGS2) output by the image pickup unit 10 in the vertical scanning period Fs2 and an image output by the image signal generation unit 20 to the display unit 40 in the vertical scanning period Fd2. The second state will be described by taking the relationship with the signal D2 (DGB2) as an example.
FIG. 14 is a timing for explaining the relationship between the imaging signal DS2 [p] (imaging signal DGS2 [m]) and the image signal D2 [m] (the image signal DGA2 [m] and the image signal DGB2 [m]). It is a chart.

  As illustrated in FIG. 14, when the imaging unit 10 starts outputting an imaging signal DGS2 [m] composed of imaging signals DS2 [m] and DS2 [m + 1], the image processing unit 21 Generation of the image signal DGA2 [m] is started based on DGS2 [m]. Then, the image processing unit 21 completes the generation of the image signal DGA2 [m] after the image processing time UA2 [m] has elapsed from the start of the image processing, and stores it in the line buffer 22.

  In the example shown in FIG. 14, the image signals D2 [1] and D2 [2] are the image signals D [m] output by the image signal output unit 23 before the time Tth, and the image signals D2 [3] and D2 [3] It is assumed that D2 [4] is an image signal D [m] output from the image signal output unit 23 after time Tth.

Before the time Tth, the line L2 precedes the line L3 in time. That is, before the time Tth, the output permission pulse PL [m] is not output from the output control unit 31 at the timing when the generation of the image signal DGA2 [m] by the image processing unit 21 is completed.
Therefore, before the time Tth, the image signal output unit 23 outputs the image signal DGB2 for the waiting time UB2 [m] until the first horizontal scanning period Hd2 [m] after the output permission pulse PL [m] is output. After waiting for the output of [m], the image signal DGB2 [m] is output in the horizontal scanning period Hd2 [m].
In the example shown in FIG. 14, the image signal output unit 23 waits for the output of the image signal DGB2 [1] for the standby time UB2 [1] after the image signal DGA2 [1] is generated, and then the horizontal scanning period Hd2 In [1], the image signal DGB2 [1] is output. Similarly, after the image signal DGA2 [2] is generated, the image signal output unit 23 waits for the output of the image signal DGB2 [2] for the standby time UB2 [2], and then in the horizontal scanning period Hd2 [2]. The image signal DGB2 [2] is output.

On the other hand, after time Tth, line L3 usually precedes line L2 in time. When the line L3 precedes the line L2, when the image processing unit 21 generates the image signal DGA2 [m], the display unit 40 immediately (in the immediately following horizontal scanning period Hd), the image signal DGB2 The image indicated by [m] can be displayed. Therefore, when the line L3 precedes the line L2, the output control unit 31 outputs the output permission pulse PL [m] at the timing when the generation of the image signal DGA2 [m] by the image processing unit 21 is completed. The
In the example shown in FIG. 14, the image signal output unit 23 generates the image signal in the first horizontal scanning period Hd2 [3] after the image signal DGA2 [3] is generated and the output permission pulse PL [3] is output. DGB2 [3] is output.
Further, in the example shown in this figure, the image signal DGA2 [4] is generated after the start of the horizontal scanning period Hd2 [4]. Therefore, the image signal output unit 23 generates the image signal DGA2 [4] and outputs the image signal DGB2 [4] in the first horizontal scanning period Hd2 [5] after the output permission pulse PL [4] is output. Is output. Then, the timing generator 32 sets the horizontal scanning period Hd2 [4] as an invalid horizontal scanning period Hd-D.

As described above, in the second state illustrated in FIG. 14, display delay due to image processing occurs after the time Tth. Therefore, the invalid horizontal scanning period Hd is included in the vertical valid data period DVI of the vertical scanning period Fd2. -D is inserted. That is, in the second state, the time length of the vertical scanning period Fd is the sum of the standard vertical scanning time Td and the extended vertical scanning time Tex.
The delay time ΔT2 from when the imaging unit 10 outputs the imaging signal DS2 to when the display unit 40 displays the image indicated by the image signal D2 is required for image processing in the image signal generation unit 20 before time Tth. The total time of the time (image processing time UA) and the time for waiting for display preparation on the display unit 40 (waiting time UB), but after time Tth, the line L3 precedes the line L2 in time. In this case, only the time required for image processing (image processing time UA) in the image signal generation unit 20 is obtained. For this reason, the delay time ΔT2 according to the second state is shorter than the delay time ΔT1 according to the first state.

Next, referring to FIG. 12 and FIG. 15, an image signal DS3 (DGS3) output by the imaging unit 10 in the vertical scanning period Fs3 and an image output by the image signal generation unit 20 to the display unit 40 in the vertical scanning period Fd3. The third state will be described by taking the relationship with the signal D3 (DGB3) as an example.
FIG. 15 is a timing for explaining the relationship between the imaging signal DS3 [p] (imaging signal DGS3 [m]) and the image signal D3 [m] (the image signal DGA3 [m] and the image signal DGB3 [m]). It is a chart.

  As illustrated in FIG. 15, when the imaging unit 10 starts outputting the imaging signal DGS3 [m] including the imaging signals DS3 [m] and DS3 [m + 1], the image processing unit 21 Generation of the image signal DGA3 [m] is started based on DGS3 [m]. Then, the image processing unit 21 completes generation of the image signal DGA3 [m] after the image processing time UA3 [m] has elapsed from the start of the image processing, and stores this in the line buffer 22.

In the third state, the line L3 usually precedes the line L2 in time. When the line L3 precedes the line L2 in time, when the image processing unit 21 generates the image signal DGA3 [m], the display unit 40 immediately (in the immediately following horizontal scanning period Hd) displays the image signal DGB3. The image indicated by [m] can be displayed. Therefore, in this case, the output permission pulse PL [m] is output from the output control unit 31 at the timing when the generation of the image signal DGA3 [m] by the image processing unit 21 is completed.
Specifically, in the example shown in FIG. 15, the image signal output unit 23 generates the image signal DGA3 [1] and outputs the first horizontal scanning period Hd3 [3 after the output permission pulse PL [1] is output. ], The image signal DGB3 [1] is output, the image signal DGA3 [2] is generated, and the image signal DGB3 is generated in the first horizontal scanning period Hd3 [5] after the output permission pulse PL [2] is output. [2] is output, the image signal DGA3 [3] is generated, and the image signal DGB3 [3] is output in the first horizontal scanning period Hd3 [7] after the output permission pulse PL [3] is output. The image signal DGB3 [4] is output in the first horizontal scanning period Hd3 [9] after the image signal DGA3 [4] is generated and the output permission pulse PL [4] is output. In this case, the timing generator 32 sets the horizontal scanning periods Hd3 [1], Hd3 [2], Hd3 [4], Hd3 [6], and Hd3 [8] as the invalid horizontal scanning period Hd-D. .

  As described above, in the third state illustrated in FIG. 15, the display delay due to the image processing occurs, and therefore, the invalid horizontal scanning period Hd-D is included in the vertical effective data period DVI of the vertical scanning period Fd3. Inserted. As a result, in the third state, the length of the vertical scanning period Fd is equal to that of the horizontal scanning period Hd so that the display unit 40 can perform display in synchronization with the imaging signal DS output in the vertical scanning period Fs. Adjusted with accuracy. That is, in the third state, when viewed roughly, the vertical scanning period Fd is adjusted to be substantially the same time as the vertical scanning period Fs.

In the third state, when the line L3 precedes the line L2, the display unit 40 is in the first horizontal scanning period Hd after the image processing unit 21 generates the image signal D [m]. An image indicated by the image signal D [m] is displayed. Therefore, the delay time ΔT3 from when the imaging unit 10 outputs the imaging signal DS3 to when the display unit 40 displays the image indicated by the image signal D3 is the time required for image processing in the image signal generation unit 20 (image processing). It is almost the same as time UA). Specifically, in the third state, a delay time from when the imaging unit 10 starts outputting the imaging signal DS [p] to when the display unit 40 starts displaying an image indicated by the image signal D [m]. ΔT3 and the image processing time UA required for the image processing unit 21 to generate the image signal D [m] are equalized with the accuracy of the horizontal scanning period Hd.
For this reason, in the third state, the delay from imaging by the imaging unit 10 to display by the display unit 40 can be minimized with the accuracy of the horizontal scanning period Hd. In this case, the delay time ΔT3 is shorter than the delay time ΔT1 according to the first state and is equal to or shorter than the delay time ΔT2 according to the second state.

In addition, as described above, the time (cycle) required for displaying one screen on the display unit 40 is shorter than the time (cycle) required for imaging one screen on the imaging unit 10. For this reason, even when the imaging display device 1 operates in the first state and the display preparation on the display unit 40 side causes a display delay that becomes a bottleneck, the display is performed every vertical scanning period Fs. The delay is reduced. That is, after the imaging display device 1 initially operates in the first state, the imaging display device 1 finally moves to the operation in the third state, and the operation in the third state is started. Can maintain the operation in the third state. As a result, the display timing on the display unit 40 side can be made to follow the completion timing of the image processing in the image processing unit 21 with the accuracy of the horizontal scanning period Hd.
For this reason, when the imaging display device 1 starts the operation in the live view mode, the delay time from the imaging by the imaging unit 10 to the image display on the display unit 40 is minimized except immediately after the operation in the live view mode is started. It is possible to maintain a converted state.

<4. Effects of First Embodiment>
As described above, in the imaging display device 1 according to the present embodiment, the display timing on the display unit 40 side is adjusted by the image processing unit 21 by adjusting the output timing of the image signal D [m]. The completion timing is made to follow the accuracy of the horizontal scanning period Hd.
Hereinafter, in order to clarify the advantages of the imaging display apparatus 1 as described above, the comparative imaging display apparatus illustrated in FIG. 16 will be described.

  FIG. 16 is a timing chart for explaining the output timing of the image signal D [m] in the imaging display device according to the comparative example. As shown in this figure, in the imaging display device according to the comparative example, the time length of the vertical scanning period Fd is fixed (fixed to the standard vertical scanning time Td), and the operation in the live view mode is executed. This is different from the imaging display device 1 according to the present embodiment that performs the operation shown in FIG.

  As shown in FIG. 16, in the comparative imaging display device, the length of the vertical scanning period Fd is constant. In contrast, for example, in the vertical scanning period Fd started after the end of the vertical scanning period Fs, an image indicated by the image signal DGB generated based on the imaging signal DS output in the vertical scanning period Fs is displayed. Assume the case. Specifically, the proportional imaging display device displays an image corresponding to the imaging signal DS1 output in the vertical scanning period Fs1 in the vertical scanning period Fd2 that starts after the vertical scanning period Fs1, and performs vertical scanning. An image corresponding to the imaging signal DS2 output in the period Fs2 is displayed in the vertical scanning period Fd4 started after the vertical scanning period Fs2, and an image corresponding to the imaging signal DS3 output in the vertical scanning period Fs3 is displayed. The display is performed in the vertical scanning period Fd5 started after the vertical scanning period Fs3. In the example shown in this figure, the vertical scanning period Fd3 is started before the end of the vertical scanning period Fs2. For this reason, in the example shown in this figure, in the vertical scanning period Fd3, an image corresponding to the imaging signal DS1 displayed in the vertical scanning period Fd2 (an image indicated by the image signal DGB1) is displayed.

As described above, in the comparative imaging display apparatus illustrated in FIG. 16, the delay time from when the imaging unit 10 outputs the imaging signal DS to when the display unit 40 displays the image indicated by the image signal DGB is The time length is longer than the vertical scanning period Fd.
Further, in the case of the example shown in FIG. 16, the image displayed in the vertical scanning period Fd2 (the image indicated by the image signal DGB1) is displayed again in the vertical scanning period Fd3. In this case, since the display time of each image varies in units of the vertical scanning period Fd, there is a possibility that the image is visually recognized as flickering. As a result, the display quality of the display unit 40 is greatly deteriorated. If no image is displayed in the vertical scanning period Fd3, there are a vertical scanning period Fd in which an image is displayed and a vertical scanning period Fd in which no image is displayed. The user of the imaging display device 1 may be visually recognized.

In contrast, in the imaging display device 1 according to the present embodiment, the image signal D [m] is output from the image signal output unit 23 when the first condition and the second condition are satisfied, and the first condition or the first condition is output. When the two conditions are not satisfied, the output timing of the image signal D [m] from the image signal output unit 23 is adjusted with the accuracy of the horizontal scanning period Hd by inserting the invalid horizontal scanning period Hd-D. That is, in the imaging display device 1 according to the present embodiment, in the first horizontal scanning period Hd after the image processing unit 21 generates the image signal D [m], the display unit 40 displays the image indicated by the image signal D [m]. Can be displayed. Thereby, the delay from the imaging by the imaging unit 10 to the display by the display unit 40 can be minimized with the accuracy of the horizontal scanning period Hd.
Further, in the imaging display device 1 according to the present embodiment, the invalid horizontal scanning period Hd-D is inserted into the vertical effective data period DVI, thereby making the time length of the vertical scanning period Fd variable and the vertical scanning period Fs. A state that is substantially the same as the time length can be maintained. For this reason, it is possible to realize a high-quality display with reduced display flicker and the like.
Further, according to the present embodiment, for example, when there is a change in image processing time due to a change in image processing technique or when the image processing time UA varies from line to line, the imaging unit 10 has a different frame rate. When the display unit 40 is replaced with another one or when the display unit 40 is replaced with one having a different frame rate, the phase difference between the imaging unit 10 and the display unit 40, the frame rate of the imaging unit 10, and the display unit 40 can display Even when part or all of the highest frame rate changes, the phase difference PD can be automatically converged to a length equal to or shorter than the image processing time UA.
<B. Second Embodiment>

In the first embodiment described above, as shown in FIG. 6, by inserting the invalid horizontal scanning period Hd-D in the vertical valid data period DVI, the output timing of the image signal D [m] is set to the horizontal scanning period Hd. The time length of the horizontal scanning period Hd was set to a fixed length by adjusting with accuracy.
On the other hand, in the imaging display device according to the second embodiment, the time length of the horizontal scanning period Hd is made variable, and the output timing of the image signal D [m] is adjusted by, for example, the cycle of the display dot clock signal DCLK. This is different from the imaging display device 1 according to the first embodiment.
Hereinafter, the imaging display device according to the second embodiment will be described with reference to FIGS. 17 to 19. In addition, about the element which an effect | action and a function are equivalent to 1st Embodiment in 2nd Embodiment illustrated below, the code | symbol referred by the above description is diverted and each detailed description is abbreviate | omitted suitably (below). The same applies to the modification described in the above.

FIG. 17 illustrates an output control signal CTR, an enable signal DEnb, and a display horizontal synchronization signal DHsync2 generated by the timing control unit 30 (the output control unit 31 and the timing generator 32) included in the imaging display device according to the second embodiment. It is explanatory drawing for demonstrating this relationship.
In the timing control unit 30 included in the imaging display device according to the second embodiment, the timing generator 32 generates a display horizontal synchronization signal DHsync2 having a horizontal synchronization pulse PlsH having a variable period instead of the display horizontal synchronization signal DHsync. The configuration is the same as that of the imaging display device 1 according to the first embodiment (see FIG. 6) except that the display vertical synchronization signal DVsync2 having the vertical synchronization pulse PlsV having a variable period is generated instead of the display vertical synchronization signal DVsync. Has been.

As illustrated in FIG. 17, the output control unit 31 according to the second embodiment performs the later time of the image processing determination time TA [m] and the display preparation determination time TB [m], as in the first embodiment. (In this figure, since the second aspect described above is employed, the output permission pulse PL [m] is set in the output control signal CTR at the image processing determination time TA [m]).
As shown in FIG. 17, the timing generator 32 according to the second embodiment has a fixed time length from the timing when the output permission pulse PL [m] is set in the output control signal CTR output by the output control unit 31. After the elapse of the reference front porch time TP, the horizontal synchronization pulse PlsH is output as the display horizontal synchronization signal DHsync2.

Therefore, when the generation of the image signal D [m] is completed by the display preparation determination time TB [m] and the image signal generation time TC [m] has elapsed (Case-1), the horizontal front porch The time length of the period DHf is the reference front porch time TP.
On the other hand, when the generation of the image signal D [m] is not completed by the display preparation determination time TB [m], that is, the image signal generation time TC [m] comes after the display preparation determination time TB [m]. In the case (Case-2), the time length of the horizontal front porch period DHf is determined from the reference front porch time TP and the display preparation determination time TB [m] to the image signal generation time TC [m] (image processing determination time TA). [m]) and the total length of the extended front porch time TPX, which is the length of time.

As described above, in the timing generator 32 according to the second embodiment, the output control unit 31 determines that the output preparation of the image signal D [m] is completed, and outputs the output permission pulse PL [m] as the output control signal CTR. Waiting for the output, the horizontal scanning period Hd [m] is started only after the reference front porch time TP after the output permission pulse PL [m] is output. In other words, the timing generator 32 according to the second embodiment extends the horizontal front porch period DHf until the preparation for outputting the image signal D [m] is completed.
Therefore, the image signal output unit 23 outputs the image signal D [m] in the horizontal scanning period Hd [m] even when the image processing of the image signal D [m] in the image processing unit 21 is delayed. It becomes possible to do. In this case, the delay time from when the imaging unit 10 outputs the imaging signal DGS [m] until the display unit 40 displays an image based on the image signal D [m] is minimum with the accuracy of the display dot clock signal DCLK. Will be converted.

  FIG. 18 shows the imaging display device according to the second embodiment in a state where display delay in the display unit 40 becomes a bottleneck (ie, the second state described in FIG. 14). It is a timing chart for explaining operation. Further, FIG. 19 shows an imaging display according to the second embodiment in a state where the image processing of the image processing unit 21 is a bottleneck and the display is delayed (that is, the third state described in FIG. 15). It is a timing chart for demonstrating operation | movement of an apparatus. In FIGS. 18 and 19, the symbols described in FIGS. 12 to 15 are used.

In FIG. 18, for example, the image signal DGA2 [3] is generated before the enable signal DEnb falls in the horizontal scanning period Hd2 [2]. Therefore, the output permission pulse PL [3] is output at the falling timing of the enable signal DEnb in the horizontal scanning period Hd2 [2]. In this case, the time length of the horizontal front porch period DHf in the horizontal scanning period Hd2 [2] is the reference front porch time TP.
On the other hand, in the example shown in this drawing, the timing at which the image signal DGA2 [4] is generated is later than the falling timing of the enable signal DEnb in the horizontal scanning period Hd2 [3]. Therefore, the output permission pulse PL [4] is output at the timing when the image signal DGA2 [4] is generated. In this case, the time length of the horizontal front porch period DHf of the horizontal scanning period Hd2 [3] is the reference front porch time TP and the extended front porch time TPX (timing of the fall of the enable signal DEnb in the horizontal scanning period Hd2 [3]. To the time until the output permission pulse PL [4] is output). That is, the horizontal scanning period Hd is extended according to the state of the image processing after the time Tth when the display delay in which the display preparation in the display unit 40 becomes a bottleneck is eliminated.

  In FIG. 19, the timing at which the image signal DGA3 [m] is generated is after the falling timing of the enable signal DEnb in the horizontal scanning period Hd3 [m−1]. Therefore, the output permission pulse PL [m] is output at the timing when the image signal DGA3 [m] is generated. In this case, the time length of the horizontal front porch period DHf of the horizontal scanning period Hd3 [m] is the reference front porch time TP and the extended front porch time TPX (timing of the fall of the enable signal DEnb in the horizontal scanning period Hd3 [m]. To the time until the output permission pulse PL [m] is output). That is, in a state where the image processing of the image processing unit 21 is a bottleneck and the display is delayed (third state), the horizontal scanning period Hd is extended according to the state of the image processing. .

As is apparent from FIGS. 18 and 19, in the second embodiment, there is no invalid horizontal scanning period Hd-D, and all the horizontal scanning periods Hd are effective horizontal scanning periods Hd-A.
In the second embodiment, since the horizontal scanning period Hd is variable in units of the display dot clock signal DCLK, for example, the vertical scanning period Fd also has a variable time length.

<C. Modification>
Each of the above forms can be variously modified. Specific modifications are exemplified below. Two or more aspects arbitrarily selected from the following examples can be appropriately combined within a range that does not contradict each other. Note that, in the modified example described below, in order to avoid duplication of description, description of points in common with the above-described embodiment of the present invention is omitted.

<Modification 1>
In the above-described embodiment, the case where the display unit 40 includes the liquid crystal panel 42 has been exemplified. However, the present invention is not limited to such a mode, and may be an OLED (organic light emitting diode) panel, a plasma display panel, or the like. A display element may be used.

<Modification 2>
In the embodiment and the modification described above, data transmission between the image processing circuit 100 and the display unit 40 is performed by a parallel interface, but may be performed by a low voltage differential (LVDS) serial interface.

<Modification 3>
In the embodiment and the modification described above, the vertical scanning period Fs defined by the imaging vertical synchronization signal SVsync has a time length equal to or longer than the vertical scanning period Fd defined by the display vertical synchronization signal DVsync (or DVsync2). However, the present invention is not limited to such an embodiment, and the vertical scanning period Fs may have a shorter time length than the vertical scanning period Fd.

<Modification 4>
In the embodiment and the modification described above, the output control unit 31 uses the image signal D based on the write completion signal PtA output from the image processing unit 21 and the output completion signal PtB output from the image signal output unit 23. It is determined whether or not the preparation for outputting [m] is completed. However, the present invention is not limited to such an aspect, and the output control unit 31 periodically refers to the line buffer 22. By determining that the image signal D [m] is recorded in the line buffer 22 and that the image signal D [m−1] is read from the line buffer 22, the image signal D [m] is obtained. It may be determined whether or not preparation for output has been completed.

<Modification 5>
In the embodiment and the modification described above, the display unit 40 is built in the imaging display device 1, but the present invention is not limited to such a mode, and the display unit 40 is connected to the outside of the digital camera. You may comprise as a finder (display apparatus) etc.

<Modification 6>
In the embodiment and the modification described above, the case where the image processing time UA [m] fluctuates for each line has been described as an example. However, the present invention is not limited to such a mode, and the image processing time UA [m] may be the same between lines.

<D. Application example>
The imaging display device 1 exemplified in the above embodiments can be used in various electronic devices. For example, the imaging display device 1 may be configured as an electronic device (display device) such as a projector device, a HUD (head-up display), or an HMD (head-mounted display). In addition, a display device that performs live view can be applied to, for example, electronic binoculars, electronic glasses, electron microscopes, medical electronic glasses finder, in-vehicle back monitor, in-vehicle side mirror monitor, etc. The delay from display to display can be reduced. In addition, in the aspect of the display device, the imaging unit 10 is not necessarily provided. That is, the image processing circuit 100 to which the imaging signal DS is supplied and the display unit 40 may be regarded as a display device.

DESCRIPTION OF SYMBOLS 1 ... Imaging display apparatus, 10 ... Imaging part, 11 ... Imaging optical system, 12 ... Image sensor, 13 ... Timing generator, 20 ... Image signal generation part, 21 ... Image processing part, 22 ... Line buffer, 23... Image signal output unit, 30... Timing control unit, 31... Output control unit, 32... Timing generator, 33. 42... Liquid crystal panel, 50... CPU, 60.

Claims (7)

Based on the imaging signal output by the imaging unit that images the subject, an image signal indicating an image to be displayed on each line of the display unit capable of displaying an image at a higher frame rate than the imaging unit is generated, and the generated An image processing method for outputting an image signal to the display unit,
The time from when the imaging unit outputs the imaging signal to when the image signal is generated is defined as an image processing time.
The time from the start of the frame of the imaging signal to the start of the frame of the image signal output to the display unit is a phase difference,
The frame rate of the imaging unit is a first frame rate,
When the highest frame rate that can be displayed on the display unit is the second frame rate,
When the phase difference is longer than the image processing time,
By outputting the image signal to the display unit so that the frame rate of the display unit becomes the second frame rate, the phase difference is gradually reduced,
After the phase difference becomes less than the image processing time,
Outputting the image signal to the display unit such that a frame rate of the display unit becomes the first frame rate;
An image processing method. When the phase difference is longer than the image processing time,
After generating the image signal, wait until an image indicated by the generated image signal can be displayed on the display unit, and output the generated image signal to the display unit.
The image processing method according to claim 1. The display unit operates in synchronization with a horizontal synchronization pulse output at a constant period,
The generated image signal is output to the display unit in synchronization with the horizontal synchronization pulse,
After the phase difference is equal to or less than the image processing time, when the image signal generation time for generating the image signal is after the displayable time at which the image indicated by the image signal can be displayed on the display unit,
The display unit outputs the image signal in synchronization with a horizontal synchronization pulse that is output first after the image signal generation time.
The image processing method according to claim 1, wherein: The display unit operates in synchronization with a horizontal synchronization pulse,
The generated image signal is output to the display unit in synchronization with the horizontal synchronization pulse,
After the phase difference is equal to or less than the image processing time, when the image signal generation time for generating the image signal is after the displayable time at which the image indicated by the image signal can be displayed on the display unit,
Until the image signal generation time, the output of the horizontal synchronization pulse is stopped and the output of the image signal is stopped,
After the image signal generation time, the horizontal synchronization pulse is output, and the image signal is output in synchronization with the output horizontal synchronization pulse.
The image processing method according to claim 1, wherein: An imaging unit that images a subject and outputs an imaging signal;
Based on the imaging signal, an image signal indicating an image to be displayed on each line of the display unit capable of displaying an image at a higher frame rate than the imaging unit is generated, and the generated image signal is output to the display unit An image signal generator to
A timing control unit for controlling the timing at which the image signal generation unit outputs the image signal;
With
The time from when the imaging unit outputs the imaging signal until the image signal generation unit generates the image signal is defined as an image processing time.
The time from the start of the frame of the imaging signal to the start of the frame of the image signal output to the display unit is a phase difference,
The frame rate of the imaging unit is a first frame rate,
When the highest frame rate that can be displayed on the display unit is the second frame rate,
The timing controller is
In the case where the phase difference is longer than the image processing time,
First timing control for gradually reducing the phase difference by causing the image signal generation unit to output the image signal so that the frame rate of the display unit becomes the second frame rate;
After the phase difference is equal to or less than the image processing time,
A second timing control for causing the image signal generation unit to output the image signal so that a frame rate of the display unit becomes the first frame rate;
Is feasible,
An imaging apparatus characterized by that. Based on the imaging signal output by the imaging unit that images the subject, an image signal indicating an image to be displayed on each line of the display unit capable of displaying an image at a higher frame rate than the imaging unit is generated, and the generated An image signal generation unit that outputs an image signal to the display unit;
A timing control unit for controlling the timing at which the image signal generation unit outputs the image signal;
With
The time from when the imaging unit outputs the imaging signal until the image signal generation unit generates the image signal is defined as an image processing time.
The time from the start of the frame of the imaging signal to the start of the frame of the image signal output to the display unit is a phase difference,
The frame rate of the imaging unit is a first frame rate,
When the highest frame rate that can be displayed on the display unit is the second frame rate,
The timing controller is
In the case where the phase difference is longer than the image processing time,
First timing control for gradually reducing the phase difference by causing the image signal generation unit to output the image signal so that the frame rate of the display unit becomes the second frame rate;
After the phase difference is equal to or less than the image processing time,
A second timing control for causing the image signal generation unit to output the image signal so that a frame rate of the display unit becomes the first frame rate;
Is feasible,
An image processing apparatus. The display unit;
An imaging device according to claim 5;
Comprising
An imaging display device characterized by that.

Images