JP2012124678A - Imaging apparatus and imaging apparatus control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus and an imaging apparatus control program for eliminating image quality degradation, when driving an image pickup device in an optimum driving frequency, due to a duplicate processing and the like for repeating the same frame in a certain frame interval because of the frame rate difference in a set dynamic image.SOLUTION: An imaging apparatus includes: an imaging unit for outputting frame image signals by performing a photo-electric conversion on a subject image; a storage unit for writing and reading the frame image signals sequentially; a monitor unit for monitoring the read and write state of the frame image signals in the storage unit; and a control unit for changing an interval of vertical synchronous signals that define the timing of outputting frame image signals from the imaging unit based on the read and write state monitored by the monitor unit.

Description

  The present invention relates to an imaging apparatus and a control program for the imaging apparatus.

There are various standards for the frame rate of a moving image output from an imaging apparatus. Therefore, a plurality of oscillators for driving the image sensor are provided to correspond to the set frame rate, and these are switched to synchronize the image output frequency from the image sensor with the frame rate. Alternatively, the image output frequency from the image sensor is changed by dividing one oscillator.
[Prior art documents]
[Patent Literature]
[Patent Document 1] JP-A-8-46871

  Each imaging element has a frequency suitable for driving. When the image sensor is driven at the optimal drive frequency, due to the difference from the set frame rate of the moving image, overlap processing to continue the same frame or frame skip processing to delete the frame must be performed at a certain frame interval. In other words, the image quality was degraded.

  In order to solve the above problems, an imaging apparatus according to a first aspect of the present invention includes an imaging unit that photoelectrically converts a subject image and outputs a frame image signal, and the frame image signal is sequentially written and read sequentially. Based on the storage unit, the monitoring unit that monitors the reading / writing status of the frame image signal in the storage unit, and the reading / writing status monitored by the monitoring unit, the interval of the vertical synchronization signal that defines the timing for outputting the frame image signal from the imaging unit And a control unit for changing.

  In order to solve the above problem, a control program for an imaging apparatus according to a second aspect of the present invention includes a writing step of sequentially writing frame image signals output from an imaging unit to a storage unit, and sequentially writing frame image signals from the storage unit. A reading step for reading, a monitoring step for monitoring the read / write status of the frame image signal in the storage unit, and a vertical synchronization signal for defining a timing for outputting the frame image signal from the imaging unit based on the read / write status monitored by the monitoring step And causing the computer to execute a control step of changing the interval.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 is a system configuration diagram of a camera according to an embodiment. It is explanatory drawing explaining the flow of the signal between each element which comprises a system. It is a conceptual diagram of the reading / writing situation of an image memory. It is a timing chart regarding a vertical synchronizing signal. It is a processing flowchart regarding control of an image memory.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a system configuration diagram of a camera 100 according to the present embodiment. A camera 100 as an imaging apparatus can record and output a subject image received by the imaging element 103 as still image data or moving image data. In the present embodiment, the generation and output of moving image data will be described in detail.

  Several types of standards have been established all over the world for the number of frames output per unit time as moving image data, that is, the frame rate of moving image output. Representative examples include NTSC (29.97 Hz, 59.94 Hz), PAL (25 Hz, 50 Hz), and other (23.976 Hz) frame rates. The camera 100 can set an arbitrary frame rate from among these by operating the operation member 119 by the user, for example.

  The user can also select an image size per frame. For example, a full HD standard of 1920 pixels × 1080 pixels can be selected. Therefore, in the camera 100, the processing load of moving image data changes according to the selected frame rate, image size, and the like. The camera 100 according to the present embodiment has a configuration that does not affect the temporal continuity of the output image and the image quality of the output image, regardless of the frame rate and image size selected by the user. A specific configuration will be described below.

  The camera 100 includes an optical system 101. The optical system 101 includes a zoom lens, a focus lens, and the like. Subject light enters the optical system 101 along the optical axis, passes through a shutter 102 disposed in front of the image sensor 103, and forms an image as a subject image on the light receiving surface of the image sensor 103.

  The imaging element 103 is an element that photoelectrically converts an optical image that is a subject image, and a CCD sensor, a CMOS sensor, or the like is used. In the present embodiment, a case where a CCD sensor is used as the image sensor 103 will be described. The image sensor 103 is driven by the driver 105 in synchronization with the clock signal supplied from the first oscillator 104 and the vertical synchronization signal and horizontal synchronization signal supplied from the DFE 107. The driver 105 controls the charge accumulation time in each pixel of the image sensor 103 as described later, and selects a pixel to be output to the AFE 106.

  The AFE 106 receives an image signal output from the image sensor 103 and performs analog processing such as gain control and noise removal on the image signal. The image signal that has been subjected to analog processing by the AFE 106 is output to the DFE 107. The DFE 107 converts the image signal analog-processed by the AFE 106 into a digital signal. Further, the DFE 107 adjusts the time interval between the vertical synchronization signal and the horizontal synchronization signal under the control of the system control unit 110 and sends the adjusted time interval to the driver 105. The image signals digitized by the DFE 107 are temporarily output to the internal memory 108 and stored.

  The image signals stored in the internal memory 108 are sequentially read out to the image processing unit 109. The image processing unit 109 performs image processing such as white balance processing, gamma processing, and interpolation processing on the read image signal to generate a frame image, and outputs the frame image to a recording medium, a display device, or the like as moving image data at a selected frame rate. .

  The internal memory 108 is a random access memory that can be read and written at high speed, and for example, a DRAM, an SRAM, or the like is used. The internal memory 108 also serves as a work memory in image processing performed by the image processing unit 109.

  The image processing unit 109 sequentially reads out image signals stored in the internal memory 108 in synchronization with the clock signal supplied from the clock switcher 111. The clock switcher 111 receives different clock signals from the second oscillator 112 and the third oscillator 113, selects one of the clock signals according to an instruction from the system control unit 110, and supplies the selected clock signal to the image processing unit 109. Here, the clock signal supplied from the clock switcher 111 to the image processing unit 109 has a frequency that matches the selected frame rate.

  The camera 100 is directly or indirectly controlled by the system control unit 110 including each element in the image processing described above. The system control unit 110 includes a system memory 114 that is an electrically erasable / recordable nonvolatile memory. The system memory 114 is configured by EEPROM (registered trademark) or the like. The system memory 114 records constants, variables, programs, and the like necessary when the camera 100 is operating so that they are not lost even when the camera 100 is not operating. The system control unit 110 develops these constants, variables, programs, and the like in the internal memory 108 as appropriate and uses them for controlling the camera 100.

  The moving image data output from the image processing unit 109 can be displayed on the display unit 116 built in the camera 100 under the control of the display control unit 115. In addition, the display control unit 115 can display various menu items related to various settings of the camera 100 on the display unit 116 together with the display of the moving image data or without displaying the moving image data.

  The external connection IF 117 is an interface for connecting to an external device. The external connection IF 117 is connected to a connection unit 118 such as a card slot into which a memory card is inserted, an HDMI terminal connected to a TV monitor or the like, and a LAN unit communicating with a PC. The moving image data output from the image processing unit 109 is recorded on, for example, a memory card via the external connection IF 117 and displayed on the TV monitor.

  The camera 100 includes an operation member 119 that receives an operation from a user. The system control unit 110 detects that the operation member 119 has been operated. The system control unit 110 executes an operation according to the detected operation. For example, when it is detected that the REC button as the operation member 119 has been operated, the system control unit 110 executes a series of shooting operations for photoelectrically converting the subject image to generate moving image data.

  The optical system 101 is controlled by the lens control unit 120. For example, the lens control unit 120 drives the zoom lens in accordance with a user instruction to change the angle of view of the subject image. Further, the lens control unit 120 drives the focus lens based on the AF information, and focuses the subject image on the light receiving surface of the image sensor 103. The lens control unit 120 may perform autofocus control on the subject designated by the user through the operation member 119 or the like.

  Here, the relationship between the drive frequency for driving the image sensor 103 and the selected frame rate is organized.

  As described above, there are a plurality of standardized frame rates. In order to cope with any of the plurality of frame rates, the camera is provided with a corresponding number of oscillators. However, one oscillator is prepared as a group for frame rates that can be multiplied or divided in frequency. The moving image data output from the image processing unit is output in synchronization with a clock signal supplied from an oscillator selected corresponding to the frame rate.

  On the other hand, the image signal output from the image sensor is synchronized with the clock signal supplied from the oscillator via the driver. Since the image sensor has an optimum frequency for outputting an image signal, an oscillator that supplies a clock signal to the image sensor is provided with a unique one according to the optimum frequency.

  Then, a mismatch between the rate of the image signal supplied from the image sensor and the rate of moving image data output from the image processing unit occurs. This inconsistency causes the image signal to be processed to be exhausted or the memory capacity to be insufficient in the internal memory where the image signal is temporarily stored. In conventional cameras, this phenomenon has been solved by overlapping output of the same frame image or skipping frame image generation. That is, time continuity is sacrificed in the output image. For example, when a frame rate of 59.94 Hz is selected in a camera having a memory capacity for one frame, frame images may be lost at a rate of one frame per 35 seconds. If continuity is not secured as moving image data, appreciation is significantly impaired.

  Alternatively, the clock signal from the oscillator corresponding to the selected frame rate is also used to drive the image sensor, and the rate of the image signal supplied from the image sensor and the rate of the moving image data output from the image processing unit are set. It can also be matched. However, as described above, the imaging device has an optimum frequency for outputting an image signal. Therefore, if the imaging device is forcibly driven at a different frequency, the imaging performance cannot be sufficiently obtained. That is, it has been a factor causing image quality degradation.

  Therefore, in the camera 100 of the present embodiment, the moving image data output from the image processing unit 109 is synchronized with a clock signal supplied from the second oscillator or the third oscillator selected according to the frame rate. In addition, the image sensor 103 is driven at an optimum frequency using a clock signal from the first oscillator 104. Further, the system control unit 110 monitors the read / write status of the internal memory 108 and adjusts the interval of the vertical synchronization signals supplied from the DFE 107. By performing such processing, it is possible to avoid the depletion of the image signal to be processed or the shortage of the memory capacity, and to realize the output of moving image data with high appreciability that ensures continuity. A more specific signal flow will be described below.

  FIG. 2 is an explanatory diagram for explaining the flow of signals between the elements constituting the system of the camera 100. Elements that are the same as those in FIG. 1 are denoted by the same reference numerals, but here, the elements are connected by arrows, and the flow of timing signals such as clock signals, image signals, and control signals is mainly shown.

  The driver 105 drives the image sensor 103 by the clock signal CLK1 supplied from the first oscillator 104. The image sensor 103 synchronizes one output from one pixel with the clock signal CLK1. The frequency of the clock signal CLK1 is a frequency at which the image sensor 103 can be optimally driven.

  Further, a vertical synchronization signal VD and a horizontal synchronization signal HD output from the synchronization signal generator 202 included in the DFE 107 are supplied to the driver 105. The driver 105 supplies the vertical synchronization signal VD to the image sensor 103 to start transmission of an image signal for one frame to the AFE 106. In addition, the driver 105 supplies the horizontal synchronization signal HD to the image sensor 103 to thereby define an output for one line to the AFE 106.

  The image signal analog-processed by the AFE 106 is sent as an image output S to the image processing circuit 201 included in the DFE 107 in synchronization with the clock signal CLK1_S. The clock signal CLK1_S is generated by multiplying or dividing the clock signal CLK. The image processing circuit 201 converts an analog signal into a digital signal, and executes preprocessing of image processing performed by the image processing unit 109. The processed image signal is sent to the image processing unit 109 as an image output D in synchronization with the clock signal CLK1_D. Note that the clock signal CLK1_D is generated by multiplying or dividing the clock signal CLK1_S. The image processing unit 109 sequentially writes and stores the received image signal in the image memory 203 which is an area of the internal memory 108.

  The system control unit 110 sends a CLK selection signal to the clock switcher 111 according to a frame rate set by a user instruction or the like. The clock switcher 111 selects one of the clock signal CLK2 input from the second oscillator 112 and the clock signal CLK3 input from the third oscillator 113 in accordance with the instruction of the CLK selection signal, and supplies the selected image to the image processing unit 109.

  Here, an example in which the second oscillator 112 and the third oscillator 113 are provided as the oscillators corresponding to a plurality of frame rates will be described. Of course, an oscillator is further provided according to the frame rate to be supported. There is also. Further, the clock switcher 111 may be configured to select a clock signal that has been multiplied and divided based on the original oscillation frequency of these oscillators. Alternatively, the frequency of the clock signal sent from the clock switcher 111 to the image processing unit 109 is the original frequency of the oscillator, and a clock signal corresponding to the set frame rate is obtained by multiplication and division on the image processing unit 109 side. It may be generated.

  The image processing unit 109 reads an image signal from the image memory 203 in synchronization with the clock signal CLK2 or CLK3 supplied from the clock switcher 111, performs image processing, and generates moving image data having a set frame rate and image size. For example, output to the HDMI terminal.

  The image processing unit 109 controls reading and writing of frame image signals to and from the image memory 203, and continuously grasps how many image signals have been written to the image memory 203 or how many image signals have been read out. is doing. That is, the image processing unit 109 monitors the read / write status of the frame image signal by grasping the write address and the read address of the image memory 203.

  The image processing unit 109 sends the monitoring status of the image memory 203 to the system control unit 110 as an image data status signal. The image data state signal indicates either a state in which the reading speed is high and the frame image signal is likely to be exhausted, a state in which the writing speed is high and the frame image signal is likely to overflow, or a state that may be regarded as equilibrium. Status signal. The state of the image memory 203 will be described in detail later.

  The system control unit 110 receives the image data state signal and adjusts the setting of the register that defines the time interval of the vertical synchronization signal VD. For example, in the image memory 203, if the readout speed is high and the frame image signal is almost exhausted, the system control unit 110 shortens the interval of the vertical synchronization signal VD in order to increase the supply of the frame image signal. Change the register. Conversely, if the writing speed is high and the frame image signal is likely to overflow, the system control unit 110 changes the register so as to increase the interval of the vertical synchronization signal VD in order to suppress the supply of the frame image signal. To do.

  The synchronization signal generator 202 generates the vertical synchronization signal VD according to the interval set in the register. When the register setting is changed by the system controller 110, the changed interval is applied at the next generation timing. To do.

  Next, the reading / writing state of the frame image signal in the image memory 203 will be described. FIG. 3 is a conceptual diagram of the read / write status of the image memory 203. The figure conceptually represents the storage area of the image memory 203 by a plane. In the figure, the frame image signal is sequentially written in the horizontal direction from the left end, and when reaching the right end, it is written down from the left end to the right end by one line down. When it reaches the lower right corner, it moves to the upper left corner again and writing continues. Reading is performed in parallel with writing. Therefore, the area where writing has been completed but not yet read is the processing waiting area. The area other than the processing waiting area is an empty area. The empty area may be in a state where the read image signal remains as it is and is overwritten.

  In the figure, a solid triangle arrow represents an input to the image memory 203 for writing an image signal. The length represents a writing speed that is a writing amount of an image signal per unit time. Similarly, a white triangle arrow represents an output from the image memory 203 that is reading an image signal. The length represents the reading speed, which is the reading amount of the image signal per unit time.

  In this embodiment, the capacity of the image memory 203 is a capacity corresponding to one frame image signal. The size of the one-frame image here corresponds to the maximum image size that can be selected. That is, the processing in this embodiment can be performed if there is a capacity corresponding to one frame image signal. Of course, more memory capacity may be allocated with a margin.

  FIG. 3A shows a state where the processing waiting area and the free area are balanced, although there is a difference between the input speed and the output speed, and the inconvenience described below does not occur immediately. Therefore, the image processing unit 109 determines such a state as an equilibrium state.

  FIG. 3B shows a state where the input speed is higher than the output speed, and the image signal to be read overflows. As shown in the figure, the processing waiting area becomes overwhelmingly larger than the free area, and the free area disappears soon. In other words, it is an over-input state where the input catches up with the output. The image processing unit 109 determines that there is an excessive input state when the free area is equal to or less than a predetermined ratio with respect to the entire area.

  When the image processing unit 109 sends the status of the excessive input state to the system control unit 110 as an image data state signal, the system control unit 110 changes the register setting to increase the time interval of the vertical synchronization signal VD. Then, since the input of the next frame image signal is delayed by the lengthened time, if the output is continued during that time, the free area can be increased, and the equilibrium state of FIG. it can.

  FIG. 3C shows a state where the output speed is higher than the input speed and the image signal to be read out is exhausted. As shown in the figure, the free area becomes overwhelmingly larger than the process waiting area, and the process waiting area disappears soon. In other words, this is an over-output state where the output catches up with the input. The image processing unit 109 determines that the output excess state occurs when the processing waiting area is equal to or less than a predetermined ratio with respect to the entire area.

  When the image processing unit 109 sends the status of the output excess state as an image data state signal to the system control unit 110, the system control unit 110 changes the register setting to shorten the time interval of the vertical synchronization signal VD. Then, since the input of the next frame image signal is started early by the shortened time, the processing waiting area can be increased, and the equilibrium state of FIG.

FIG. 4 is a timing chart relating to the vertical synchronization signal VD. In the figure, the first vertical synchronization period is set to an interval longer by α than the standard time t ₀ corresponding to the frame rate by setting the register. When 29.97 Hz is set as the frame rate, t ₀ = 1 / 29.97 sec.

  When a pixel reset signal is output after a while from the output of the vertical synchronization signal VD, each pixel of the image sensor 103 starts to accumulate charges. The charge accumulation is continued until the next vertical synchronizing signal VD is output. That is, this period is the exposure period. When the next vertical synchronizing signal VD is output, each pixel finishes storing charges and sequentially outputs image signals to the AFE 106.

  The period from the completion of the image signal output to the output of the next vertical synchronizing signal VD is V blanking. By providing V blanking, the interval of the vertical synchronization signal VD can be shortened or lengthened. That is, since this period is an adjustment allowance, the set frame rate can be maintained in the subsequent processing.

In the figure, when the register setting is changed during the first vertical synchronization period, the new register is applied to the next vertical synchronization signal VD, and the length of the vertical synchronization period until the next vertical synchronization signal VD. Is changed. Here, the changed vertical synchronization period is shorter by β than the standard time t ₀ .

  With the change of the vertical synchronization period, the output timing of the pixel reset signal is also changed. For example, in the figure, when the vertical synchronization period is shortened by (α + β), the output timing of the pixel reset signal is also advanced by (α + β) and output. By changing in this way, a constant exposure period can be maintained even if the vertical synchronization period is changed. Therefore, uneven brightness does not occur between frame images in the output image signal.

  Next, a series of moving image shooting processes will be described, particularly focusing on processes related to image memory control. FIG. 5 is a process flow diagram relating to the control of the image memory 203. For example, the flow starts when the camera 100 is turned on and switched to the moving image shooting mode.

  In step S101, the system control unit 110 receives a moving image setting in accordance with, for example, an operation of the operation member 119 by the user. The moving image setting here includes a frame rate of moving image output and an image size per frame. In step S102, the system control unit 110 sends a CLK selection signal to the clock switcher 111 according to the accepted frame rate. The clock switcher 111 switches a clock signal supplied to the image processing unit 109 in accordance with a command of the CLK selection signal.

  In step S103, the system control unit 110 waits for a pressing operation of the REC button, which is an instruction to start moving image shooting. Steps S101 to S103 are repeated until an instruction to start moving image shooting is given.

  When an instruction to start moving image shooting is received in step S103, a series of shooting operations for photoelectrically converting the subject image to generate moving image data is started. The photoelectrically converted image signals are sequentially written into the image memory 203 as described above. Then, it is read out to the image processing unit 109. In step S105, the image processing unit 109 monitors the state of the image memory 203. Then, the image processing unit 109 sends a status indicating the state to the system control unit 110 as an image data state signal.

  In step S106, the system control unit 110 determines whether the state of the image memory 203 is in an equilibrium state based on the image data state signal received from the image processing unit 109. If it is determined that the input excess state or the output excess state is determined instead of the equilibrium state, the system control unit 110 proceeds to step S107 and changes the setting of the register that defines the time interval of the vertical synchronization signal VD. When the register is changed, the synchronization signal generator 202 changes the interval of the vertical synchronization signal VD at the above-described timing.

  When it is determined in step S106 that an equilibrium state is present, or after register change is performed in step S107, the process proceeds to step S108, and the system control unit 110 determines whether there is an instruction to end moving image shooting. If there is no instruction, the process returns to step S105 again to continue monitoring the image memory 203. During this loop, the camera 100 continues the moving image shooting process. If there is an instruction, the process proceeds to step S109, and the system control unit 110 executes a shooting end process, and ends a series of shooting sequences.

  In the present embodiment described above, the camera 100 having the moving image shooting function has been described as an example, but of course, a camera having a still image shooting function may be used. Further, the optical system 101 may not be integrally formed, and may be a lens interchangeable single-lens camera having a moving image shooting function. Further, the present embodiment can be applied even to a portable device having a moving image shooting function.

  In the present embodiment described above, a global shutter operation using a CCD sensor as the image sensor 103 has been described. However, the present invention is not limited to this, and the above processing can also be applied to a rolling shutter operation using a CMOS sensor. In the case of the rolling shutter operation, pixel reset signals are continuously generated for each line, but the generation timings of these continuous pixel reset signals are also changed in accordance with the timing change of the vertical synchronization signal VD. By such adjustment, even if the timing of the vertical synchronization signal VD is changed, the exposure time in each line is constant, and the entire frame does not cause a luminance difference with respect to the previous and subsequent frames.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 100 Camera, 101 Optical system, 102 Shutter, 103 Image sensor, 104 1st oscillator, 105 driver, 106 AFE, 107 DFE, 108 Internal memory, 109 Image processing part, 110 System control part, 111 Clock switcher, 112 2nd oscillator , 113 Third oscillator, 114 System memory, 115 Display control unit, 116 Display unit, 117 External connection IF, 118 Connection unit, 119 Operation member, 120 Lens control unit, 201 Image processing circuit, 202 Synchronization signal generation unit, 203 image memory

Claims (6)

An imaging unit that photoelectrically converts a subject image and outputs a frame image signal;
A storage unit in which the frame image signals are sequentially written and sequentially read;
A monitoring unit that monitors the read / write status of the frame image signal in the storage unit;
An imaging apparatus comprising: a control unit that changes an interval of a vertical synchronization signal that defines a timing for outputting the frame image signal from the imaging unit based on the read / write state monitored by the monitoring unit. A first oscillator that generates a first synchronization signal for driving the imaging unit;
The imaging apparatus according to claim 1, further comprising: a second oscillator that generates a second synchronization signal for sequentially reading the frame image signal from the storage unit. A setting unit for setting a frame rate of moving image data generated from the frame image signal;
The imaging apparatus according to claim 2, further comprising: a selection unit that selects one of the plurality of second oscillators based on the frame rate set by the setting unit.   The imaging device according to any one of claims 1 to 3, wherein a storage capacity of the storage unit is a capacity corresponding to one frame of the frame image signal.   5. The control unit according to claim 1, wherein when the interval of the vertical synchronization signals is changed, the generation timing of a reset signal for resetting the pixels of the imaging unit is also changed within the same vertical synchronization period. The imaging device described in 1. A writing step of sequentially writing frame image signals output from the imaging unit to the storage unit;
A reading step of sequentially reading out the frame image signals from the storage unit;
A monitoring step of monitoring the read / write status of the frame image signal in the storage unit;
An imaging apparatus control program for causing a computer to execute a control step of changing an interval of a vertical synchronization signal that defines a timing for outputting the frame image signal from the imaging unit based on the read / write state monitored in the monitoring step .

Images