JP4561452B2 - IMAGING DEVICE, VIDEO OUTPUT METHOD, AND VIDEO REPRODUCTION PROGRAM - Google Patents

Description

  The present invention relates to an imaging device capable of variable speed imaging, a video output method, and a video reproduction program.

  In special shooting or the like, variable speed shooting may be performed in which shooting is performed on the assumption that the shot subject appears to move faster than the shooting speed. For example, when shooting with film, high-speed projection is realized by shooting with the film rotating at a slower speed than normal shooting and rotating the film at normal speed during projection. can do.

  Also, for example, when recording captured video as digital video data, the subject is shot with the frame frequency of the video frame at the time of shooting set lower than the frame frequency of the video frame at the time of playback. There has been proposed an imaging apparatus capable of variable-speed imaging that realizes high-speed reproduction by reproducing as a video signal.

  A variable speed imaging method using a conventional imaging apparatus will be schematically described. An object is imaged at a predetermined frame frequency, and an imaging frame obtained by imaging is written in a memory. The memory has a plurality of banks as areas for writing imaging frames, and can write imaging frames in each bank. The imaging frame written in the memory is read out as a video frame having a predetermined frame frequency based on a synchronization reference signal supplied from the outside.

  At this time, since the frame frequency of the imaging frame is different from the frame frequency of the video frame, there may be a timing for reading the imaging frame while the imaging frame is being written. The written imaging frame cannot be read out as a video frame until the writing is completed. In such a case, the immediately preceding imaging frame written in another bank is read out again. That is, depending on the timing, the same imaging frame is read out redundantly.

  As described above, Patent Document 1 discloses an imaging apparatus that can perform variable speed imaging by reading imaging frames as video frames having different frame frequencies.

JP 2000-125210 A

  Consider a case where a captured image is viewed with a viewfinder. The viewfinder displays captured images based on, for example, 30 Hz video frames. As described above, there is an overlapping part depending on the timing of reading out the video frame, so when displayed on the viewfinder as it is, the video frame does not correspond to the time axis, so it looks like an unnatural movement. There was a problem of end.

  Accordingly, an object of the present invention is to provide an image pickup apparatus and a video output method capable of smoothly displaying an image without a sense of incongruity when an image taken by an image pickup apparatus capable of variable speed imaging is displayed on a display unit such as a viewfinder. And providing a video playback program.

In order to solve the above-described problem, the invention described in claim 1 is configured to capture an image of an imaging frame that captures light from a subject and an imaging frame having a frame frequency lower than the frame frequency based on the synchronization reference signal. An imaging drive unit that drives the imaging unit, a storage unit that sequentially stores imaging frames output from the imaging unit, and an imaging frame stored in the storage unit and an imaging frame immediately before the imaging frame are mixed at a predetermined ratio generates a reproduced video frame is, possess a signal processing unit for outputting on the basis of the sync reference signal, the storage unit, a bank for storing the captured frame comprises at least three, imaging frame, the timing of the image pickup frame The frame update information is written in the selected bank among the three banks that are sequentially switched in synchronization, and indicates the bank switching in synchronization with the synchronization reference signal. Hazuki switches the three banks in a predetermined, an imaging apparatus adapted to select each of the previous imaging frame of the imaging frame and imaging frames used to generate the reproduced video frame.

According to a third aspect of the present invention, the timing of the imaging frame of the storage unit that captures an imaging frame having a frame frequency lower than the frame frequency based on the synchronization reference signal and has at least three banks for storing the imaging frames is provided. given the selected bank of the sequentially switched three banks in synchronization, storing sequential the captured imaged frame, the three banks on the basis of a frame update information indicating the switching of the banks in synchronism with the sync reference signal And the imaging frame used when generating the playback video frame and the imaging frame immediately before the imaging frame are respectively selected, and the selected imaging frame and the imaging frame immediately before the imaging frame are selected at a predetermined ratio. in to generate a mixed reproduced video frame was, are the movies image output method to output on the basis of the sync reference signal

According to a fourth aspect of the present invention, the timing of the imaging frame of the storage unit that captures an imaging frame having a frame frequency lower than the frame frequency based on the synchronization reference signal and has at least three banks for storing the imaging frames is provided. given the selected bank of the sequentially switched three banks in synchronization, storing sequential the captured imaged frame, the three banks on the basis of a frame update information indicating the switching of the banks in synchronism with the sync reference signal And the imaging frame used when generating the playback video frame and the imaging frame immediately before the imaging frame are respectively selected, and the selected imaging frame and the imaging frame immediately before the imaging frame are selected at a predetermined ratio. A video output method that generates a playback video frame mixed in step 1 and outputs it based on the synchronization reference signal. Yuta a Film image output program that is executed in the apparatus.

As described above, according to the first, third, and fourth aspects of the present invention, the storage unit that captures an imaging frame having a frame frequency lower than the frame frequency based on the synchronization reference signal and has at least three banks for storing the imaging frames. of the selected bank of the three banks are sequentially switched in synchronism with the timing of the image pickup frame, stores sequential the captured image pick-up frame, the frame update indicating the switching of the banks in synchronism with the sync reference signal Based on the information, the three banks are switched to a predetermined one, the imaging frame used when generating the playback video frame and the imaging frame immediately before the imaging frame are respectively selected, and the selected imaging frame and the imaging frame immediately before the imaging frame are selected . Generate a playback video frame that mixes the imaging frame with a predetermined ratio and output it based on the synchronization reference signal Therefore, the image can be smoothly displayed on a display device such as a viewfinder.

  According to the present invention, when variable speed imaging is performed, a video frame at a certain time point and a previous video frame are mixed at a predetermined ratio to generate a playback video frame. There is an effect that can be made.

  An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows an example of the configuration of an imaging apparatus 11 applicable to an embodiment of the present invention. The imaging device 11 includes, for example, an imaging unit 21, an A / D conversion unit 22, a digital signal processing unit 23, a memory 24, a timing signal generation unit 25, an imaging timing signal generation unit 26, a system controller 27, and a display device 28.

  The timing signal generation unit 25 generates a timing signal based on the synchronization reference signal supplied from the outside, supplies the timing signal to the digital signal processing unit 23, and supplies the timing signal generation unit 26 with the timing signal. This synchronization reference signal is a signal that is commonly used in broadcasting equipment in a broadcasting station or studio when the imaging apparatus 11 is used in the broadcasting station or studio. For example, in a broadcast station or a studio, video frames can be synchronized using this synchronization reference signal.

  The system controller 27 includes, for example, a CPU (Central Processing Unit), and uses a RAM (Random Access Memory) (not shown) as a work memory when executing the program according to a program stored in a ROM (Read Only Memory) (not shown). The respective units of the imaging device 11 are controlled. The system controller 27 generates a signal for determining the frame frequency of the imaging frame of the imaging signal based on a control command supplied from the outside, and supplies the signal to the imaging timing signal generator 26.

  The imaging timing signal generation unit 26 determines the imaging timing by the imaging unit 21 based on the timing signal supplied from the timing signal generation unit 25 and the signal for determining the frame frequency of the imaging frame supplied from the system controller 27. An imaging timing signal to be determined is generated and supplied to the imaging unit 21 and the digital signal processing unit 23.

  The imaging unit 21 uses, for example, a CCD (Charge Coupled Devices) as an imaging element, and converts light received by the imaging element into an imaging signal by photoelectric conversion. For example, the imaging element has 2200 pixels × 1125 lines as effective pixels. The imaging unit 21 outputs an imaging signal for one frame based on the imaging timing signal supplied from the imaging timing signal generation unit 26. The imaging signal output from the imaging unit 21 is supplied to the A / D conversion unit 22, converted into a digital imaging signal, and supplied to the digital signal processing unit 23.

  The digital signal processing unit 23 performs predetermined signal processing such as γ (gamma) correction and white balance adjustment on the supplied digital image pickup signal, and writes it in the memory 24. The memory 24 has at least three banks, a bank a, a bank b, and a bank c, in which video signals for at least one frame can be written, respectively, as data writing areas. Data of the imaging frame based on the digital imaging signal supplied from the digital signal processing unit 23 is written in a bank selected in advance among the three banks. The data of the imaging frame written in the bank of the memory 24 is read from the bank of the memory 24 in synchronization with the timing signal supplied from the timing signal generator 25, and the read data is output to the outside as a video signal. And supplied to the display device 28.

  The display device 28 is used as, for example, a viewfinder, and displays the video signal output from the digital signal processing unit 23 at a frame frequency synchronized with the synchronization reference signal.

  Next, a method for performing variable speed imaging with the imaging device 11 will be described. After the image pickup unit 21 picks up an image of the CCD and performs predetermined signal processing, the data of the image pickup frame converted into the digital image pickup signal by the A / D conversion unit 22 is written in the bank of the memory 24. The data of the imaging frame written in the memory 24 is read from the first position of the video frame synchronized with the synchronization reference signal after all of the imaging frame data is written and the writing of the imaging frame data is completed. .

  It is assumed that the video frame synchronized with the synchronization reference signal has a frame frequency of 30 Hz (hertz), 1125 lines per frame, and 2200 pixels per line.

  Here, for example, let us consider a case where the playback speed is 1.5 times the imaging speed. This can be realized, for example, by reproducing an image captured at a frame frequency of 20 Hz at a frame frequency of 30 Hz. Therefore, when the ratio of the frame frequency of the imaging frame and the frame frequency of the video frame synchronized with the synchronization reference signal is used as the frame ratio, this value is 2/3. When the clock frequency for driving the CCD and the number of lines per frame are fixed, the number of pixels per line of the imaging frame is the number of pixels per line when the frame frequency of the imaging frame is 30 Hz. Multiply 2200 pixels by 3/2, which is the reciprocal of the frame ratio, to obtain 3300 pixels.

  That is, when the reproduction speed is X times the imaging speed, the frame ratio may be 1 / X. Therefore, the number of pixels per line is determined based on the synchronization reference signal. Multiply 2200 pixels, which is the number of pixels per line at the frame frequency, by X, which is the reciprocal of the frame ratio, to obtain “2200 × X” pixels.

  On the other hand, the clock frequency for driving the CCD is fixed, and the number of effective pixels per line of the CCD is fixed regardless of the frame frequency. Therefore, as shown in FIG. 2, the difference between the number of pixels per line (3300 pixels in this example) corresponding to the frame frequency of the imaging frame and the number of pixels per line of the CCD is 1100 pixels. As for time, for example, the blanking portion is set so that nothing is captured. In the bank of the memory 24, data of an imaging frame of 2200 pixels × 1125 lines is written, excluding the blanking portion.

  A method for performing variable speed imaging by the imaging apparatus 11 will be described more specifically with reference to FIG. 3A shows the timing of writing and reading data in bank a, FIG. 3B shows the timing of writing and reading data in bank b, and FIG. 3C shows the timing of writing and reading data in bank c. . FIG. 3D shows the timing corresponding to the frame head position based on the synchronization reference signal.

  For example, the data of the imaging frame is sequentially written in three banks of the memory 24, and the data of the imaging frame that has been written is read out at the head position of the video frame. In addition, even if the reading of the video frame data is completed, if the writing of the next imaging frame is not completed, the data of the video frame is read again at the head of the next video frame.

  In FIG. 3, the data of the imaging frame # 1 is written into the bank a of the memory 24 (FIG. 3A). The data of the imaging frame # 1 written in the bank a is read based on the synchronization reference signal as the video frame # 11 from the head position of the frame based on the synchronization reference signal that comes after the completion of the writing of the data of the imaging frame # 1. .

  When the writing of the data of the imaging frame # 1 is completed, the data of the imaging frame # 2 is written into the bank b of the memory 24 (FIG. 3B). When the reading of the video frame # 11 is completed, the writing of the data of the imaging frame # 2 is not completed, so the data of the imaging frame # 1 written in the bank a is the head of the frame based on the synchronization reference signal. Is read out again as video frame # 12. The data of the imaging frame # 2 written to the bank b is read based on the synchronization reference signal as the video frame # 13 from the head position of the frame based on the synchronization reference signal that comes after the completion of the writing of the data of the imaging frame # 2. .

  When the writing of the data of the imaging frame # 2 is completed, the data of the imaging frame # 3 is written into the bank c of the memory 24 (FIG. 3C). The data of the imaging frame # 3 written to the bank c is read based on the synchronization reference signal as the video frame # 14 from the head position of the frame based on the synchronization reference signal that comes after the completion of the writing of the data of the imaging frame # 3. .

  When the writing of the data of the imaging frame # 3 is completed, the data of the imaging frame # 4 is written into the bank a of the memory 24. When the reading of the video frame # 14 is completed, the writing of the data of the imaging frame # 4 is not completed. Therefore, the data of the imaging frame # 3 written in the bank c is the head of the frame based on the synchronization reference signal. Is read out again as video frame # 15.

  After the reading of the video frame # 15 is completed, the data of the imaging frame # 4 written in the bank a is the video frame from the head position of the frame based on the synchronization reference signal that comes after the writing of the data of the imaging frame # 4 is completed. As # 16, it is read based on the synchronization reference signal.

  Thereafter, in the same manner as described above, the imaging frame is written into each bank of the memory 24 and the video frame is read out.

  In this way, when an imaging frame with a frame frequency of 20 Hz is read out as a video frame with a frame frequency of 30 Hz and this video signal is reproduced by removing the duplicated video frame, the speed of the subject is higher than the speed at the time of shooting. Can also be seen 1.5 times faster.

  Next, the variable speed imaging method when the frame ratio is 4/5 will be described with reference to FIG. In this case, an image captured at a frame frequency of 24 Hz can be reproduced at 1.25 times the speed at the time of imaging by reproducing at a frame frequency of 30 Hz. 4A shows the timing for writing and reading data in bank a, FIG. 4B shows the timing for writing and reading data in bank b, and FIG. 4C shows the timing for writing and reading data in bank c. . FIG. 4D shows the timing corresponding to the frame head position based on the synchronization reference signal.

  In FIG. 4, the data of the imaging frame # 1 is written into the bank a of the memory 24 (FIG. 4A). The data of the imaging frame # 1 written in the bank a is read based on the synchronization reference signal as the video frame # 11 from the head position of the frame based on the synchronization reference signal that comes after the completion of the writing of the data of the imaging frame # 1. .

  When the writing of the data of the imaging frame # 1 is completed, the data of the imaging frame # 2 is written into the bank b of the memory 24 (FIG. 4B). When the reading of the video frame # 11 is completed, the writing of the data of the imaging frame # 2 is not completed, so the data of the imaging frame # 1 written in the bank a is the head of the frame based on the synchronization reference signal. Is read out again as video frame # 12. The data of the imaging frame # 2 written to the bank b is read based on the synchronization reference signal as the video frame # 13 from the head position of the frame based on the synchronization reference signal that comes after the completion of the writing of the data of the imaging frame # 2. .

  When the writing of the data of the imaging frame # 2 is completed, the data of the imaging frame # 3 is written into the bank c of the memory 24 (FIG. 4C). The data of the imaging frame # 3 written to the bank c is read based on the synchronization reference signal as the video frame # 14 from the head position of the frame based on the synchronization reference signal that comes after the completion of the writing of the data of the imaging frame # 3. .

  When the writing of the data of the imaging frame # 3 is completed, the data of the imaging frame # 4 is written into the bank a of the memory 24. The data of the imaging frame # 4 written to the bank a is read based on the synchronization reference signal as the video frame # 15 from the head position of the frame based on the synchronization reference signal that comes after the completion of the data writing of the imaging frame # 4. .

  When the writing of the data of the imaging frame # 4 is completed, the data of the imaging frame # 5 is written into the bank b of the memory 24. The data of the imaging frame # 5 written to the bank b is read based on the synchronization reference signal as the video frame # 16 from the head position of the frame based on the synchronization reference signal that comes after the completion of the writing of the data of the imaging frame # 5. .

  Thereafter, in the same manner as described above, the imaging frame is written into each bank of the memory 24 and the video frame is read out.

  In this way, when an image frame with a frame frequency of 24 Hz is read out as a video frame with a frame frequency of 30 Hz and this video signal is reproduced by removing duplicate video frames, the speed of the subject is higher than the speed at the time of shooting. Can also appear 1.25 times faster.

  For example, the output video signal is recorded on a recording medium, and then the video frame overlap is removed and output. By doing so, a video frame having a frame frequency different from the frame frequency of the imaging frame can be reproduced.

  Here, a case where this video signal is displayed on the display device 28 will be considered. When the video signal is displayed on the display device 28, the video supplied from the digital signal processing unit 23 is displayed as it is, so that duplicate video frames are not removed. For this reason, the video image appears to stop at portions where the video frames overlap, and cannot be smoothly projected. For example, in the example of FIG. 3, the video frame overlaps with video frames # 11 and # 12, video frames # 14 and # 15, and video frames # 17 and # 18. In the example of FIG. 4, these are portions corresponding to video frames # 11 and # 12 and video frames # 16 and # 17.

  Therefore, in the embodiment of the present invention, when the imaging device 11 performs variable speed imaging and displays the video on a display device such as a viewfinder, the video frame at a certain time point and the previous video frame are displayed. A playback video frame mixed at a predetermined ratio is generated, and the playback video frame obtained in this way is displayed on the display device 28 so that the video displayed on the display device 28 can be seen smoothly. .

  FIG. 5 shows an example of the configuration of the digital signal processing unit 23 according to the embodiment of the present invention. The memory controller 30 controls writing and reading of the imaging frame supplied from the A / D conversion unit 22 to and from the memory 24. The imaging frame supplied from the A / D conversion unit 22 is written by switching the three banks of the bank a, the bank b, and the bank c in a predetermined manner in synchronization with the imaging frame under the control of the memory controller 30. The memory controller 30 switches the three banks of the memory 24 to predetermined ones in synchronization with the video frames, and synchronizes the imaging frame at a certain timing and the imaging frame immediately before the imaging frame with the synchronization reference signal. Are read out as a video frame (n) and a video frame (n-1), respectively.

  The video frame (n) read from the bank of the memory 24 is supplied to the multiplier 31 and output to the outside as an output video signal. The multiplier 31 multiplies each pixel of the supplied video frame (n) by a coefficient k and supplies the result to the adder 33. The coefficient k is a value between 0 and 1.

  On the other hand, the video frame (n−1) read from another bank of the memory 24 is supplied to the multiplier 32. The multiplier 32 multiplies the coefficient (1-k) for each pixel of the supplied video frame (n−1) and supplies the result to the adder 33. The adder 33 adds the two supplied video frames for each pixel to obtain a reproduced video frame in which the two video frames are mixed. The obtained reproduced video frame is supplied to the display device 28. The display device 28 displays the supplied playback video frame.

  Next, the coefficient k described above will be described. The coefficient k is determined according to the frame ratio and the frame sequence number. FIG. 6 shows an example of the correspondence between the frame ratio, the frame sequence number, and the coefficient k.

  Note that the frame sequence number is a video frame in which the start position of the imaging frame and the start position of the video frame match from the video frame in which the start position of the imaging frame and the start position of the video frame match. The number assigned to the video frame within the time of one cycle is shown with the video frame from which the first imaging frame within the cycle is read first as the head.

  The coefficient k determines the ratio at the time of mixing the video frame (n) and the video frame (n−1) used when generating the playback video frame. This ratio is determined based on the time ratio of the respective imaging frames read out as the video frame (n) and the video frame (n−1). In this example, the coefficient k is “1”, “0.5”, and “0.5” when the frame ratio is “2/3” and the frame sequence numbers are “0”, “1”, and “2”, respectively. 1 ". When the frame ratio is 4/5 and the frame sequence numbers are “0”, “1”, “2”, “3”, and “4”, “1” and “0.75”, respectively. , “0.5”, “0.25”, and “1”.

  The frame update indicates whether or not the bank from which the video frame (n) and the video frame (n−1) are read is switched at the cycle of the next video frame. The frame update timing of the video frame is determined according to the frame ratio and the frame sequence number. “◯” in FIG. 6 indicates that the frame is updated. When the frame is updated, the bank of the bank from which the video frame (n) is read based on the previous frame synchronized with the synchronization reference signal. The imaging frame data stored in the next bank is read out as a video frame (n). That is, when performing frame update, the bank to be read is switched to the next bank. The same applies to the video frame (n−1).

  On the other hand, “x” in FIG. 6 indicates that the frame is not updated. When the frame is not updated, the video frame (n) is read based on the previous frame synchronized with the synchronization reference signal. The data of the imaging frame stored in the same bank as the performed bank is read again as the video frame (n). That is, when the frame is not updated, the bank to be read is not switched. The same applies to the video frame (n−1).

  The correspondence relationship among the frame ratio, the frame sequence number, and the coefficient k shown in FIG. 6 may be stored in advance as a table in a ROM (not shown) connected to the system controller 27, for example. When the control command is supplied, the system controller 27 refers to this table and determines the coefficient k based on the frame ratio and the frame sequence number superimposed on the control command.

  The coefficient k is not limited to this, and may be determined uniquely by the user. Further, for example, it may be calculated by a predetermined calculation. For example, the coefficient k of the video frame corresponding to the first frame sequence number and a frame sequence number that will not be updated as described later may be set to a value that changes by the reciprocal of the numerator value of the frame ratio. it can.

  In this example, the coefficient k takes a value that varies in a linear function according to the frame sequence number. However, the coefficient k is not limited to this, and may be a value that varies in a quadratic function, for example. Good.

  Next, an operation when the memory controller 30 reads the video frame (n) and the video frame (n−1) from each bank of the memory 24 will be described. The memory controller 30 selects a bank from which data of the imaging frame is read from information indicating frame update based on the frame ratio and the frame sequence number, and the video frame (n) and the video frame (n−1) synchronized with the synchronization reference signal. I am trying to read as.

  FIG. 7 shows an exemplary configuration of the memory controller 30. The memory controller 30 includes, for example, a switch 40, a switch 41, and a switch 42, and an output terminal of each switch is selected based on information indicating frame update.

  Each of the switches 40, 41 and 42 has three output terminals. The output terminals 40a, 41a and 42a are terminals for outputting the video frame (n). The output terminals 40b, 41b, and 42b are terminals that output the video frame (n-1). The output terminals 40c, 41c and 42c are terminals that do not output anything. For the switches 40, 41 and 42, different output terminals are selected, and the output terminals are switched based on the synchronization reference signal.

  FIG. 8 shows an example of a selection state of switches corresponding to each bank according to the frame sequence number shown in the example of FIG. 6 when the frame ratio is 2/3. Note that “n” in FIG. 8 indicates that the terminal that outputs the video frame (n) of the switch corresponding to the bank is selected, and “n−1” indicates the video frame (n−1). Indicates that the output terminal has been selected. “X” indicates that a terminal from which nothing is output is selected. In this example, for convenience, the terminal for outputting the video frame (n) of the switch 41 corresponding to the bank b is first selected.

  When the frame sequence number is 0, the output terminal 41a of the switch 41 is selected, the imaging frame data written in the bank b is output as a video frame (n), and the output terminal 40b of the switch 40 is selected. The data of the imaging frame written in the bank a is output as the video frame (n−1). At this time, in the switch 42 corresponding to the bank c, the output terminal 42c is selected, and the data of the imaging frame written in the bank c is not output.

  Since the frame is updated when the frame sequence number is 1, the output terminals of the switches 40, 41 and 42 are switched one by one. That is, the output terminal 42a of the switch 42 is selected, and the imaging frame data written in the bank c is read as the video frame (n), and the output terminal 41b of the switch 41 is selected and written in the bank b. The data of the imaging frame that is present is read out as a video frame (n−1). At this time, in the switch 40 corresponding to the bank a, the output terminal 40c is selected, and the data of the imaging frame written in the bank a is not output.

  When the frame sequence number is 2, since the frame is not updated, the selection state of the output terminals of the switches 40, 41 and 42 is maintained, the output terminal 42a of the switch 42 is selected, and the imaging frame written in the bank c is selected. The data is read again as a video frame (n), the output terminal 41b of the switch 41 is selected, and the data of the imaging frame written in the bank b is read again as the video frame (n-1). At this time, in the switch 40 corresponding to the bank a, the output terminal 40c is selected, and the data of the imaging frame written in the bank a is not output.

  Hereinafter, in the same manner as described above, the output terminals of the switches 40, 41, and 42 are sequentially selected according to the frame sequence number, and the data of the imaging frame written in each bank is the video frame (n) or the video frame (n -1).

  FIG. 9 shows an example of the selection state of switches corresponding to each bank corresponding to the frame sequence number shown in the example of FIG. 6 when the frame ratio is 4/5. Note that “n” in FIG. 9 indicates that the terminal that outputs the video frame (n) of the switch corresponding to the bank is selected, and “n−1” indicates the video frame (n−1). Indicates that the terminal to output is selected. “X” indicates that a terminal from which nothing is output is selected. In this example, for convenience, the terminal for outputting the video frame (n) of the switch 41 corresponding to the bank b is first selected.

  When the frame sequence number is 0, the output terminal 41a of the switch 41 is selected, the imaging frame data written in the bank b is read out as a video frame (n), and the output terminal 40b of the switch 40 is selected. Data of the imaging frame written in the bank a is read out as a video frame (n−1). At this time, in the switch 42 corresponding to the bank c, the output terminal 42c is selected, and the data of the imaging frame written in the bank c is not output.

  Since the frame is updated when the frame sequence number is 1, the output terminal 42a of the switch 42 is selected, and the data of the imaging frame written in the bank c is read as the video frame (n) and the output of the switch 41 The terminal 41b is selected, and the data of the imaging frame written in the bank b is read out as a video frame (n−1). At this time, in the switch 40 corresponding to the bank a, the output terminal 40c is selected, and the data of the imaging frame written in the bank a is not output.

  Since the frame is updated when the frame sequence number is 2, the output terminal 40a of the switch 40 is selected, and the data of the imaging frame written in the bank a is read out as the video frame (n) and the output of the switch 42 The terminal 42b is selected, and the data of the imaging frame written in the bank c is read as a video frame (n-1). At this time, in the switch 41 corresponding to the bank b, the output terminal 41c is selected, and the data of the imaging frame written in the bank b is not output.

  Since the frame is updated when the frame sequence number is 3, the output terminal 41a of the switch 41 is selected, and the data of the imaging frame written in the bank b is read as the video frame (n) and the output of the switch 40 The terminal 40b is selected, and the data of the imaging frame written in the bank a is read out as the video frame (n−1). At this time, in the switch 42 corresponding to the bank c, the output terminal 42c is selected, and the data of the imaging frame written in the bank c is not output.

  When the frame sequence number is 4, since the frame is not updated, the selection state of the output terminals of the switches 40, 41 and 42 is maintained, the output terminal 41a of the switch 41 is selected, and the imaging frame written in the bank b is selected. The data is read again as a video frame (n), the output terminal 40b of the switch 40 is selected, and the data of the imaging frame written in the bank a is read again as the video frame (n-1). At this time, in the switch 42 corresponding to the bank c, the output terminal 42c is selected, and the data of the imaging frame written in the bank c is not output.

  Hereinafter, in the same manner as described above, the output terminals of the switches 40, 41, and 42 are sequentially selected according to the frame sequence number, and the data of the imaging frame written in each bank is the video frame (n) or the video frame (n -1).

  As described above, the frame update is performed according to the frame sequence number shown in FIG. 6, and the data of the imaging frame written in each bank is converted into a frame based on the synchronization reference signal except when the frame update is not performed. In synchronism, video frames (n), video frames (n−1), and “none” are sequentially read out in this order.

  The above-described switch configuration and switch switching control in the memory controller 30 are examples for enabling the operation according to this embodiment, and are not limited thereto.

  Next, a method of generating a playback video frame when the frame ratio is 2/3 using the correspondence relationship between the frame ratio, the frame sequence number, and the coefficient k shown in FIG. 6 will be described with reference to FIG. To do. FIG. 10A shows the timing of writing and reading data in bank a, FIG. 10B shows the timing of writing and reading data in bank b, and FIG. 10C shows the timing of writing and reading data in bank c. FIG. 10D shows the timing of the imaging frame, and FIG. 10E shows the timing of the playback video frame.

  A frame ratio and a frame sequence number are supplied from the outside to the system controller 27, and one is sent for each frame sequence number at an arbitrary timing within one frame of the synchronization reference signal. Alternatively, the frame ratio and the frame sequence number are generated based on the synchronization reference signal by the system controller 27 of the imaging apparatus 11. Based on the frame ratio and the frame sequence number, the system controller 27 refers to the above table and determines the coefficient k and the number of pixels per line of the imaging frame. Based on the number of pixels per line of the imaging frame, the frame size of the imaging frame is set and imaging is started. Note that the relationship between reading and output of the bank a, bank b, and bank c follows the sequence of the switches 40, 41, and 42 as described with reference to FIGS.

  Data of the captured image frame # 1 is written into the bank a of the memory 24 (FIG. 10A). When the writing of the data of the imaging frame # 1 is completed, the data of the imaging frame # 2 is written into the bank b (FIG. 10B). When the writing of the data of the imaging frame # 2 is completed, the data of the imaging frame # 3 is written into the bank c (FIG. 10C).

  The frame ratio and the frame sequence number (frame ratio = 2/3, frame sequence number = 0) are supplied to the system controller 27. The data of the imaging frame # 2 written to the bank b and the data of the imaging frame # 1 written to the bank a are the position of the head of the frame based on the synchronization reference signal immediately after the frame ratio and the frame sequence number are supplied. And the data of the imaging frame # 2 is multiplied by the coefficient k (= 1), and the data of the imaging frame # 1 is multiplied by the coefficient 1-k (= 0), and the two data are mixed for each pixel. And output as playback video frame # 14.

  When the writing of the data of the imaging frame # 3 is completed, the data of the imaging frame # 4 is written into the bank a. The frame ratio and the frame sequence number (frame ratio = 2/3, frame sequence number = 1) are supplied to the system controller 27. The data of the imaging frame # 3 written in the bank c and the data of the imaging frame # 2 written in the bank b are the head position of the frame based on the synchronization reference signal immediately after the frame ratio and the frame sequence number are supplied. And the data of the imaging frame # 3 is multiplied by the coefficient k (= 0.5), and the data of the imaging frame # 2 is multiplied by the coefficient 1-k (= 0.5) to obtain two data. Each pixel is mixed and output as a reproduction video frame # 15.

  The frame ratio and the frame sequence number (frame ratio = 2/3, frame sequence number = 2) are supplied to the system controller 27. Since the frame is not updated when the frame sequence is C, the frame ratio and the frame sequence number are supplied to the data of the imaging frame # 3 written in the bank c and the data of the imaging frame # 2 written to the bank b. The data is read again from the head position of the frame based on the immediately subsequent synchronization reference signal, the data of the imaging frame # 3 is multiplied by the coefficient k (= 1), and the data of the imaging frame # 2 is multiplied by the coefficient 1-k (= 0). ) And the two data are mixed for each pixel and output as a reproduction video frame # 16. When the writing of the data of the imaging frame # 4 is completed, the data of the imaging frame # 5 is written into the bank b. Note that the value of the coefficient k multiplied by the data of the imaging frame # 3 is 1 because the imaging frame that is multiplied by the coefficient k is read out from the same imaging frame as before the one frame sequence number without being updated. is there.

  Here, the data of the imaging frame # 5 is written to the bank b while the data of the imaging frame # 2 is being read from the bank b. In this case, for example, it is written in the portion where the reading of the data of the imaging frame is completed.

  Thereafter, in the same manner as described above, the imaging frame is written to and read from each bank of the memory 24, and a reproduced video frame is generated.

  Next, a method for generating a playback video frame when the frame ratio is 4/5 will be described with reference to FIG. FIG. 11A shows data write and read timings for bank a, FIG. 11B shows data write and read timings for bank b, and FIG. 11C shows data write and read timings for bank c. FIG. 11D shows the timing of the imaging frame, and FIG. 11E shows the timing of the playback video frame.

  Data of the imaged frame # 1 is written in the bank a of the memory 24 (FIG. 11A). When the writing of the data of the imaging frame # 1 is completed, the data of the imaging frame # 2 is written to the bank b (FIG. 11B). When the writing of the data of the imaging frame # 2 is completed, the data of the imaging frame # 3 is written to the bank c (FIG. 11C).

  The frame ratio and the frame sequence number (frame ratio = 2/3, frame sequence number = 0) are supplied to the system controller 27. The data of the imaging frame # 2 written to the bank b and the data of the imaging frame # 1 written to the bank a are the position of the head of the frame based on the synchronization reference signal immediately after the frame ratio and the frame sequence number are supplied. And the data of the imaging frame # 2 is multiplied by the coefficient k (= 1), and the data of the imaging frame # 1 is multiplied by the coefficient 1-k (= 0), and the two data are mixed for each pixel. And output as playback video frame # 14.

  When the writing of the data of the imaging frame # 3 is completed, the data of the imaging frame # 4 is written into the bank a. The frame ratio and the frame sequence number (frame ratio = 2/3, frame sequence number = 1) are supplied to the system controller 27. The data of the imaging frame # 3 written in the bank c and the data of the imaging frame # 2 written in the bank b are the head position of the frame based on the synchronization reference signal immediately after the frame ratio and the frame sequence number are supplied. And the data of the imaging frame # 3 is multiplied by a coefficient k (= 0.75) and the data of the imaging frame # 2 is multiplied by a coefficient 1-k (= 0.25). Each pixel is mixed and output as a reproduction video frame # 15.

  When the writing of the data of the imaging frame # 4 is completed, the data of the imaging frame # 5 is written into the bank b. The frame ratio and the frame sequence number (frame ratio = 2/3, frame sequence number = 2) are supplied to the system controller 27. The data of the imaging frame # 4 written in the bank a and the data of the imaging frame # 3 written in the bank c are the head position of the frame based on the synchronization reference signal immediately after the frame ratio and the frame sequence number are supplied. And the data of the imaging frame # 4 is multiplied by a coefficient k (= 0.5), and the data of the imaging frame # 3 is multiplied by a coefficient 1-k (= 0.5) to obtain two data. Each pixel is mixed and output as a reproduction video frame # 16.

  The frame ratio and the frame sequence number (frame ratio = 2/3, frame sequence number = 3) are supplied to the system controller 27. The data of the imaging frame # 5 written to the bank b and the data of the imaging frame # 4 written to the bank a are the head position of the frame based on the synchronization reference signal immediately after the frame ratio and the frame sequence number are supplied. And the data of the imaging frame # 5 is multiplied by a coefficient k (= 0.25) and the data of the imaging frame # 4 is multiplied by a coefficient 1-k (= 0.75). Each pixel is mixed and output as a reproduction video frame # 17.

  When the writing of the data of the imaging frame # 5 is completed, the data of the imaging frame # 6 is written to the bank c. The frame ratio and the frame sequence number (frame ratio = 2/3, frame sequence number = 4) are supplied to the system controller 27. Since the frame is not updated when the frame sequence is E, the data of the imaging frame # 5 written in the bank b and the data of the imaging frame # 4 written in the bank a are supplied with the frame ratio and the frame sequence number. The data is read again from the head position of the frame based on the immediately subsequent synchronization reference signal, the data of the imaging frame # 5 is multiplied by the coefficient k (= 1), and the data of the imaging frame # 4 is multiplied by the coefficient 1-k (= 0). ) And the two data are mixed for each pixel and output as a reproduction video frame # 18. Note that the value of the coefficient k multiplied by the data of the imaging frame # 3 is 1 because the imaging frame that is multiplied by the coefficient k is read out from the same imaging frame as before the one frame sequence number without being updated. is there.

  Here, the data of the imaging frame # 5 is written while the data of the imaging frame # 2 written in the bank b is being read. In this case, for example, data is written in a portion where reading has been completed.

  Thereafter, in the same manner as described above, the imaging frame is written to and read from each bank of the memory 24, and a reproduced video frame is generated.

  In this way, by continuously multiplying the video frame at a certain point and the previous video frame by a predetermined coefficient, the reproduced video frame obtained by mixing the data of the two frames for each pixel is displayed. The video signal can be shown smoothly.

  In this example, the variable speed imaging method in the case where the frame ratio is 2/3 and 4/5 has been described, but this is not limited to this example. For example, the frame ratio is applicable to a frame ratio represented by an arbitrary integer ratio between 1/30 and 30/30. That is, it can cope with 1 to 30 times speed shooting. Further, the case where the frame size of the video frame is 2200 pixels × 1125 lines has been described, but this is an example, and the present invention can be applied to other frame sizes.

  Further, the coefficient k and the information indicating the frame update are described based on the frame ratio and the frame sequence number sent from the controller 10 to the system controller 27. However, the present invention is not limited to this. Information indicating the coefficient k and frame update may be directly supplied to the controller 27.

1 is a block diagram illustrating a configuration of an example of an imaging apparatus 11 that can be applied to an embodiment of the present invention. It is a basic diagram which shows an example of an imaging frame. It is a basic diagram which shows the timing of the imaging frame and video frame at the time of performing variable speed imaging with a frame ratio of 2/3. It is a basic diagram which shows the timing of the imaging frame and video frame at the time of performing variable speed imaging with a frame ratio of 4/5. It is a block diagram which shows the structure of an example of a digital signal processing part. It is a basic diagram which shows the correspondence of an example of a frame ratio, a frame sequence number, and a coefficient. It is a block diagram which shows the structure of an example of a memory controller. It is a basic diagram which shows an example of operation | movement of a memory controller according to a frame sequence number when a frame ratio is 2/3. It is a basic diagram which shows an example of operation | movement of a memory controller according to the frame sequence number when a frame ratio is set to 4/5. It is a basic diagram which shows the timing of the reproduction | regeneration video frame when a frame ratio is 2/3. It is a basic diagram which shows the timing of the reproduction | regeneration video frame when a frame ratio is set to 4/5.

Explanation of symbols

11 Imaging device 23 Digital signal processing unit 24 Memory 25 Timing signal generating unit 26 Imaging timing signal generating unit 27 System controller 28 Display device 30 Memory controller 31, 32 Multiplier 33 Adder 40, 41, 42 Switch

Claims (4)

An imaging unit for imaging light from a subject;
An imaging drive unit that drives the imaging unit to capture an imaging frame having a frame frequency lower than the frame frequency based on the synchronization reference signal;
A storage unit for sequentially storing imaging frames output from the imaging unit;
A signal processing unit that generates a reproduction video frame obtained by mixing the imaging frame stored in the storage unit and the imaging frame immediately before the imaging frame at a predetermined ratio, and outputs the frame based on the synchronization reference signal; I have a,
The storage unit includes at least three banks for storing the imaging frames,
The imaging frame is written to a selected bank among the three banks sequentially switched in synchronization with the timing of the imaging frame,
Based on frame update information indicating the bank switching in synchronization with the synchronization reference signal, the three banks are switched to a predetermined value, and the imaging frame used when generating the reproduced video frame and the imaging immediately before the imaging frame An image pickup apparatus that selects a frame . The imaging device according to claim 1,
The signal processing unit is configured to generate the reproduced video frame by mixing based on a time ratio between the output reproduced video frame and the imaging frame used for the reproduced video frame. Imaging device. Capture an imaging frame with a frame frequency lower than the frame frequency based on the synchronization reference signal,
The storage portion having at least three banks for storing image pickup frame, the selected bank of the sequentially switched the three banks in synchronization with the timing of the image pickup frame, stores sequential the imaged captured frame ,
Based on frame update information indicating the bank switching in synchronization with the synchronization reference signal, the three banks are switched to a predetermined value, and the imaging frame used when generating the reproduced video frame and the imaging immediately before the imaging frame Select each frame and
Generate one previous playback video frame and an image pickup frame obtained by mixing a predetermined ratio of selected said image pickup frame and the image pickup frame, film image output method configured to output on the basis of the sync reference signal. Capture an imaging frame with a frame frequency lower than the frame frequency based on the synchronization reference signal,
The storage portion having at least three banks for storing image pickup frame, the selected bank of the sequentially switched the three banks in synchronization with the timing of the image pickup frame, stores sequential the imaged captured frame ,
Based on frame update information indicating the bank switching in synchronization with the synchronization reference signal, the three banks are switched to a predetermined value, and the imaging frame used when generating the reproduced video frame and the imaging immediately before the imaging frame Select each frame and
A video output method for generating a reproduction video frame in which the selected imaging frame and the imaging frame immediately before the imaging frame are mixed at a predetermined ratio and outputting the generated video frame based on the synchronization reference signal Film image output program that is executed devices.

Images