JP2015207944A - Video camera apparatus, processing method of video signal, and video signal processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video camera apparatus which reduces flickering, a processing method of a video signal, and a video signal processor.SOLUTION: The video camera apparatus according to the present invention is configured to perform imaging under lighting of which the illuminance is cyclically varied. The video camera apparatus includes: an imaging section which outputs video signals at a frame rate of a final video signal generated by the video camera apparatus and a frame rate to be a least common multiple with a fluctuation frequency of the luminance of lighting; and a signal processing section including an adder which adds video signals outputted by the imaging section over a time equal to a variation cycle of the luminance of an illumination for the unit of a frame. The signal processing section outputs the final video signal by defining the video signals which are added by the adder over the time equal to the variation cycle, as video signals for one frame.

Description

  The present invention relates to a video camera device, a video signal processing method, and a video signal processing device that reduce the occurrence of flicker.

  Video signals for television and video employ an interlaced scanning method or a progressive scanning method, and many of them have the ability to capture and display 30 to 60 frame images per second. On the other hand, in recent years, TVs and video cameras have been developed with higher frame rate and a higher frame rate in order to cope with the higher recognition ability of human animals as well as higher resolution of TVs. Yes. In Super Hi-Vision, which is being studied for standardization as a next-generation broadcasting system, a frame rate of 60 to 120 frames / second has been adopted as an international standard.

  By the way, the frequency of domestic commercial power supply is 60 Hz or 50 Hz depending on the region where it is used. In a lighting fixture using a general fluorescent lamp that does not employ a high-frequency lighting method (inverter method), the luminance is 100 times per second (in the case of a 50 Hz region) or 120 times (in the case of a 60 Hz region). There are fluctuations. Even overseas, for example, 50 Hz is adopted as the frequency of the commercial power supply in Europe, and the luminance variation of 100 times per second occurs in the lighting of fluorescent lamps.

  A flicker problem may occur when a video camera with a frame rate of, for example, 60 frames / second corresponding to Super Hi-Vision is used under illumination using a fluorescent lamp other than the high-frequency lighting method. Here, the flicker generation mechanism will be described with reference to FIG. In an area where the frequency of the commercial power supply is 50 Hz, the waveform of the supplied AC power supply changes in a sine wave shape with a period of 1/50 seconds as shown in FIG. At this time, the luminance of the fluorescent lamp other than the high-frequency lighting method varies at 100 Hz which is a double frequency as shown in FIG. The video signal output of the image sensor based on the fluorescent lamp illumination is the time integration of the luminance of the fluorescent lamp shown in FIG. 4B, but one frame period (1/60 seconds) of the video camera and the fluctuation of the fluorescent lamp luminance. Since the period (1/100 second) does not match, the integrated value varies from frame to frame as shown in FIG. This is a phenomenon called so-called flicker. When a video shot under such conditions is played back, flickering occurs due to the brightness of the image changing from frame to frame. On the other hand, in a region where the commercial power supply is 60 Hz, one frame period of the video camera is exactly twice the luminance fluctuation period of the fluorescent lamp, and thus flicker does not occur.

  In order to prevent the flicker generated by such a principle, for example, in Patent Document 1, an electronic shutter is used and the exposure time is set to 1/100 second. With such a configuration, the integral value of the amount of light incident on the image sensor within the exposure time becomes constant for each frame, and flicker due to fluorescent lamp illumination can be reduced. The principle of the invention will be described with reference to FIG. In the present invention, as shown in FIG. 5C, the time during which the electronic shutter is open in each frame is matched with the luminance fluctuation period of the fluorescent lamp shorter than one frame period of the image sensor. As a result, the degree of influence of each frame from the fluorescent lamp becomes uniform, and flicker due to luminance fluctuation of the fluorescent lamp can be reduced as shown in FIG.

Japanese Patent No. 3123653 Japanese Patent No. 2553123

  However, in a video system that operates at 120 frames / second, which is adopted in Super Hi-Vision, one frame cycle is 1/120 seconds, which is shorter than the fluorescent light luminance fluctuation cycle (1/100 seconds). Since the exposure time cannot be set to 1/100 second using an electronic shutter, the technique as disclosed in Patent Document 1 that has been effective for removing flickers does not function effectively. Therefore, flicker as shown in FIG. 6D occurs.

  Further, Patent Document 2 proposes a method for performing flicker removal by generating an exposure end pulse at an interval that is an integral multiple of the luminance fluctuation cycle of illumination light. However, even in this method, one frame cycle ( (1/120 seconds) is shorter than the luminance fluctuation period (1/100 seconds) of the illumination light, so that the exposure start pulse and the exposure end pulse cannot be reliably obtained for each frame, and flicker is effectively removed. I can't.

  As another method, a method of reducing flicker by detecting a change in the signal level for each frame and changing the gain of the read amplifier in accordance with the change is conceivable. However, according to that method, when illumination with varying luminance and illumination with unvariable luminance are mixed, the occurrence of flicker may be promoted.

  An object of the present invention made in view of such a point is to provide a video camera device, a video signal processing method, and a video signal processing device in which occurrence of flicker is reduced.

  In order to solve the above-described problem, a video camera device according to the present invention is a video camera device, and is a minimum of a frame rate of a final video signal output from the video camera device and a variation frequency of luminance of illumination that illuminates a subject. An imaging unit that generates a video signal at a frame rate that is a common multiple, and a signal processing unit that includes an addition unit that adds the video signal generated by the imaging unit over a time equal to the fluctuation cycle of the luminance of the illumination in units of frames. The signal processing unit outputs the final video signal using the video signal added by the adding unit over a time equal to the fluctuation period as a video signal of one frame.

  The adding unit includes a first adder and a second adder, and the second adder is delayed by a time corresponding to a frame period of the final video signal from the first adder. The addition is started from a frame, and the first adder and the second adder each add a video signal generated by the imaging unit over a time equal to the fluctuation period, and the signal processing unit Preferably, the video signal after the addition by the first adder and the video signal after the addition by the second adder are alternately output in the frame cycle of the final video signal.

  In order to solve the above-described problem, a video signal processing method according to the present invention is a video signal processing method, in which a frame rate of a final video signal generated by the processing method and a luminance of illumination that illuminates a subject. An imaging step for generating a video signal at a frame rate that is the least common multiple of the fluctuation frequency of the image, and an addition step for adding the video signal generated by the imaging step in units of frames over a time equal to a fluctuation cycle of the luminance of the illumination. And a video output step of outputting the final video signal by using the video signal added over the time equal to the fluctuation period by the adding step as a one-frame video signal.

  In order to solve the above-described problem, a video signal processing device according to the present invention is a video signal processing device having a frame rate of a final video signal output from the video signal processing device and a luminance of illumination for illuminating a subject. A receiving unit that receives a video signal having a frame rate that is the least common multiple of the fluctuation frequency from the imaging unit, and an addition that adds the video signal from the imaging unit in units of frames over a time equal to the fluctuation cycle of the luminance of the illumination. And the final video signal is output using the video signal added by the adding unit over a time equal to the fluctuation period as a video signal of one frame.

  According to the present invention, it is possible to provide a video camera device, a video signal processing method, and a video signal processing device that reduce the occurrence of flicker.

1 is a block diagram of a video camera apparatus according to an embodiment of the present invention. FIG. 4 is a timing diagram for explaining the principle of flicker removal in the video camera device according to the embodiment of the present invention. It is a block diagram (at the time of operation) of a video camera device concerning one embodiment of the present invention. It is a timing diagram for explaining the principle of flicker generation in a conventional video camera device or the like. 10 is a timing chart for explaining the principle of flicker removal in the solid-state imaging device according to Patent Document 1. FIG. FIG. 10 is a timing chart when the frame rate is increased to 120 frames / second in the solid-state imaging device according to Patent Document 1.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings.

(One embodiment)
FIG. 1 is a block diagram of a video camera apparatus 400 according to an embodiment of the present invention. A video camera device 400 according to the present embodiment includes an imaging unit 401, a video signal processing unit 402, a recording processing unit 403, and a control unit 404. In FIG. 1, the flow of the video signal is indicated by a solid line, and the flow of a control signal such as a timing signal is indicated by a broken line.

  First, the configuration of the imaging unit 401 will be described.

  The imaging unit 401 includes a lens 41, an imaging element 42, and an imaging element drive circuit 43. The imaging unit 401 plays a role of forming a subject image on the image sensor 42 and obtaining a video signal.

  The lens 41 is a plurality of lenses for forming an image of a subject on an image sensor 42 described later. Some of the plurality of lenses are arranged on a movable portion of a driving device (not shown) and are driven in the optical axis direction by a control signal from a control unit 404 described later. By this lens driving, the zoom lens function and the autofocus function of the video camera device 400 are realized.

  The image sensor 42 detects light from the subject guided through the lens 41 and generates a video signal. For example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor can be used as the imaging element 42. The image sensor 42 can be configured to obtain video signals of three primary colors from a single-plate image sensor by disposing various color filters (not shown) on the image sensor. In addition, the image pickup element 42 can increase the light use efficiency by arranging a microlens or the like (not shown) on the light receiving element.

  The image sensor driving circuit 43 is a circuit for operating the image sensor 42. The image sensor drive circuit 43 receives a timing signal and the like from the control unit 404 as indicated by a broken line in FIG. Based on the timing signals and the like, the image sensor driving circuit 43 generates signals such as an exposure clock and an accumulation clock for operating the image sensor 42 and supplies the signals to the image sensor 42.

  The image sensor drive circuit 43 performs control so that the image sensor 42 operates at a frame rate of 600 frames / second based on a control signal from the control unit 404. Details will be described later.

  Next, the configuration of the video signal processing unit 402 will be described.

  The video signal processing unit 402 includes a first adder 44, a second adder 45, and a switch 46. In the present embodiment, the first adder 44 and the second adder 45 have a function of adding and outputting six consecutive frames of a video signal of 600 frames / second output from the image sensor 42. Here, the second adder 45 starts addition from the sixth frame after the first adder 44 adds five consecutive frames. That is, each of the first adder 44 and the second adder 45 adds six consecutive video signals, but only one of the six frames overlaps between the adders 44 and 45. Add. The first adder 44 and the second adder 45 output the added video signals for 6 frames to the switch 46 for at least 1/120 second. The second adder 45 is configured to output a video signal to the switch 46 with a delay of 1/120 second with respect to the first adder 44. Based on the control signal from the control unit 404, the switch 46 transmits the output of the first adder 44 to the recording processing unit 103 for 1/120 seconds during which the first adder 44 outputs the video signal. Can be switched to. The switch 46 is switched so that the output of the second adder 45 is transmitted to the recording processing unit 103 during 1/120 second when the second adder 45 outputs the video signal. Note that the switch 46 is configured by a relay, a transistor, or the like.

  Next, the configuration of the recording processing unit 403 will be described.

  The recording processing unit 403 includes an encoder 47, a modulation circuit 48, and a recording device 49.

  The encoder 47 has a function of encoding the video signal from the image sensor 42 processed by the video signal processing unit 402 and performing image compression. Examples of the encoding method supported by the encoder 47 include the MPEG-2 method and the MPEG-4 Part 10 Advanced Video Coding method.

  The modulation circuit 48 has a function of modulating the data output from the encoder 47 into a format that can be recorded on a recording medium in the recording device 49.

  The recording device 49 has a function of recording the video signal modulated by the modulation circuit 48 on a recording medium. In this embodiment, recording media that can be used for recording video signals include, for example, flash memory, HDD (registered trademark) (Hard Disc Drive), DVD (registered trademark) (Digital Versatile Disc), or BD (Blu-ray ( (Registered trademark) Disc).

  Note that a buffer memory (not shown) or the like may be provided as appropriate in the recording processing unit 403 so that the processing in each block is performed more smoothly.

  In this embodiment, the recording device 49 is configured to record the video signal on the recording medium. However, the present invention is not limited to this form. For example, the video signal encoded by the encoder 47 may be transmitted to a data server via a wired or wireless network, and the video signal may be managed and stored on the data server.

  Next, the control unit 404 will be described.

  The control unit 404 includes a lens 41 and an image sensor driving circuit 43 in the imaging unit 401, a first adder 44, a second adder 45 and a switch 46 in the video signal processing unit 402, and an encoder 47 in the recording processing unit 403. In addition, it has a function of transmitting control signals to the modulation circuit 48 and the recording device 49, etc., and performing these controls. The control unit 404 can be configured by, for example, a microcomputer that can execute a program.

  The control unit 404 and the video signal processing unit 402 in the present embodiment constitute a “video signal processing device” of the present invention.

  Next, the principle of flicker removal according to the present embodiment will be described with reference to the timing chart shown in FIG.

  FIG. 2A shows a waveform (50 Hz) of a commercial power source supplied from the system, and FIG. 2B shows a time change of the luminance of a fluorescent lamp that is turned on by supplying power from the commercial power source. . As can be seen from FIG. 2B, the luminance waveform of the fluorescent lamp is a waveform that fluctuates at a cycle of 100 Hz, which is a frequency twice the commercial power supply frequency.

  FIG. 2C shows a temporal change of the luminance signal in the video signal output by the image sensor 42 at a frame rate of 600 frames / second. It can be seen that the luminance signal of the video signal changes corresponding to the temporal change of the luminance of the fluorescent lamp shown in FIG. The adders 44 and 45 respectively add the continuous 6 frames of video signals output at 600 frames / second. The first adder 44 and the second adder 45 are used alternately. First, the first adder 44 adds the video signals of the first six frames in FIG. Then, the added video signal is output to the switch 46 for at least 1/120 second. The controller 404 switches the switch 46 at the same time that the first adder 44 outputs the added video signal, and performs control so that the video signal output of the first adder 44 is transmitted to the recording processing unit 403.

  When the first adder 44 adds the sixth frame, the second adder 45 starts adding six consecutive frames with the frame as the first frame. That is, the first adder 44 and the second adder 45 add 6 consecutive frames while overlapping only one frame.

  The second adder 45 performs addition of video signals of six consecutive frames from the sixth frame in FIG. Then, the added video signal is output to the switch 46 for at least 1/120 second. The control unit 404 switches the switch 46 at the same time as the second adder 45 outputs the added video signal, and controls the video signal output of the second adder 45 to be transmitted to the recording processing unit 403. As can be seen from the above description, the first adder 44 and the second adder 45 are shifted by 5 frames, that is, 1/600 × 5 = 1/120 seconds. Accordingly, the timing at which the adders 44 and 45 output the added video signal is also shifted by 1/120 seconds. Therefore, the control unit 404 performs switching control of the switch 46 every 1/120 second, and alternately outputs the video signal from the first adder 44 and the video signal from the second adder 45 to the recording processing unit 403. And transmit.

  As described above, the video signal is output from the first adder 44 and the second adder 45 every 1/120 second, whereby the video signal processing unit 402 generates a video signal of 120 frames / second. By the way, the generated signal of each frame is generated from subject light detected for 6 consecutive frames of the video signal of 600 frames / second output from the image sensor 42, that is, 1/100 second. Since this 1/100 second is equal to the luminance fluctuation cycle of the fluorescent lamp, which is the illumination light, the 120 frame / second video signal output from the video signal processing unit 402 is not affected by the luminance fluctuation of the fluorescent lamp. FIG. 2D shows a temporal change in the magnitude of the luminance signal in the video signal output from the video signal processing unit 402, and flicker does not occur.

  As described above, the video signal processing unit 402 adds one frame of a 120-frame / second video signal by adding the continuous 6-frame video signal from the image sensor 42 shown in FIG. Configured to generate. In order to show the correspondence, it is described that the video signal after addition is output in FIG. 2D when 6 consecutive frames in FIG. 2C are output. However, the video signal in FIG. 2D is determined only when the sixth frame in FIG. 2C is output. Accordingly, the output of the video signal in FIG. 2D is actually performed after the sixth frame in FIG. 2C is output.

  FIG. 3 schematically shows the flow of each video signal in the video camera device 400 when the video signal is processed at the timing shown in FIG. In FIG. 3, frames (1) to (6) out of frames (1) to (11) in a signal of 600 frames / second supplied from the image sensor 42 are added by the first adder 44 to obtain a frame (6 ) To (11) are added by the second adder 45. Each video signal added in the adders 44 and 45 is switched by the switch 46 to become a video signal of 120 frames / second, and is transmitted to the recording processing unit 403.

  In addition, although this embodiment demonstrated the case where the fluctuation frequency of the brightness | luminance of illumination light was 100 Hz, it is not limited to this form. For example, by configuring the first adder 44 and the second adder 45 to add and output consecutive five frames, it is possible to cope with a luminance fluctuation frequency of 120 Hz.

  In the present embodiment, the case where the frame rate is 120 frames / second has been described. However, the present invention is not limited to this mode. For example, the first adder 44 and the second adder 45 are configured such that the frames to be added are shifted by 4 frames, that is, 1/600 × 4 = 1/150 seconds, and the switch 46 is changed every 1/150 seconds. By switching to, a video signal of 150 frames / second can be generated.

  In the present embodiment, the case where the fluctuation period of the illumination light is longer than the frame period has been described, but the present invention can also be applied to a case where the fluctuation period of the illumination light is shorter.

  As described above, according to the present embodiment, since the continuous 6 frames of the video signal of 600 frames / second output from the image sensor 42 are added to form a video signal of 1 frame, Flickering can be suppressed even when shooting is performed under illumination light that fluctuates with frequency. In addition, since only one image sensor is used, a low-cost device can be provided.

  Further, according to the present embodiment, since the frame rate of the video signal output from the video signal processing unit 402 is 120 frames / second, the occurrence of flicker in the video signal compatible with Super Hi-Vision can be suppressed.

  Although the present invention has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention. For example, the functions included in each component, each step, etc. can be rearranged so that there is no logical contradiction, and multiple components, steps, etc. can be combined or divided into one It is. Further, although the present invention has been described mainly with respect to the apparatus, the present invention can also be realized as a method, a program executed by a processor included in the apparatus, or a storage medium storing the program, and is within the scope of the present invention. It should be understood that these are also included.

DESCRIPTION OF SYMBOLS 41 Lens 42 Image pick-up element 43 Image pick-up element drive circuit 44 1st adder 45 2nd adder 46 Switch 47 Encoder 48 Modulation circuit 49 Recording device 400 Video camera apparatus 401 Image pick-up part 402 Video signal processing part 403 Recording processing part 404 Control part

Claims (4)

A video camera device,
An imaging unit that generates a video signal at a frame rate that is a least common multiple of a frame rate of a final video signal output by the video camera device and a luminance variation frequency of illumination that illuminates a subject;
A signal processing unit having an addition unit that adds a video signal generated by the imaging unit over a time equal to a fluctuation cycle of the luminance of the illumination in units of frames;
The video processing apparatus, wherein the signal processing unit outputs the final video signal by using the video signal added by the adding unit over a time equal to the fluctuation period as a video signal of one frame. The adding unit includes a first adder and a second adder,
The second adder starts the addition from a frame delayed by a time corresponding to a frame period of the final video signal from the first adder;
The first adder and the second adder each add a video signal output by the imaging unit over a time equal to the fluctuation period in units of frames,
The signal processing unit alternately outputs the video signal after addition by the first adder and the video signal after addition by the second adder in a frame cycle of the final video signal. The video camera device described. A method of processing a video signal,
An imaging step of outputting the video signal at a frame rate that is the least common multiple of the frame rate of the final video signal generated by the processing method and the luminance variation frequency of the illumination that illuminates the subject;
An addition step of adding the video signal generated by the imaging step in units of frames over a time equal to the fluctuation cycle of the luminance of the illumination; and a video signal added over a time equal to the fluctuation cycle by the addition step A video signal processing method comprising: a video output step of outputting the final video signal as a video signal. A video signal processing apparatus,
A receiving unit that receives from the imaging unit a video signal having a frame rate that is a least common multiple of the frame rate of the final video signal output by the video signal processing device and the luminance variation frequency of the illumination that illuminates the subject;
An addition unit that adds the video signal from the imaging unit in units of frames over a time equal to the fluctuation cycle of the luminance of the illumination;
The video signal processing apparatus, wherein the final video signal is output by using the video signal added by the adder over a time equal to the fluctuation period as a video signal of one frame.

Images