JP5360589B2 - Imaging device - Google Patents

Description

  The present invention relates to an image capturing apparatus such as an in-vehicle camera used by being mounted on a vehicle, and more particularly to an NTSC output image capturing apparatus.

  In general, in-vehicle cameras shoot at a shooting frame number of about 30 fps with an image sensor such as a CMOS sensor in order to match an NTSC display device or the like. When, for example, an LED type traffic light on the road is photographed with such a vehicle-mounted camera, the following problems occur.

  Since the LED type traffic light is driven by a driving voltage obtained by full-wave rectification of a commercial AC power supply, it cannot be visually confirmed, but the lighting and extinguishing of the signal is repeated periodically in a very short time. Therefore, if the charge accumulation time of the image sensor and the signal extinction period overlap in a certain frame, the extinction state of that frame is actually off even if any of red, blue, or green is on As a result. Then, when such frames are continued, an image obtained by photographing a traffic light in an extinguished state is output over several tens of frames. This is a particularly serious problem in a drive recorder. A drive recorder records video taken by an in-vehicle camera and is widely used when conducting accident analysis. However, if the exact state of a traffic light at the time of an accident or near-miss can not be determined, Obstacles will arise in responsibility determination.

  In order to cope with the above problems, it is known to set the frame frequency of the image sensor to a frame frequency (offset frame frequency) offset by a predetermined offset frequency with respect to the NTSC frame frequency (for example, , See Patent Document 1). According to this, even if an LED type traffic light or the like is photographed, an unlit video is not output over several tens of frames at the time of lighting, but the frame frequency of the video signal is out of the NTSC frame frequency. Therefore, it cannot be displayed on the NTSC display device as it is.

  As a countermeasure against this, image data captured from the image sensor is stored in a frame memory at a frame frequency (offset frame frequency) that is excluded from the NTSC frame frequency, and is read from the frame memory at the NTSC frame frequency. Can be output to However, in this case, in addition to the existing NTSC system oscillator, the clock frequency of the existing NTSC system oscillator for outputting image data at a frame frequency that deviates from the NTSC frame frequency. It is necessary to mount a new oscillator that generates clocks having different clock frequencies.

  Normally, reading of image data from the image sensor is performed in a progressive manner. On the other hand, an NTSC display device employs an interlace method. Therefore, the image data read from the image sensor is written to the frame memory by the progressive method, and read from the frame memory by the interlace method. In this case, in the above method, the time for writing one frame of image data to the frame memory is different from the time for reading one frame of image data from the frame memory. Therefore, even if the frame memory is a double buffer, it is peculiar to the interlace method. Comb noise is generated. Comb noise is comb-like image shift that occurs in a frame image composed of odd and even fields in different frames.

The present invention makes it difficult to be affected by a blinking cycle when a lighting device such as an LED traffic light is lit, and an image pickup device at a frame frequency that is out of the NTSC frame frequency so that the NTSC display device can be used as it is. In the image pickup apparatus in which image data is read out from the frame memory, written in the frame memory, and read out from the frame memory at the frame frequency of the NTSC system, the clock oscillator mounted on the apparatus is only an oscillator for generating a clock signal in the NTSC system An object is to realize a low-cost captured image device and to minimize the influence of comb noise generation.

According to the first aspect of the present invention, an optical system for forming an optical image of a subject and an optical image formed by the optical system are picked up, and image data is output at a frame frequency Ffs deviating from the NTSC frame frequency Ffr. an imaging device that includes a frame memory for storing image data, writing the image data output from the imaging element to the frame memory, from the frame memory and a control unit that reads the frame frequency Ffr of the NTSC system, the NTSC system corresponding A single oscillator for generating a clock signal , and the control unit controls reading from the frame memory at the frame frequency Ffr of the NTSC system based on the clock of the clock frequency corresponding to the NTSC system from the oscillator. The image sensor operates by inputting a clock having a clock frequency corresponding to the NTSC system from the oscillator. The image pickup device outputs image data at a frame frequency Ffs deviating from the NTSC frame frequency Ffr by changing the blanking period, and the image pickup device has an adjacent frame period indefinite, In addition, the image data is output at a variable frame frequency Ffs, which averages the frame period of the NTSC system.

According to a second aspect of the present invention, in the image pickup apparatus according to the first aspect , the image pickup device includes a register, and the vertical blanking period is changed according to a parameter value set in the register.

According to a third aspect of the present invention, in the image imaging device according to the second aspect , the CPU has a CPU, and the CPU sets a parameter value of a register in the imaging element.

According to a fourth aspect of the present invention, an optical system that forms an optical image of a subject and an optical image formed by the optical system are picked up, and image data is output at a frame frequency Ffs that deviates from the NTSC frame frequency Ffr. An image sensor that stores image data, a frame memory that stores image data, a controller that writes image data output from the image sensor into the frame memory and reads the frame memory at an NTSC frame frequency Ffr, and an NTSC system compatible A single oscillator for generating a clock signal is provided, and the control unit receives an NTSC-compatible clock from the oscillator.
Based on the clock of the lock frequency, reading from the frame memory at the frame frequency Ffr of the NTSC system is controlled, and the image sensor operates by inputting a clock of the clock frequency corresponding to the NTSC system from the oscillator, and blanking. By changing the period, the image pickup apparatus outputs image data at a frame frequency Ffs that deviates from the NTSC frame frequency Ffr. The image data is output at the frame frequency Ffr, and the image data is output at the frame frequency Ffs deviating from the NTSC frame frequency Ffr only when the exposure time is a predetermined value or less.

According to a fifth aspect of the present invention, in the image pickup device according to the fourth aspect , the image pickup device has a register, and outputs image data at an NTSC frame frequency Ffr according to a parameter value set in the register. Alternatively, the blanking period is changed so that image data is output at a frame frequency Ffs deviating from the NTSC frame frequency Ffr.

A sixth aspect of the present invention is the image pickup apparatus according to the fifth aspect , further comprising a CPU, which fetches an exposure time from the image sensor and changes a parameter value of a register of the image sensor in accordance with the exposure time. It is characterized by doing.

According to the seventh aspect of the present invention, an optical system that forms an optical image of a subject and an optical image formed by the optical system are picked up, and image data is output at a frame frequency Ffs that deviates from the NTSC frame frequency Ffr. An imaging device, a frame memory for storing image data, and the imaging
A control unit for writing image data output from the element into the frame memory and reading out from the frame memory at a frame frequency Ffr of the NTSC system; and a single oscillator for generating a clock signal compatible with the NTSC system, the control unit Controls reading from the frame memory at the frame frequency Ffr of the NTSC system based on the clock of the clock frequency corresponding to the NTSC system from the oscillator, and the imaging device has a clock frequency of the NTSC system compatible from the oscillator. An image pickup apparatus that operates by inputting a clock and changes the blanking period to output image data at a frame frequency Ffs deviating from the NTSC frame frequency Ffr, wherein the control unit includes the image pickup element. Progressively writes image data output from the Look, characterized in that the read from the frame memory in interlaced.

  According to a tenth aspect of the present invention, in the image pickup device according to any one of the first to ninth aspects, the control unit performs a predetermined image conversion process when reading image data from the frame memory. It is characterized by.

In the present invention, an image sensor is used at a frame frequency that is out of the NTSC frame frequency so that it is not easily affected by the blinking cycle when the lighting device such as an LED traffic light is turned on, and the NTSC display device can be used as it is. In the image pickup apparatus in which image data is read out from the frame memory, written in the frame memory, and read out from the frame memory at the frame frequency of the NTSC system, the clock oscillator mounted on the apparatus is only an oscillator for generating a clock signal compatible with the NTSC system The image sensor is operated with the clock of the clock frequency corresponding to the NTSC system from the oscillator, and the blanking period is changed so that the image data is output at a frame frequency that is out of the frame frequency of the NTSC system. Thus, a low-cost captured image device can be provided.

  In the image capturing apparatus of the present invention, the image sensor outputs image data at a frame frequency that deviates from the NTSC frame frequency only when the exposure time is a predetermined time or less, that is, when the subject is brighter than a certain level. As a result, generation of comb noise can be avoided except when shooting a very bright subject.

  Note that various functions and effects of the image pickup apparatus of the present invention other than those described above will become apparent from the description of the following embodiment.

1 is an overall configuration diagram of an embodiment of an image capturing apparatus of the present invention. The figure which shows the relationship between the output signal from the conventional image sensor, and the read signal from a frame memory. The figure which shows the relationship between the output signal from the image pick-up element of this invention, and the read signal from a frame memory. The detailed block diagram of the control part in FIG. The figure which shows the process image of the coordinate calculation means in FIG. Shows exposure timing of the illumination light and an image sensor that flashing period T _o (Part 1). Shows a reflected way of the traffic due to the exposure timing of the imaging element of the illumination light and the rotor ring shutter for flashing period T _o. Shows exposure timing of the illumination light and an image sensor that flashing period T _o (Part 2). The figure which shows the transition of the writing / reading in the case of the conventional double buffer, and the state of an output image (writing time = reading time). Write / read timing chart for double buffer (write time> read time). The figure which shows the transition of the writing / reading in the case of a double buffer, and the state of an output image (writing time> reading time). Write / read timing diagram for double buffer (write time <read time). The figure which shows the transition of writing and reading in the case of a double buffer, and the state of an output image (writing time <reading time). The figure which shows the specific example of the output image of comb noise generation | occurrence | production. The figure which shows the processing flow in CPU of a com noise countermeasure.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows an overall configuration diagram of an embodiment of an image pickup apparatus according to the present invention. The image capturing apparatus includes an optical system 101, an image sensor 102, a processing unit 103, an NTSC encoder 104, and a CPU 108 that controls the overall operation. The processing unit 103 includes a frame memory 105, a control unit 106, and an oscillator 107 therein. The NTSC encoder 104 is connected to an NTSC display device or an image recorder, but is omitted in FIG. Here, the oscillator 107 is a crystal element for generating a clock signal compatible with the NTSC system. Specifically, the clock frequency is 13.5 MHz. The clock (CK) of the oscillator 107 is supplied to the control unit 106 in the processing unit 10 and also to the image sensor 102.

The image sensor 102 receives an NTSC clock frequency (13.5 MHz) clock from the oscillator 107 as an input, and an offset frame frequency Ffs offset by a predetermined offset frequency from the NTSC frame frequency Ffr (about 30 FPS). The image data photographed through the optical system 101 is read progressively (for example, 27 FPS) and sent to the processing unit 103. Specifically, the image sensor 102 includes a register (sensor register), and the frame frequency of the image data is changed by changing the blanking period in the vertical direction or the horizontal direction according to the parameter value set by the CPU 108 in the register. Is set to Ffs (for example, 27 FPS). The image sensor 102 sends the vertical synchronization signal Sv having the frame frequency Ffs to the processing unit 103 together with the image data.

  Image data output from the image sensor 102 is sequentially written, for example progressively, into the frame memory 105 in the processing unit 103 at the timing of the frame frequency Ffs (for example, 27 FPS). The image data written in the frame memory 105 is sequentially read out, for example, in an interlaced manner at the timing of the NTSC frame frequency Ffr (about 30 FPS) under the control of the control unit 106. The control unit 106 designates a read address for the frame memory 105, controls the read timing of the frame memory 105 based on the vertical synchronization signal Sv from the image sensor 102, and receives a clock corresponding to the NTSC system from the oscillator 107. A clock having a frequency (13.5 MHz) is received, and interlaced reading from the frame memory 105 at the timing of the NTSC frame frequency Ffr is controlled. The configuration of the control unit 106 will be described later.

  The image data read from the frame memory 105 is subjected to digital / analog conversion processing by the NTSC encoder 104 to be converted into an NTSC video signal and sent to a display device or recorder (not shown).

  As shown in FIG. 1, image data is captured from the image sensor 102 at an offset frame frequency Ffs offset by a predetermined offset frequency with respect to the NTSC frame frequency Ffr, written into the frame memory 105, and NTSC is read from the frame memory. By reading out at the frame frequency Ffr of the system, it is possible to realize an image pickup apparatus that is not easily affected by the blinking cycle when a lighting device such as an LED traffic light is lit, and that can use an NTSC display device or the like as it is. Can do.

Further, the image sensor 102 operates by inputting a clock (CK) having a clock frequency corresponding to the NTSC system from an oscillator 107 mounted in the processing unit 103, and changes a blanking period in a vertical direction or a horizontal direction , Since the image data output from the image sensor 102 has a frame frequency Ffs that deviates from the NTSC frame frequency Ffr, a separate oscillator for the frame frequency Ffs is unnecessary, and low-cost imaging is performed. An apparatus can be realized. This will be described in detail below.

  FIG. 2 shows the relationship between the output signal from the conventional image sensor and the read signal from the frame memory. FIG. 2 (A) shows the output signal (progressive) from the image sensor, and FIG. Is a read signal (interlace) from the frame memory. That is, conventionally, in accordance with the NTSC system, the vertical blanking period of the output signal from the image sensor and the read signal from the frame memory is made the same. For this reason, in order to set the image data output from the image sensor to a frame frequency Ffs that is out of the NTSC frame frequency Ffr, an NTSC-compatible clock signal generation for reading image data from the frame memory is generated. It is necessary to prepare an oscillator (for example, 12.2 MHz) having a different clock frequency in addition to the oscillator (13.5 MHz) for reading image data from the image sensor. In the embodiment of the present invention, such an oscillator is not necessary.

FIG. 3 shows the relationship between the output signal from the image sensor and the readout signal from the frame memory in the embodiment of the present invention, and FIG. 3 (A) is the output signal (progressive) from the image sensor. 3 (B) is a read signal (interlace) from the frame memory. Here, the read signal from the frame memory is the same as in FIG. On the other hand, the vertical blanking period of the output signal from the image sensor is changed. That is, in the present embodiment , image data reading from the image sensor 102 and image data reading from the frame memory 105 are performed with the same clock (13.5 MHz) from the oscillator 107 for generating a clock signal compatible with the NTSC system. At this time, by changing the vertical blanking period for reading the image data from the image sensor 102, the output image data of the image sensor 102 has a frame frequency Ffs deviating from the NTSC frame frequency Ffr. I have to. As described above, the image sensor 102 includes a register (sensor register), and changes the blanking period in the vertical direction according to the parameter value set in the register from the CPU 108. The example of FIG. 3 is a case where Ffs <Ffr, but it is also possible to satisfy Ffs> Ffr by changing the parameter value of the sensor register. Also, by sequentially changing the parameter value of the sensor register for each frame by the CPU 108, as will be described later, when the adjacent frame periods are indeterminate and averaged, the image sensor is arranged at the NTSC frame frequency Ffr. It becomes possible to drive.
FIG. 3 shows an example of changing the blanking period in the vertical direction. However, the blanking period in the horizontal direction or both the vertical / horizontal directions is changed to change the output image data of the image sensor 102 to the frame frequency Ffr in the NTSC direction. Needless to say, the frame frequency Ffs may be out of the range.

  Next, the configuration operation of the control unit 106 will be described. FIG. 4 shows a detailed configuration diagram of the control unit 106. The control unit 106 includes a frame memory read start trigger creation unit 201, an output vertical / horizontal synchronization signal creation unit 202, and an address creation unit 203. Note that the address creation unit 203 may include coordinate calculation means 204 for distortion correction, viewpoint conversion, and the like.

  The frame memory read start trigger creation unit 201 receives the vertical synchronization signal Sv from the image sensor 102, and operates as follows immediately after the imaging apparatus is powered on. Note that the image sensor 102 is assumed to perform progressive reading. One Sv immediately after power-on of the imaging device is skipped, and the frame memory read start trigger (Tr) is asserted (validated) in the second Sv.

  By doing so, it is possible to prevent image data from being read out from the frame memory 105 when no image data exists from the frame memory 105.

  The output vertical horizontal synchronizing signal generation unit 202 uses the NTSC clock frequency (13.5 MHz) clock from the oscillator 107 to read the image from the frame memory 105 to the NTSC frame frequency Ffr (about 30 FPS). In this way, a vertical synchronization signal and a horizontal synchronization signal are created.

  The address creating unit 203 sequentially designates the read address of the frame memory 105 based on the clock (CK) from the oscillator 107, the vertical synchronizing signal and the horizontal synchronizing signal from the output vertical and horizontal synchronizing signal creating unit 202, and the frame The image data written in the memory 105 is read at the timing of the NTSC frame frequency Ffr (about 30 FPS).

  Here, the address creating unit 203 does not sequentially specify the read address of the frame memory 105, but instead uses a certain coordinate conversion rule (for example, distortion correction, viewpoint conversion such as a look-down image, center cutout conversion, side cutout conversion, three-plane conversion, etc. Coordinate calculation means 204 for designating according to mirror conversion or the like may be included.

  FIG. 5 shows a processing image of the coordinate calculation means 204. This is an example of outputting an image obtained by rotating and deforming image data (input image) from an image sensor by 90 °. Here, FIG. 5A shows a positional relationship between coordinates before and after conversion of a certain two pixels, and FIG. 5B shows an input image and an output image stored in the frame memory 105. The positional relationship of corresponding pixels is shown.

  When the image data is read from the frame memory 105, the coordinate calculation unit 204 reads pixel data of a pixel having coordinates (x1, y1) on the input image as a pixel having coordinates (X1, Y1) on the output image. As described above, the read address is calculated. Further, as the pixel at the coordinates (X2, Y2) of the output image, the read address is calculated so that the image data of the pixel at the coordinates (x2, y2) on the input image is read. By reading the image data from the corresponding address in the frame memory 105 based on this read address, an image obtained by rotating the input image by 90 ° can be obtained as the output image.

  Similarly, the image of the fisheye camera can be transformed to correct the distortion of the image caused by the fisheye image, or it can be transformed into an image seen from directly above by image transformation based on viewpoint transformation. An easy-to-see image can be displayed on the driver on a back monitor or the like that looks at the rear part. Since various configurations and processing algorithms of the coordinate calculation means 204 have been proposed in the past, detailed description thereof will be omitted.

  Next, by reading out image data from the image sensor 102 at a frame frequency (offset frame frequency) Ffs offset by a predetermined offset frequency with respect to the NTSC frame frequency Ffr, a lighting device such as an LED traffic light is turned on. The effect of the flicker cycle in the case will be specifically described.

  FIG. 6 and FIG. 7 show the case where the sampling period of the frame is performed at a constant period excluded from the NTSC system. In this case, the parameter value of the sensor register in the image sensor 102 is constant.

6, the graph on the 1st, represents the light amount change of the illumination light flashing period T _o (e.g., 60 Hz signal machine). The convex part of the graph is the brightest state (bright) and the valley part is dark (dark). Here, the turn-off period and _{t o.}

FIGS. 6A and 6B show the exposure timing of an image sensor for progressive readout with a global shutter. In the figure, t _i is the exposure time. (A) is a case where the image sensor is driven at 30 FPS, and (b) is a case where the image sensor is removed from 30 FPS. In the case of (a), since it is almost synchronized with the frame frequency Ffr (29.97 Hz) of the NTSC system, when the exposure enters the timing of the illumination light extinction, the illumination light of the extinction is continuously captured for several seconds. The On the other hand, in the case of (b), since it does not synchronize with the frame frequency Ffr of the NTSC system, even if the exposure enters the timing of extinction of the illumination light, it is easy to escape from it. It is avoided that the image is taken.

The same can be said for the rolling shutter. FIGS. 6C and 6D show the exposure timing of an image sensor for progressive reading with a rolling shutter. In the figure, t _i is the exposure time. Unlike the global shutter, the rolling shutter exposes each line independently. FIGS. 6C and 6D show that the exposure time gradually shifts to the long side, such as the first row, the second row, the third row,... (C) When the image sensor is driven at 30 FPS, (d) is removed from 30 FPS, and the line exposed in the shaded area in the figure shows the blinking illumination light in that line. The image is captured in a state. In the case of (c), since it is almost synchronized with the NTSC frame frequency Ffr (29.97 Hz), if the exposure enters the timing of the extinction of the illumination light, the extinction illumination light is continuously captured for several seconds. The On the other hand, in the case of (d), since it is not synchronized with the frame frequency Ffr of the NTSC system, even if the exposure enters the timing of extinction of the illumination light, it is easy to escape from it, and the illumination light of the extinction continues for several seconds. However, imaging is avoided.

7, an illumination light flashing period T _o, a rolling shutter, shows the reflected how traffic by the exposure timing of the image pickup device of the progressive read. On the left side of FIG. 7, since the exposure has entered the illumination light extinction timing, the color of the traffic light is not imaged. On the right side, since the exposure is out of the illumination light extinction timing, the color of the traffic light is ( For example, blue) is imaged.

  Next, FIG. 8 shows a case in which the sampling period of adjacent frames is indefinite and averaged so as to correspond to the NTSC system (30 frames per second). In this case, the parameter value of the sensor register in the image sensor 102 changes for each frame.

Graph top of FIG. 8, similarly to FIG. 6 represent the light amount change of the illumination light flashing period T _o (e.g., 60 Hz signal machine). The convex part of the graph is the brightest state (bright) and the valley part is dark (dark). Again, the turn-off period and _{t o.}

  8A to 8D, ti is an exposure time, Tf, Tf ′, Tf ″ is a frame sampling period. Here, Tf is an NTSC sampling period, and Tf ′, Tf ″ is NTSC. This is a period shifted from the sampling period of the system to the short time side or the long time side. However, in order to support the NTSC system, 30 frames can be output per second.

  FIGS. 8A and 8B show the exposure timing of an image sensor for progressive readout with a global shutter. (A) shows a case where the sampling period is fixed (Tf) to drive the image sensor, and (b) shows a case where the sampling period is indefinite (Tf ′, Tf ″, etc.) and the image sensor is driven. In the case of a), if the exposure enters the timing of extinction of the illumination light, the extinction illumination light is continuously imaged for several seconds, whereas in the case of (b), the exposure is performed at the timing of extinction of the illumination light. Even if it enters, it is easy to escape from the timing of extinction, so it is avoided that the extinction illumination light is continuously imaged for several seconds, and the sampling period of adjacent frames is made indefinite and averaged. By outputting 30 frames per second, it is possible to correspond to the NTSC system.

  The same can be said for the rolling shutter. FIGS. 8C and 8D show the exposure timing of the progressive reading image sensor with a rolling shutter. Unlike the global shutter, the rolling shutter exposes each line independently. Similarly to FIGS. 6C and 6D, in FIGS. 8C and 8D, the exposure time is gradually increased as shown in the first row, the second row, the third row,... It shows that it has moved to. The row exposed in the shaded area in the figure is imaged in a blinking state when the blinking illumination light is transferred to that row. (C) shows a case where the sampling period is fixed (Tf) and the image pickup element is driven, and (d) shows a case where the sampling period is set indefinite (Tf ′, Tf ″, etc.) and the image pickup element is driven. In the case of c), if the exposure enters the timing of extinction of the illumination light, the extinction illumination light is continuously imaged for several seconds, whereas in the case of (d), the exposure is performed at the timing of extinction of the illumination light. Even if it enters, it is easy to escape from the timing of extinction, so it is avoided that the extinction illumination light is continuously imaged for several seconds, and the sampling period of adjacent frames is made indefinite and averaged. By outputting 30 frames per second, it is possible to correspond to the NTSC system.

  The image data progressively read out from the image sensor 102 at the frame frequency Ffs deviating from the NTSC frame frequency Ffr is written in the frame memory 105 as it is, and is interleaved from the frame memory 105 at the NTSC frame frequency. Read to the race. Thereby, an NTSC display device and an image recorder can be used as they are. However, since the time for writing one frame of image data to the frame memory 105 is different from the time for reading one frame of image data from the frame memory 105, comb noise peculiar to the interlace method is generated. This will be specifically described below.

  First, a case will be described in which progressive writing is performed to the frame memory 105 at the NTSC frame frequency Ffr, and interlace reading is performed from the frame memory 105 at the same NTSC frame frequency Ffr. In this case, if the frame memory 105 is divided into at least two areas such as area A and area B (double buffer), each area is set as a write area and a read area, and their roles are switched sequentially, no comb noise occurs.

  FIG. 9 shows the transition of writing / reading in area A and area B and the state of the output image (receiver side) in this case. Here, writing is 30 frames / second, progressive, and reading is 60 fields / second, interlaced. Note that the same pattern in the figure represents image data of the same frame output from the image sensor 102. The same applies to the subsequent drawings.

  In the case of FIG. 9, the time for writing one frame of image data to one area of the frame memory 105 is the same as the time for reading one frame (odd field + even field) of image data from the other area. Once the start and read start timings are matched once (for example, (a)), the write end and the read end are permanently aligned. Therefore, the write area and the read area can be switched at the write end timing. For example, when the writing to the area A is completed, the writing to the area B is started and the reading is started from the area A (for example, (d)). As a result, a frame composed of an odd field and an even field of the same frame can be output at all times, so that no comb noise is generated (for example, (c), (e)).

  Next, a case will be described in which progressive writing is performed to the frame memory 105 at a frame frequency Ffs removed from the NTSC frame frequency Ffr, and interlaced reading is performed from the frame memory 105 at the NTSC frame frequency Ffr. In this case, comb noise is generated even if the frame memory 105 is a double buffer.

  First, a case where Ffs <Ffr, that is, a case where the writing time of one frame of image data is longer than the reading time of one frame of image data will be described.

  FIG. 10 shows a timing chart of writing / reading to / from the double buffer (area A, area B) in this case. Attention is now paid to the dotted line in FIG. 10, that is, the timing when reading from the region A is completed. Since the reading from the area A is completed, it is desired to start the reading from the area B and to start the writing to the area A by switching the writing / reading area. However, at this time, the writing is not finished in the region B. For this reason, reading (interlacing) is performed again from the area A, and when the writing of the area B is completed, the reading / writing area is switched, reading from the area B is started, and writing to the area A is started. It becomes. In this case, comb noise in which odd field rows and even field rows of different frames are mixed in the output image occurs. The same applies to the case where the relationship between the target read area and the write area is reversed.

  FIG. 11 shows the writing / reading of the double buffer (area A, area B) when Ffs <Ffr, that is, when the writing time of one frame of image data is longer than the reading time of one frame of image data. And the state of the output image (receiver side). FIG. 11 shows a case where reading is performed at 30 frames / second and interlaced reading corresponding to the NTSC system, while writing is performed at 27 frames / second and progressive writing.

  Now, progressive writing to the area A is started, and at the same time, interlaced reading is started from the area B (a), and attention is paid to the time when the reading from the area B is completed (c). At this time, in the area A, writing is in progress. For this reason, the interlace reading is started again from the area B, and when the writing of the area A is completed (d), the writing / reading area is switched, the area B is written, and the area A is read. At this time, from area A, interlaced reading after the state of (d) is continued (e). As a result, the output image when one frame of image data is read is as shown in (f). That is, comb noise is generated.

  Next, a case where Ffs> Ffr, that is, a case where the writing time of one frame of image data is shorter than the reading time of one frame of image data will be described.

  FIG. 12 shows a timing chart of writing / reading to / from the double buffer (area A, area B) in this case. Attention is now paid to the dotted line in FIG. 12, that is, the timing at which writing to the region B is completed. Since the writing to the area B is completed, the writing / reading area is switched, the writing to the area A is started, and the reading from the area B is started. However, at this point, the area A is being read out. When the write / read area is switched in this state, comb noise in which odd field rows and even field rows of different frames are mixed is generated in the output image as in the case of Ffs <Ffr. The same applies when the relationship between the noted write area and read area is reversed.

  FIG. 13 shows writing / reading of the double buffer (area A, area B) when Ffs> Ffr, that is, when the writing time of one frame of image data is shorter than the reading time of one frame of image data. And the state of the output image (receiver side). FIG. 13 shows a case where reading is performed at 30 frames / second and interlaced reading corresponding to the NTSC system, while writing is performed at 33 frames / second and progressive writing.

  Now, the progressive writing to the area A is started, and at the same time, the interlaced reading from the area B is started (a), and attention is paid to the time when the writing to the area A is completed (b). At this time, in the region B, reading is still in progress. Specifically, the reading of the odd field has been completed, but the even field is still being read. However, since the writing to the area A is completed, the writing / reading area is switched, the area B is written, and the area A is read. At this time, interlaced reading from the area A after the state of (b) is continued. As a result, the output image when one frame of image data is read is as shown in (c). That is, comb noise is generated.

  FIG. 14 shows an example of the output image. FIG. 14A shows an image sensor driven at an NTSC frame frequency Ffr, and progressively writes image data output from the image sensor to a frame memory. The frame memory is also interlaced at an NTSC frame frequency Ffr. No noise is generated. On the other hand, FIG. 14B shows that the image sensor is driven at a frame frequency Ffs (here, Ffs <Ffr) deviating from the NTSC frame frequency, and output image data from the image sensor is progressively written to the frame memory. It can be seen that the image is an interlaced read from the frame memory at the NTSC frame frequency Ffr, and that comb noise is generated in a part of the image.

  Hereinafter, countermeasures against such comb noise will be described. For example, as can be seen from FIG. 7, when the exposure time ti is sufficiently long, the traffic light is completely transmitted even if the frame frequency for reading image data from the image sensor is the frame frequency Ffr (29.97 Hz) of the NTSC system. The camera will not shoot with the light turned off. As the exposure time ti becomes shorter, the light state of the traffic light becomes darker and finally the timing at which it becomes completely invisible occurs. Therefore, only when the exposure time is below a certain level, that is, only when the subject is brighter than a certain level, comb noise is minimized by setting the frame frequency of the image sensor to a frame frequency Ffs that is out of the NTSC frame frequency Ffr. be able to. That is, generation of comb noise can be avoided except when shooting a very bright subject.

  An image sensor generally has an electronic shutter function. The electronic shutter time changes according to the amount of received light, and is long when the subject is dark and shortens as it is brighter. This electronic shutter time, that is, the exposure time ti is represented.

  In the configuration of FIG. 1, the image sensor 102 is provided with a second register in addition to the previous sensor register (hereinafter referred to as the first register), and the electronic register time (hereinafter referred to as exposure time) is provided in the second register. To set. The CPU 108 reads the exposure time of the second register in the image sensor 102 and compares it with a predetermined threshold value. Here, when the exposure time ≧ the threshold, the CPU 108 sets a predetermined parameter value (in a first register in the image sensor 102 so that the frame frequency of the output image data from the image sensor 102 becomes the frame frequency Ffr of the NTSC system. In the following, the first parameter value) is set, and the frame frequency obtained by offsetting the frame frequency of the output image data from the image sensor 102 by the predetermined offset frequency from the NTSC frame frequency Ffr only when the exposure time <threshold value. A predetermined parameter value (hereinafter referred to as a second parameter) is set in the first register in the image sensor 102 such that (offset frame frequency) Ffs.

The image sensor 102 changes the blanking period in the vertical direction or the horizontal direction according to the parameter value set in the first register (sensor register) in the image sensor 102 as described above. Yes. When the first parameter value is set in the first register from the CPU 108, the image sensor 102 determines a blanking period in the vertical direction so that image data is output at the frame frequency Ffr of the NTSC system. On the other hand, when the second parameter value is set from the CPU108 into the first register, the vertical direction or so that the image data is output at the frame frequency Ffs which is offset from the frame frequency Ffr of the NTSC system by a predetermined offset frequency Change the horizontal blanking period.

  FIG. 15 shows a processing flowchart of the CPU 108. The CPU 108 reads the exposure time of the second register in the image sensor 102 (step 1) and compares it with a predetermined threshold value (step 2). The threshold is 1/180 seconds, for example, but can be adjusted from the outside depending on the type of traffic light or image sensor. Here, if exposure time ≧ threshold, a predetermined parameter value (first register) is stored in the sensor register (first register) in the image sensor 102 so that the output image data from the image sensor 101 has an NTSC frame frequency. Parameter value) is set (step 3). Further, if the exposure time is smaller than the threshold value, that is, if the subject is brighter than a certain level, a predetermined value is stored in the sensor register so that the output image data from the image sensor 101 has a frame frequency Ffs deviating from a predetermined value by the NTSC frame frequency Ffr. Parameter value (second parameter value) is set (step 4).

  Here, the CPU 108 reads the exposure time from the image sensor 102 and compares it with a predetermined threshold value. However, the CPU 108 takes in the output image data from the image sensor 102 and determines the brightness (brightness of the subject) based on the sum of the brightness of the entire screen. ) May be calculated, the brightness may be compared with a predetermined threshold value, and a value may be set in a sensor register in the image sensor 102. Furthermore, a separate illuminance sensor may be provided, and the CPU 108 may capture the output of the illuminance sensor, compare it with a predetermined threshold value, and similarly set a value in a sensor register in the image sensor 102. As a result, only when the subject is brighter than a certain level, the frame frequency of the image sensor 102 can be shifted from the frame frequency of the NTSC system to avoid the occurrence of comb noise.

DESCRIPTION OF SYMBOLS 101 Optical system 102 Image pick-up element 103 Processing part 104 NTSC encoder 105 Frame memory 106 Control part 107 Oscillator 108 CPU
201 frame memory read start trigger creation unit 202 output vertical / horizontal synchronization signal creation unit 203 address creation unit 204 coordinate calculation means

JP 2009-181339 A

Claims (8)

An optical system that forms an optical image of the subject;
An image pickup device that picks up an optical image formed by the optical system and outputs image data at a frame frequency Ffs deviating from the NTSC frame frequency Ffr;
A frame memory for storing image data;
A controller that writes image data output from the image sensor into the frame memory and reads out from the frame memory at an NTSC frame frequency Ffr ;
It has a single oscillator for clock signal generation compatible with the NTSC system ,
The control unit controls reading from the frame memory at the frame frequency Ffr of the NTSC system based on the clock of the clock frequency corresponding to the NTSC system from the oscillator,
The image pickup device operates by inputting a clock having a clock frequency corresponding to the NTSC system from the oscillator, and outputs image data at a frame frequency Ffs deviating from the frame frequency Ffr of the NTSC system by changing the blanking period. An imaging device,
The image pickup device outputs image data at a variable frame frequency Ffs, in which adjacent frame periods are indefinite, and when averaged, an NTSC frame period is obtained. The image capturing apparatus according to claim 1, wherein the image sensor includes a register, and the blanking period is changed according to a parameter value set in the register. The image capturing apparatus according to claim 2, further comprising a CPU, wherein the CPU sets a parameter value of a register in the image sensor. An optical system that forms an optical image of the subject;
An image pickup device that picks up an optical image formed by the optical system and outputs image data at a frame frequency Ffs deviating from the NTSC frame frequency Ffr;
A frame memory for storing image data;
A controller that writes image data output from the image sensor into the frame memory and reads out from the frame memory at an NTSC frame frequency Ffr;
It has a single oscillator for clock signal generation compatible with the NTSC system,
The control unit controls reading from the frame memory at the frame frequency Ffr of the NTSC system based on the clock of the clock frequency corresponding to the NTSC system from the oscillator,
The image pickup device operates by inputting a clock having a clock frequency corresponding to the NTSC system from the oscillator, and outputs image data at a frame frequency Ffs deviating from the frame frequency Ffr of the NTSC system by changing the blanking period. An imaging device,
The image sensor outputs image data at the NTSC frame frequency Ffr when the exposure time is equal to or greater than a predetermined value, and only when the exposure time is equal to or less than the predetermined value, the frame frequency Ffs deviates from the NTSC frame frequency Ffr. And outputting image data. The image pickup device has a register, and outputs image data at an NTSC frame frequency Ffr according to a parameter value set in the register, or outputs image data at a frame frequency Ffs deviating from the NTSC frame frequency Ffr. The image capturing apparatus according to claim 4, wherein the blanking period is changed so as to be output. 6. The image capturing apparatus according to claim 5, further comprising a CPU, wherein the CPU takes an exposure time from the image sensor and changes a parameter value of a register of the image sensor in accordance with the exposure time. An optical system that forms an optical image of the subject;
An image pickup device that picks up an optical image formed by the optical system and outputs image data at a frame frequency Ffs deviating from the NTSC frame frequency Ffr;
A frame memory for storing image data;
A controller that writes image data output from the image sensor into the frame memory and reads out from the frame memory at an NTSC frame frequency Ffr;
It has a single oscillator for clock signal generation compatible with the NTSC system,
The control unit controls reading from the frame memory at the frame frequency Ffr of the NTSC system based on the clock of the clock frequency corresponding to the NTSC system from the oscillator,
The image pickup device operates by inputting a clock having a clock frequency corresponding to the NTSC system from the oscillator, and outputs image data at a frame frequency Ffs deviating from the frame frequency Ffr of the NTSC system by changing the blanking period. An imaging device,
The image pickup apparatus, wherein the control unit writes the image data output from the image pickup device progressively to the frame memory and reads the image data from the frame memory in an interlaced manner. The image capturing apparatus according to claim 1, wherein the control unit performs a predetermined image conversion process when reading image data from the frame memory.

Images