WO2015037474A1 - Electronic device and image sensor - Google Patents

Electronic device and image sensor Download PDF

Info

Publication number
WO2015037474A1
WO2015037474A1 PCT/JP2014/073055 JP2014073055W WO2015037474A1 WO 2015037474 A1 WO2015037474 A1 WO 2015037474A1 JP 2014073055 W JP2014073055 W JP 2014073055W WO 2015037474 A1 WO2015037474 A1 WO 2015037474A1
Authority
WO
WIPO (PCT)
Prior art keywords
illuminance
pixels
image sensor
pixel
output
Prior art date
Application number
PCT/JP2014/073055
Other languages
French (fr)
Japanese (ja)
Inventor
英樹 庄山
勇一 有働
賢政 坂本
裕介 池田
央一 熊谷
Original Assignee
ソニー株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニー株式会社 filed Critical ソニー株式会社
Publication of WO2015037474A1 publication Critical patent/WO2015037474A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00—Photometry, e.g. photographic exposure meter
    • G01J1/42—Photometry, e.g. photographic exposure meter using electric radiation detectors
    • G01J1/4204—Photometry, e.g. photographic exposure meter using electric radiation detectors with determination of ambient light
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/71—Circuitry for evaluating the brightness variation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H04N25/44—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array
    • H04N25/445—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array by skipping some contiguous pixels within the read portion of the array
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00—Photometry, e.g. photographic exposure meter
    • G01J1/42—Photometry, e.g. photographic exposure meter using electric radiation detectors
    • G01J1/44—Electric circuits
    • G01J2001/4446—Type of detector
    • G01J2001/448—Array [CCD]
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B13/00—Viewfinders; Focusing aids for cameras; Means for focusing for cameras; Autofocus systems for cameras
    • G03B13/32—Means for focusing
    • G03B13/34—Power focusing
    • G03B13/36—Autofocus systems

Definitions

  • the present disclosure relates to an electronic apparatus and an image sensor, and more particularly to an electronic apparatus and an image sensor suitable for use in measuring illuminance based on an output of an image sensor mounted to realize an imaging function.
  • home appliances such as television receivers, air conditioners, refrigerators, and the like have been incorporated with illuminance meters, and the home appliances are operated in accordance with the measured illuminance. Specifically, for example, the screen brightness is adjusted according to the illuminance, the touch sensor for malfunction prevention is disabled, or a predetermined function is stopped to reduce power consumption. Yes.
  • the present disclosure has been made in view of such a situation, and is intended to realize power saving and improved accuracy of measurement values when measuring illuminance using the output of an image sensor.
  • An electronic device includes an image sensor unit including a plurality of pixels, and a control for periodically driving the pixels to be thinned out while driving the plurality of pixels forming the image sensor unit. And an illuminance calculation unit that calculates illuminance using the output of the image sensor unit when the plurality of pixels are thinned out.
  • the illuminance calculation unit can calculate illuminance every time the pixels are thinned out, and can further output a moving average of the calculated illuminance.
  • the electronic device may further include a drive unit that operates according to control based on the illuminance output from the illuminance calculation unit.
  • the control unit can divide the large number of pixels forming the image sensor unit into regions of a predetermined size and periodically change the region to be driven for each frame or each activation.
  • the image sensor unit includes a short accumulation pixel and a long accumulation pixel, and the illuminance calculation unit can calculate the illuminance by using the output of the short accumulation pixel and the output of the long accumulation pixel that are simultaneously driven.
  • the image sensor unit includes a pixel that also serves as an image plane phase difference detection application, and the illuminance calculation unit can compute illuminance without using the output of the driven pixel that also serves as an image plane phase difference detection application. .
  • an image sensor in which a large number of pixels forming an image sensor unit are driven to be thinned out, and the pixels to be thinned out are periodically changed so that the plurality of pixels are driven to be thinned out. Illuminance is calculated using the output of the unit.
  • An image sensor is an image sensor including a large number of pixels.
  • the normal number of pixels is driven simultaneously during normal imaging, and the large number of pixels are thinned out during illuminance measurement.
  • the pixels to be driven are periodically changed.
  • the large number of pixels are simultaneously driven, and during illuminance measurement, the large number of pixels are driven to be thinned and the pixels to be thinned are periodically changed.
  • the first aspect of the present disclosure it is possible to perform highly accurate illuminance measurement with low power consumption using the image sensor output.
  • pixel values that can be used for illuminance measurement can be output.
  • FIG. 1 is an embodiment of the present disclosure, and shows a configuration example of an electronic device that performs illuminance measurement using an image sensor output and controls an operation state according to the measured illuminance.
  • the electronic device 10 includes a control unit 11, an input unit 12, a setting unit 19, and a drive unit 22.
  • the control unit 11 controls the operation of the input unit 12 and controls the drive unit 22 based on data from the input unit 12. More specifically, the operating state of the drive unit 22 is controlled based on the illuminance input from the input unit 12.
  • the input unit 12 includes an image sensor 13 that generates an image signal and an image processing unit 14 that processes the image signal.
  • the image sensor 13 is composed of a large number of photoelectric conversion elements (hereinafter referred to as pixels). According to control from the control unit 11, all pixels are simultaneously driven during normal imaging, and a pixel signal as an output thereof is output to the image processing unit 14. Output to. Further, the image sensor 13 thins out some of all the pixels (details will be described later) during illuminance measurement, and outputs a pixel signal as an output to the image processing unit 14.
  • the image sensor 13 is driven in accordance with the control of the control unit 11, but the image sensor 13 itself is provided with a control unit, and the image sensor 13 controls the operations during normal imaging and illuminance measurement. You may do it.
  • the image processing unit 14 calculates the illuminance based on the image signal input from the image sensor 13 as the illuminance calculation unit and outputs the illuminance to the control unit 11 when measuring the illuminance.
  • the illuminance calculation in the image processing unit 14 the illuminance for each frame is calculated based on the output of the image sensor 13 in which different regions are thinned out for each frame, and the illuminance is moved and averaged between the frames for each frame. It is used for control.
  • the input unit 12 is a timer 12 for measuring time, a microphone 16 for inputting sound, a user interface (I / F) 17 such as a mouse, a keyboard, and a touch panel for inputting a user operation, and the position and displacement of the electronic device 10.
  • the gyro sensor 18 to detect may be included.
  • the setting unit 19 includes a memory 20 that holds a control program, various coefficients, and the like, and a communication unit 21 that communicates data by connecting to a network.
  • the drive unit 22 can include a power supply unit 23, a storage unit 24, an output interface (I / F) 25, a motor 26, a display unit 27, a speaker 28, an illumination unit 29, and the like.
  • FIGS. 2 and 3 are diagrams for explaining an example of the first thinning driving of the image sensor 13.
  • the shaded area in the figure shows the pixel area to be driven.
  • FIG. 2 shows the drive area of the first frame when 8 frames are taken as one cycle
  • FIG. 3 shows 8 frames as one cycle.
  • the drive area of the second frame in this case is shown.
  • the smallest rectangle in FIG. 2A represents one pixel
  • the smallest rectangle in FIGS. 2B and 3 represents 2 ⁇ 2 pixels.
  • all pixels of the image sensor 13 are divided into strip-like regions having a width of 2 pixels and a length of 16 pixels that are long in the vertical direction, and the regions are driven only once every 8 frames. .
  • the order of driving each region is as indicated by the numbers on the upper side of the image sensor 13 in the figure, and is not simply moved to the adjacent region in one direction, but is moved left and right at intervals.
  • the color arrangement of the pixels constituting the image sensor 13 is a Bayer arrangement, and the short accumulation pixels R, Gr, Gb, B having different accumulation times until the pixels of each color are saturated. Or long accumulation pixels r, gr, gb, b are arranged. In each strip-shaped region, 2 ⁇ 4 pixel short accumulation pixels and 2 ⁇ 4 pixel long accumulation pixels sharing FD (Floating Diffusion) are alternately arranged.
  • FD Floating Diffusion
  • the center of gravity of the short accumulation pixels and the long accumulation pixels to which the pixel values are added is premised on the same imaging region. Can be close to the center of the area.
  • the arrangement of the short accumulation pixel and the long accumulation pixel is sufficient if their centers of gravity are aligned.
  • the short accumulation pixel is sandwiched between the long accumulation pixels or the short accumulation pixel is sandwiched between the long accumulation pixels. Also good.
  • FIG. 4 shows the relationship between the accumulation time and the pixel value of each of the short accumulation pixel and the long accumulation pixel. As shown in the figure, by combining the outputs of the short accumulation pixel and the long accumulation pixel having different characteristics, a pixel having a wide dynamic range and high accuracy can be obtained.
  • FIG. 5 shows an example in which the output of the short accumulation pixel is converted into the output of the long accumulation pixel when the output of each of the short accumulation pixel and the long accumulation pixel is synthesized. Specifically, the output is switched from the saturation point of the long accumulation pixel to a value obtained by multiplying the output of the short accumulation pixel by the accumulation time ratio.
  • This threshold value may be set for each color according to pixel characteristics, or may be set from the outside so that fine adjustment is possible.
  • FIG. 6 shows the arrangement of short accumulation pixels and long accumulation pixels in each strip-shaped region. As shown in the figure, before synthesizing the outputs of the short accumulation pixels and the long accumulation pixels, in each of the strip-shaped areas, 8 pixels of the short accumulation pixels sharing the FD and 8 pixels of the long accumulation pixels In each case, the pixel value is added for each color.
  • FIG. 7 shows a conversion formula for converting pixel values R, G, and B of the Bayer array into luminance Y and an example thereof.
  • the coefficients of the conversion formula are recorded in advance in the memory 20 at the time of manufacture. However, these coefficients may be set from the outside via the communication unit 21.
  • luminance Y is converted into illuminance Lux.
  • FIG. 8 shows a conversion formula for converting luminance Y into illuminance Lux and an example thereof.
  • the coefficients of the conversion formula are recorded in advance in the memory 20 at the time of manufacture. However, these coefficients may be set from the outside via the communication unit 21.
  • FIG. 9 shows an example in which the vertical axis represents the illuminance and the horizontal axis represents the frame, and the illuminance calculated for each frame is moving averaged between frames using an IIR filter with a feedback rate of 50%.
  • the influence can be suppressed by taking a moving average between frames.
  • the sixth and eighth frames are used.
  • the calculated illuminance is significantly higher than the illuminance of other frames.
  • the influence of the high-intensity light source can be suppressed.
  • TAPs in the moving average may be changed according to the demand for moving average stability and responsiveness.
  • FIR filter may be used instead of the IIR filter.
  • moving the pixel drive area for each frame can equalize the number of accesses for each pixel, so that a secondary effect of making the deterioration of the pixels due to the drive time uniform can also be expected.
  • the movement of the region for driving the pixels may be changed not every frame but every time the image sensor 13 is activated.
  • FIGS. 10 and 11 are diagrams for explaining an example of the second thinning driving of the image sensor 13.
  • the example of the second thinning drive is obtained by further adding a thinning in the vertical direction to the above-described example of the first thinning driving in which only the thinning in the horizontal direction is performed.
  • FIG. 10 shows the driving area of the second frame when 16 frames are taken as one period
  • FIG. 11 shows 16 frames as one period.
  • the drive area of the third frame in this case is shown.
  • the smallest rectangle in FIG. 10A represents one pixel
  • the smallest rectangle in FIGS. 10B and 11 represents 2 ⁇ 2 pixels.
  • all the pixels of the image sensor 13 are divided into strip-like regions that are 2 pixels wide and 16 pixels long in the vertical direction, and the region is driven only once every 16 frames. .
  • the order of driving each region is as indicated by the numbers on the upper side of the image sensor 13 in the figure, and is not simply moved to the adjacent region in one direction, but is moved left and right at intervals.
  • the power consumption can be further reduced to half compared to the case of performing the first thinning drive.
  • an OB (Optical Black) region (not shown) of the image sensor 13 may be insufficient. In that case, the same pixel is driven in the vertical direction, or a frame that is not driven is provided.
  • the black level of the previous frame is used or the black level is not changed.
  • the arrangement of the colors of the pixels constituting the image sensor 13 is the same as the first thinning drive example described with reference to FIG. Description is omitted.
  • FIG. 12 shows the entire area of the image sensor 13 for explaining an example of the third thinning driving of the image sensor 13, and the hatched area in the drawing shows the area of the pixel to be driven. That is, in the third thinning drive example, the entire area of the image sensor 13 is divided into a plurality of rectangular areas, and the first or second thinning driving described above is performed only in a predetermined rectangular area of the rectangular areas. To do.
  • each bit of 16-bit ALS_GRID_EN turns on driving of each rectangular area. Can be used for configuration.
  • the specific rectangular area for driving the pixel may be designated from the outside, or may be changed as needed for each frame.
  • the image obtained when the first to third thinning driving examples are performed has an illuminance calculation although the number of pixels is small and the resolution is inferior to that of an image in which all normal pixels are driven. There is no problem in identifying the position of the light source and determining the moving object.
  • “Variation in First to Third Thinning Drive of Image Sensor 13” 13A shows an example of the first to third thinning driving of the image sensor 13, and FIG. 13B shows a modified example of the first to third thinning driving.
  • An example in which pixels are driven independently is shown.
  • the number of pixels to be driven can be halved while maintaining the spatial frequency of the obtained image, and power consumption can be reduced by that amount.
  • a false color may be generated when the pixels in the R column and the pixels in the B column are spatially separated, but this is not a problem for use in illuminance measurement.
  • FIG. 14 shows an arrangement example when an image plane phase difference detection pixel is provided in an image sensor 13 mounted on a digital camera or the like having an image plane phase difference AF (Auto Focus) function, and an example of illuminance calculation corresponding to that case. Is shown.
  • image plane phase difference AF Auto Focus
  • the image plane phase difference detection pixel When the image plane phase difference detection pixel is provided in the image sensor 13, the image plane phase difference detection pixel is arranged at the position of Gr, gr in the Bayer array so that the output of Gr, gr is not used for the illuminance calculation. Similarly, when an infrared detection pixel is provided in the image sensor 13, the output of the infrared detection pixel should not be used for illuminance calculation.
  • the control unit 11 can control the operation of the drive unit 22 based on the illuminance input from the image processing unit 14.
  • the illuminance measurement mode is switched from the illuminance measurement mode to the normal imaging mode, such as zooming the area where the illuminance suddenly increases and becomes brighter, or imaging. You can start recording a video shot in the mode or sound an alarm.
  • the electronic device 10 when the electronic device 10 is applied to an in-vehicle camera, it is possible to control a motor that moves the irradiation axis of the light so as to shift the direction of the light of the own vehicle from the position of the oncoming vehicle. it can.
  • the time is also referred to according to the measured illuminance, and the power consumption is reduced when the time zone is night and there is no moving object in the dark. It can be controlled to operate in the power saving mode to be suppressed.
  • the electronic device 10 when the electronic device 10 is applied to a mobile phone, a smart phone, etc., it is determined that the mobile phone has been put in a pocket or a bag when the illumination is low even though the time zone is daytime. To prevent malfunctions, turn off the touch panel or increase the ringtone volume. Conversely, when the time zone is night and the illuminance is low, it is determined that the user is sleeping and the volume of the speaker is lowered. In this way, the control operation can be changed according to the surrounding environment.
  • the electronic device 10 of the present embodiment it is possible to measure illuminance without providing a dedicated circuit configuration (illuminance meter or the like) for illuminance measurement. Further, since the pixels of the image sensor 13 are driven by thinning out during illuminance measurement, power consumption can be suppressed as compared with the case where all the pixels of the image sensor 13 are driven. Further, the illuminance can be calculated more quickly than when all the pixels are read out.
  • the pixel area to be driven by thinning is changed for each frame or the like, and the illuminance for each frame is moving averaged between the frames, variation in the finally obtained illuminance can be suppressed and accuracy can be maintained.
  • the pixel area to be thinned and driven for each frame or the like it is possible to expect a secondary effect to make the deterioration of the pixels uniform.
  • Pixel values obtained for illuminance measurement can be output as low-resolution images.
  • this indication can also take the following structures.
  • An image sensor unit comprising a large number of pixels; A controller that thinly drives the pixels forming the image sensor unit and periodically changes the pixels to be thinned; and An illuminance calculation unit that calculates illuminance using an output of the image sensor unit when the plurality of pixels are thinned out.
  • the illuminance calculation unit calculates illuminance every time the large number of pixels are driven to be thinned, and further outputs a value obtained by moving average the calculated illuminance.
  • the control unit divides the large number of pixels forming the image sensor unit into regions of a predetermined size, and periodically changes the region to be driven for each frame or for each activation.
  • An electronic device according to any one of the above.
  • the image sensor unit includes a pixel that also serves as an image plane phase difference detection application, The electronic device according to any one of (1) to (5), wherein the illuminance calculation unit calculates illuminance without using an output of a driven pixel that also serves as an image plane phase difference detection application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Sustainable Development (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Studio Devices (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

The present disclosure pertains to an electronic device and image sensor that make it possible to achieve reduced power consumption and accurate measured values when illuminance is measured using the output of an image sensor. An electronic device that is the first aspect of this disclosure is provided with an image sensor unit comprising a plurality of pixels, a control unit for driving a portion of the plurality of pixels comprising the image sensor unit and periodically varying the driven portion of the pixels, and an illuminance calculation unit for calculating an illuminance using the output of the image sensor unit during the driving of the portions of the plurality of pixels. The present disclosure is applicable to a device for controlling an operation on the basis of illuminance.

Description

電子装置、およびイメージセンサElectronic device and image sensor
　本開示は、電子装置およびイメージセンサに関し、特に、撮像機能を実現するために搭載されているイメージセンサの出力に基づいて照度を計測する場合に用いて好適な電子装置およびイメージセンサに関する。 The present disclosure relates to an electronic apparatus and an image sensor, and more particularly to an electronic apparatus and an image sensor suitable for use in measuring illuminance based on an output of an image sensor mounted to realize an imaging function.
　近年、例えばテレビジョン受像機、エアコンディショナ、冷蔵庫などの家庭電化製品には照度計が組み込まれており、該家庭電化製品は計測された照度に応じて動作するようになされている。具体的には、例えば照度に応じて画面の明るさを調整したり、誤動作防止のためのタッチセンサを無効化したり、消費電力を抑えるために所定の機能を停止したりする制御が行なわれている。 In recent years, home appliances such as television receivers, air conditioners, refrigerators, and the like have been incorporated with illuminance meters, and the home appliances are operated in accordance with the measured illuminance. Specifically, for example, the screen brightness is adjusted according to the illuminance, the touch sensor for malfunction prevention is disabled, or a predetermined function is stopped to reduce power consumption. Yes.
　また、撮像機能を有するパーソナルコンピュータやスマートフォンなどの電子装置では、撮像機能を実現するためのイメージセンサの出力を用いて照度を計測するものがある（例えば、特許文献１参照）。 Also, some electronic devices such as personal computers and smartphones having an imaging function measure illuminance using the output of an image sensor for realizing the imaging function (see, for example, Patent Document 1).
特開２００８－３０６２５４号公報JP 2008-306254 A
　該電子装置においてイメージセンサ出力を照度計測に用いるには、イメージセンサの全画素出力を用いる（全画素を駆動する）方法と、イメージセンサの一部の画素出力を用いる（画素を間引いて駆動する）方法がある。 In order to use the image sensor output for illuminance measurement in the electronic device, a method using all the pixel outputs of the image sensor (driving all pixels) and a part of the pixel outputs of the image sensor (driving the pixels) There is a way.
　全画素出力を用いる方法の場合、画角全体の輝度が平均化されるので実際の照度に近い値を計測できる。しかしながら、全画素を駆動するために消費電力の観点で問題がある。 In the case of the method using all pixel outputs, the brightness of the entire angle of view is averaged, so a value close to the actual illuminance can be measured. However, since all the pixels are driven, there is a problem in terms of power consumption.
　一方、イメージセンサの一部の画素出力を用いる方法の場合、画角面内に高輝度被写体、点滅被写体、移動被写体などが存在したり、イメージセンサ自身が揺れたりしたことに起因して、実際の照度から離れた値が計測されてしまうことがある。 On the other hand, in the case of the method using the pixel output of a part of the image sensor, it is actually caused by the presence of a high-brightness subject, blinking subject, moving subject, etc. in the angle of view or the image sensor itself shaking. A value far from the illuminance may be measured.
　本開示はこのような状況に鑑みてなされたものであり、イメージセンサの出力を用いて照度を計測するに際し、省電力化と計測値の精度向上を実現できるようにするものである。 The present disclosure has been made in view of such a situation, and is intended to realize power saving and improved accuracy of measurement values when measuring illuminance using the output of an image sensor.
　本開示の第１の側面である電子装置は、多数の画素から成るイメージセンサ部と、前記イメージセンサ部を成す前記多数の画素を間引き駆動させるとともに、間引き駆動させる画素を周期的に変更する制御部と、前記多数の画素が間引き駆動されたときの前記イメージセンサ部の出力を用いて照度を演算する照度演算部とを備える。 An electronic device according to a first aspect of the present disclosure includes an image sensor unit including a plurality of pixels, and a control for periodically driving the pixels to be thinned out while driving the plurality of pixels forming the image sensor unit. And an illuminance calculation unit that calculates illuminance using the output of the image sensor unit when the plurality of pixels are thinned out.
　前記照度演算部は、前記多数の画素が間引き駆動される毎に照度を演算し、さらに演算した前記照度を移動平均した値を出力することができる。 The illuminance calculation unit can calculate illuminance every time the pixels are thinned out, and can further output a moving average of the calculated illuminance.
　本開示の第１の側面である電子装置は、前記照度演算部から出力された前記照度に基づく制御に従って動作する駆動部をさらに備えることができる。 The electronic device according to the first aspect of the present disclosure may further include a drive unit that operates according to control based on the illuminance output from the illuminance calculation unit.
　前記制御部は、前記イメージセンサ部を成す前記多数の画素を所定のサイズの領域に区切り、駆動させる前記領域をフレーム毎または起動毎に周期的に変更することができる。 The control unit can divide the large number of pixels forming the image sensor unit into regions of a predetermined size and periodically change the region to be driven for each frame or each activation.
　前記イメージセンサ部は、短蓄画素と長蓄画素から成り、前記照度演算部は、同時に駆動された前記短蓄画素の出力と前記長蓄画素の出力を用いて照度を演算することができる。 The image sensor unit includes a short accumulation pixel and a long accumulation pixel, and the illuminance calculation unit can calculate the illuminance by using the output of the short accumulation pixel and the output of the long accumulation pixel that are simultaneously driven.
　前記イメージセンサ部は、像面位相差検出用途を兼ねる画素を含み、前記照度演算部は、駆動された前記像面位相差検出用途を兼ねる画素の出力を用いることなく照度を演算することができる。 The image sensor unit includes a pixel that also serves as an image plane phase difference detection application, and the illuminance calculation unit can compute illuminance without using the output of the driven pixel that also serves as an image plane phase difference detection application. .
　本開示の第１の側面においては、イメージセンサ部を成す多数の画素が間引き駆動されるとともに、間引き駆動される画素が周期的に変更され、前記多数の画素が間引き駆動されたときのイメージセンサ部の出力を用いて照度が演算される。 In the first aspect of the present disclosure, an image sensor in which a large number of pixels forming an image sensor unit are driven to be thinned out, and the pixels to be thinned out are periodically changed so that the plurality of pixels are driven to be thinned out. Illuminance is calculated using the output of the unit.
　本開示の第２の側面であるイメージセンサは、多数の画素から成るイメージセンサにおいて、通常撮像時には、前記多数の画素を同時に駆動し、照度計測時には、前記多数の画素を間引き駆動するとともに、間引き駆動する画素を周期的に変更する。 An image sensor according to a second aspect of the present disclosure is an image sensor including a large number of pixels. The normal number of pixels is driven simultaneously during normal imaging, and the large number of pixels are thinned out during illuminance measurement. The pixels to be driven are periodically changed.
　本開示の第２の側面においては、通常撮像時には、多数の画素が同時に駆動され、照度計測時には、前記多数の画素が間引き駆動されるとともに、間引き駆動される画素が周期的に変更される。 In the second aspect of the present disclosure, during normal imaging, a large number of pixels are simultaneously driven, and during illuminance measurement, the large number of pixels are driven to be thinned and the pixels to be thinned are periodically changed.
　本開示の第１の側面によれば、イメージセンサ出力を用い、少ない消費電力で精度の高い照度計測を行なうことができる。 According to the first aspect of the present disclosure, it is possible to perform highly accurate illuminance measurement with low power consumption using the image sensor output.
　本開示の第２の側面によれば、照度計測に利用可能な画素値を出力することができる。 According to the second aspect of the present disclosure, pixel values that can be used for illuminance measurement can be output.
本開示を適用した電子装置の構成例を示すブロック図である。It is a block diagram showing an example of composition of an electronic device to which this indication is applied. イメージセンサの第１の間引き駆動の例を説明するための図である。It is a figure for demonstrating the example of the 1st thinning drive of an image sensor. イメージセンサの第１の間引き駆動の例を説明するための図である。It is a figure for demonstrating the example of the 1st thinning drive of an image sensor. 短蓄画素と長蓄画素それぞれの蓄積時間と画素値の関係を示す図である。It is a figure which shows the relationship between the accumulation time and pixel value of each short accumulation pixel and long accumulation pixel. 短蓄画素と長蓄画素それぞれの出力を合成する例を示す図である。It is a figure which shows the example which synthesize | combines each output of a short accumulation pixel and a long accumulation pixel. 短冊状の各領域における短蓄画素と長蓄画素の配置を示す図である。It is a figure which shows arrangement | positioning of the short accumulation pixel and long accumulation pixel in each area | region of a strip shape. Bayer配列のＲ，Ｇ，Ｂ画素値を輝度に変換する例を示す図である。It is a figure which shows the example which converts the R, G, B pixel value of a Bayer arrangement | sequence into a brightness | luminance. 輝度Ｙを照度Luxに変換する変換式とその例を示す図である。It is a figure which shows the conversion formula which converts the brightness | luminance Y to illumination intensity Lux, and its example. フレーム毎の照度をフレーム間で移動平均した例を示す図である。It is a figure which shows the example which carried out moving average of the illumination intensity for every flame | frame between frames. イメージセンサの第２の間引き駆動の例を説明するための図である。It is a figure for demonstrating the example of the 2nd thinning drive of an image sensor. イメージセンサの第２の間引き駆動の例を説明するための図である。It is a figure for demonstrating the example of the 2nd thinning drive of an image sensor. イメージセンサの第３の間引き駆動の例を説明するための図である。It is a figure for demonstrating the example of the 3rd thinning drive of an image sensor. 第１乃至第３の間引き駆動の変形例を説明するための図である。It is a figure for demonstrating the modification of the 1st thru | or 3rd thinning drive. イメージセンサが像面位相差検出画素を含む場合の例示す図である。It is a figure which shows the example in case an image sensor contains an image surface phase difference detection pixel.
　以下、本開示を実施するための最良の形態（以下、実施の形態と称する）について、図面を参照しながら詳細に説明する。 Hereinafter, the best mode for carrying out the present disclosure (hereinafter referred to as an embodiment) will be described in detail with reference to the drawings.
「電子装置の構成例」
　図１は、本開示の実施の形態であり、イメージセンサ出力を用いて照度計測を行い、計測された照度に応じて動作状況を制御する電子装置の構成例を示している。 “Example of Electronic Device Configuration”
FIG. 1 is an embodiment of the present disclosure, and shows a configuration example of an electronic device that performs illuminance measurement using an image sensor output and controls an operation state according to the measured illuminance.
　該電子装置１０は、制御部１１、入力部１２、設定部１９、および駆動部２２を有する。制御部１１は、入力部１２の動作を制御するとともに、入力部１２からのデータに基づいて駆動部２２を制御する。より具体的には、入力部１２から入力される照度に基づいて、駆動部２２の動作状況を制御する。 The electronic device 10 includes a control unit 11, an input unit 12, a setting unit 19, and a drive unit 22. The control unit 11 controls the operation of the input unit 12 and controls the drive unit 22 based on data from the input unit 12. More specifically, the operating state of the drive unit 22 is controlled based on the illuminance input from the input unit 12.
　入力部１２は、画像信号を生成するイメージセンサ１３と、画像信号を処理する画像処理部１４を有する。イメージセンサ１３は、多数の光電変換素子（以下、画素と称する）からなり、制御部１１からの制御に従い、通常撮像時には全ての画素を同時に駆動し、その出力としての画素信号を画像処理部１４に出力する。また、イメージセンサ１３は、照度計測時には全ての画素のうちの一部（詳細後述）を間引き駆動し、その出力としての画素信号を画像処理部１４に出力する。 The input unit 12 includes an image sensor 13 that generates an image signal and an image processing unit 14 that processes the image signal. The image sensor 13 is composed of a large number of photoelectric conversion elements (hereinafter referred to as pixels). According to control from the control unit 11, all pixels are simultaneously driven during normal imaging, and a pixel signal as an output thereof is output to the image processing unit 14. Output to. Further, the image sensor 13 thins out some of all the pixels (details will be described later) during illuminance measurement, and outputs a pixel signal as an output to the image processing unit 14.
　なお、本実施の形態において、イメージセンサ１３は制御部１１の制御に従って駆動されるが、イメージセンサ１３自身に制御部を設け、通常撮像時と照度計測時の動作をイメージセンサ１３が自ら制御するようにしてもよい。 In the present embodiment, the image sensor 13 is driven in accordance with the control of the control unit 11, but the image sensor 13 itself is provided with a control unit, and the image sensor 13 controls the operations during normal imaging and illuminance measurement. You may do it.
　画像処理部１４は、照度計測時において、照度演算部としてイメージセンサ１３から入力される画像信号に基づき照度を演算して制御部１１に出力する。画像処理部１４における照度演算は、フレーム毎に異なる領域が間引き駆動されるイメージセンサ１３の出力に基づいてフレーム毎の照度が演算され、フレーム毎に照度がフレーム間で移動平均されて駆動部２２の制御に利用される。 The image processing unit 14 calculates the illuminance based on the image signal input from the image sensor 13 as the illuminance calculation unit and outputs the illuminance to the control unit 11 when measuring the illuminance. In the illuminance calculation in the image processing unit 14, the illuminance for each frame is calculated based on the output of the image sensor 13 in which different regions are thinned out for each frame, and the illuminance is moved and averaged between the frames for each frame. It is used for control.
　さらに、入力部１２は、計時を行うタイマ１２、音声を入力するマイクロフォン１６、ユーザの操作を入力するマウス、キーボード、タッチパネルなどのユーザインタフェース(I/F)１７および電子装置１０の位置や変位を検出するジャイロセンサ１８などを含むことができる。 Further, the input unit 12 is a timer 12 for measuring time, a microphone 16 for inputting sound, a user interface (I / F) 17 such as a mouse, a keyboard, and a touch panel for inputting a user operation, and the position and displacement of the electronic device 10. The gyro sensor 18 to detect may be included.
　設定部１９は、制御用プログラムや各種の係数などを保持するメモリ２０、およびネットに接続してデータを通信する通信部２１を有する。駆動部２２は、電源部２３、記憶部２４、出力インタフェース(I/F)２５、モータ２６、表示部２７、スピーカ２８、照明部２９などを含むことができる。 The setting unit 19 includes a memory 20 that holds a control program, various coefficients, and the like, and a communication unit 21 that communicates data by connecting to a network. The drive unit 22 can include a power supply unit 23, a storage unit 24, an output interface (I / F) 25, a motor 26, a display unit 27, a speaker 28, an illumination unit 29, and the like.
「イメージセンサ１３の第１の間引き駆動の例について」
　次に、図２および図３はイメージセンサ１３の第１の間引き駆動の例を説明するための図である。なお、図中の斜線領域が駆動される画素の領域を示しており、図２は８フレームを１周期とした場合の１フレーム目の駆動領域を示し、図３は８フレームを１周期とした場合の２フレーム目の駆動領域を示している。また、図２のＡの最小の矩形は１画素を、図２のＢおよび図３の最小の矩形は２×２画素を示している。 “Example of first thinning drive of image sensor 13”
Next, FIGS. 2 and 3 are diagrams for explaining an example of the first thinning driving of the image sensor 13. The shaded area in the figure shows the pixel area to be driven. FIG. 2 shows the drive area of the first frame when 8 frames are taken as one cycle, and FIG. 3 shows 8 frames as one cycle. The drive area of the second frame in this case is shown. Further, the smallest rectangle in FIG. 2A represents one pixel, and the smallest rectangle in FIGS. 2B and 3 represents 2 × 2 pixels.
　第１の間引き駆動の例では、イメージセンサ１３の全画素を、横幅２画素、縦幅１６画素の縦方向に長い短冊状の領域に区切って、該領域を８フレーム毎に１回だけ駆動させる。各領域を駆動させる順序は、図中のイメージセンサ１３の上辺の記載された数字のとおりであり、単に一方向に隣の領域に移動させるのではなく、間隔を空けて左右に移動させる。 In the first thinning driving example, all pixels of the image sensor 13 are divided into strip-like regions having a width of 2 pixels and a length of 16 pixels that are long in the vertical direction, and the regions are driven only once every 8 frames. . The order of driving each region is as indicated by the numbers on the upper side of the image sensor 13 in the figure, and is not simply moved to the adjacent region in one direction, but is moved left and right at intervals.
　なお図２のＡに示されるように、イメージセンサ１３を構成する画素の色の配列はBayer配列であり、各色の画素は飽和するまでの蓄積時間が異なる短蓄画素Ｒ，Ｇｒ，Ｇｂ，Ｂ、または長蓄画素ｒ，ｇｒ，ｇｂ，ｂが配置されている。短冊状の各領域は、ＦＤ(Floating Diffusion)を共有する２×４画素の短蓄画素と、２×４画素の長蓄画素とが交互に配置されている。 As shown in FIG. 2A, the color arrangement of the pixels constituting the image sensor 13 is a Bayer arrangement, and the short accumulation pixels R, Gr, Gb, B having different accumulation times until the pixels of each color are saturated. Or long accumulation pixels r, gr, gb, b are arranged. In each strip-shaped region, 2 × 4 pixel short accumulation pixels and 2 × 4 pixel long accumulation pixels sharing FD (Floating Diffusion) are alternately arranged.
　このように、短蓄画素のＦＤと長蓄画素のＦＤを交互に配置することにより、同一撮像領域であることが前提の、画素値をそれぞれ加算する短蓄画素と長蓄画素の重心を短冊状の該領域の中心に近づけることができる。なお、短蓄画素と長蓄画素の配置は、それらの重心が揃えばよく、上記の他、短蓄画素で長蓄画素を挟んだり、反対に長蓄画素で短蓄画素を挟んだりしてもよい。また、縦に並べるのではなく、横に並べてもよい。さらに、他にＦＤ共有単位で長蓄と短蓄を行う必要はなく、画素別に蓄積時間を変更できる機構を備えていればＦＤ共有の中で長蓄画素と短蓄画素を織り交ぜてもよい。 In this way, by alternately arranging the FDs of the short accumulation pixels and the FDs of the long accumulation pixels, the center of gravity of the short accumulation pixels and the long accumulation pixels to which the pixel values are added is premised on the same imaging region. Can be close to the center of the area. In addition, the arrangement of the short accumulation pixel and the long accumulation pixel is sufficient if their centers of gravity are aligned. In addition to the above, the short accumulation pixel is sandwiched between the long accumulation pixels or the short accumulation pixel is sandwiched between the long accumulation pixels. Also good. Moreover, you may arrange not horizontally but horizontally. Furthermore, it is not necessary to perform long accumulation and short accumulation in units of FD sharing, and long accumulation pixels and short accumulation pixels may be interlaced in FD sharing as long as a mechanism capable of changing the accumulation time for each pixel is provided. .
　図４は、短蓄画素と長蓄画素それぞれの蓄積時間と画素値の関係を示している。同図に示されるように、特性が異なる短蓄画素と長蓄画素それぞれの出力を合成することにより、ダイナミックレンジが広く、精度の高い画素を得ることができる。 FIG. 4 shows the relationship between the accumulation time and the pixel value of each of the short accumulation pixel and the long accumulation pixel. As shown in the figure, by combining the outputs of the short accumulation pixel and the long accumulation pixel having different characteristics, a pixel having a wide dynamic range and high accuracy can be obtained.
　図５は、短蓄画素と長蓄画素それぞれの出力を合成する場合に、短蓄画素の出力を長蓄画素の出力に換算して合成する例を示している。具体的には、長蓄画素の飽和点から、短蓄画素の出力に蓄積時間比をかけた値に切り替える。この閾値は画素特性に応じて色別に設定してもよい、また微調整ができるように外部から設定できるようにしてもよい。 FIG. 5 shows an example in which the output of the short accumulation pixel is converted into the output of the long accumulation pixel when the output of each of the short accumulation pixel and the long accumulation pixel is synthesized. Specifically, the output is switched from the saturation point of the long accumulation pixel to a value obtained by multiplying the output of the short accumulation pixel by the accumulation time ratio. This threshold value may be set for each color according to pixel characteristics, or may be set from the outside so that fine adjustment is possible.
　図６は、短冊状の各領域における短蓄画素と長蓄画素の配置を示している。同図に示されるように、短蓄画素と長蓄画素それぞれの出力を合成する前に、短冊状の各領域において、ＦＤを共有する短蓄画素の８画素と、長蓄画素の８画素のそれぞれで各色毎に画素値を加算しておく。 FIG. 6 shows the arrangement of short accumulation pixels and long accumulation pixels in each strip-shaped region. As shown in the figure, before synthesizing the outputs of the short accumulation pixels and the long accumulation pixels, in each of the strip-shaped areas, 8 pixels of the short accumulation pixels sharing the FD and 8 pixels of the long accumulation pixels In each case, the pixel value is added for each color.
　このようにして２×１６画素から合成された２×２画素の画素値Ｒ，Ｇ，Ｂを輝度値Ｙに変換する。図７は、Bayer配列の画素値Ｒ，Ｇ，Ｂを輝度Ｙに変換する変換式とその例を示している。この変換式の係数は、予め製造時にメモリ２０などに記録されているが、これらの係数を、通信部２１を介して外部から設定できるようにしてもよい。 The pixel values R, G, B of 2 × 2 pixels synthesized from 2 × 16 pixels in this way are converted into luminance values Y. FIG. 7 shows a conversion formula for converting pixel values R, G, and B of the Bayer array into luminance Y and an example thereof. The coefficients of the conversion formula are recorded in advance in the memory 20 at the time of manufacture. However, these coefficients may be set from the outside via the communication unit 21.
　さらに、輝度Ｙを照度Luxに変換する。図８は、輝度Ｙを照度Luxに変換する変換式とその例を示している。この変換式の係数は、予め製造時にメモリ２０などに記録されているが、これらの係数を通信部２１を介して外部から設定できるようにしてもよい。 Furthermore, luminance Y is converted into illuminance Lux. FIG. 8 shows a conversion formula for converting luminance Y into illuminance Lux and an example thereof. The coefficients of the conversion formula are recorded in advance in the memory 20 at the time of manufacture. However, these coefficients may be set from the outside via the communication unit 21.
　図９は、縦軸は照度、横軸はフレームであって、フレーム毎に算出した照度を帰還率50%のIIRフィルタによりフレーム間で移動平均した例を示している。同図に示されるように、フレーム毎の照度がばらついていても、フレーム間で移動平均をとることにより、特定のフレームで照度が突出したとしても、その影響を抑えることができる。 FIG. 9 shows an example in which the vertical axis represents the illuminance and the horizontal axis represents the frame, and the illuminance calculated for each frame is moving averaged between frames using an IIR filter with a feedback rate of 50%. As shown in the figure, even if the illuminance varies from frame to frame, even if the illuminance protrudes in a specific frame, the influence can be suppressed by taking a moving average between frames.
　例えば、図２および図３に輝点として示されるように、第６フレームおよび第８フレームで駆動される領域に高輝度光源からの光が照射されていた場合、第６フレームおよび第８フレームで演算される照度は他のフレームの照度に比較して突出して高くなる。しかしながら、最終的にはフレーム間で移動平均された照度が駆動部２２の制御に採用されるので、高輝度光源の影響を抑止することができる。 For example, as shown as bright points in FIGS. 2 and 3, when light from a high-intensity light source is irradiated on the areas driven by the sixth and eighth frames, the sixth and eighth frames are used. The calculated illuminance is significantly higher than the illuminance of other frames. However, since the illuminance that has been moving averaged between the frames is finally adopted for the control of the drive unit 22, the influence of the high-intensity light source can be suppressed.
　なお、移動平均の安定性と応答性に対する要求に応じて、移動平均のTAP数を変更してもよい。また、IIRフィルタの代わりにFIRフィルタを用いるようにしてもよい。 It should be noted that the number of TAPs in the moving average may be changed according to the demand for moving average stability and responsiveness. Further, an FIR filter may be used instead of the IIR filter.
　さらに、フレーム毎に画素の駆動領域を移動させることは、画素毎のアクセス回数が均一化されるため、駆動時間に起因する画素の劣化を均一化する副次効果も期待できる。なお、画素を駆動させる領域の移動はフレーム毎ではなくイメージセンサ１３の起動毎に変更するようにしてもよい。 Furthermore, moving the pixel drive area for each frame can equalize the number of accesses for each pixel, so that a secondary effect of making the deterioration of the pixels due to the drive time uniform can also be expected. The movement of the region for driving the pixels may be changed not every frame but every time the image sensor 13 is activated.
「イメージセンサ１３の第２の間引き駆動の例について」
　次に、図１０および図１１はイメージセンサ１３の第２の間引き駆動の例を説明するための図である。第２の間引き駆動の例は、上述した第１の間引き駆動の例が横方向の間引きのみであったものに、さらに縦方向の間引きを追加したものである。 “Example of second thinning drive of image sensor 13”
Next, FIGS. 10 and 11 are diagrams for explaining an example of the second thinning driving of the image sensor 13. The example of the second thinning drive is obtained by further adding a thinning in the vertical direction to the above-described example of the first thinning driving in which only the thinning in the horizontal direction is performed.
　なお、図中の斜線領域が駆動される画素の領域を示しており、図１０は１６フレームを１周期とした場合の２フレーム目の駆動領域を示し、図１１は１６フレームを１周期とした場合の３フレーム目の駆動領域を示している。また、図１０のＡの最小の矩形は１画素を、図１０のＢおよび図１１の最小の矩形は２×２画素を示している。 The hatched area in the figure shows the area of the pixel to be driven. FIG. 10 shows the driving area of the second frame when 16 frames are taken as one period, and FIG. 11 shows 16 frames as one period. The drive area of the third frame in this case is shown. Further, the smallest rectangle in FIG. 10A represents one pixel, and the smallest rectangle in FIGS. 10B and 11 represents 2 × 2 pixels.
　第２の間引き駆動の例では、イメージセンサ１３の全画素を、横幅２画素、縦幅１６画素の縦方向に長い短冊状の領域に区切って、該領域を１６フレーム毎に１回だけ駆動させる。各領域を駆動させる順序は、図中のイメージセンサ１３の上辺の記載された数字のとおりであり、単に一方向に隣の領域に移動させるのではなく、間隔を空けて左右に移動させる。 In the second thinning driving example, all the pixels of the image sensor 13 are divided into strip-like regions that are 2 pixels wide and 16 pixels long in the vertical direction, and the region is driven only once every 16 frames. . The order of driving each region is as indicated by the numbers on the upper side of the image sensor 13 in the figure, and is not simply moved to the adjacent region in one direction, but is moved left and right at intervals.
　第２の間引き駆動を実施することにより、第１の間引き駆動を実施する場合に比較してさらに消費電力を半分に抑えることができる。 By implementing the second thinning drive, the power consumption can be further reduced to half compared to the case of performing the first thinning drive.
　ただし、第２の間引き駆動の例のように縦方向の間引きを追加した場合、イメージセンサ１３のＯＢ(Optical Black)領域（不図示）が不足してしまうことがある。その場合、縦方向に同じ画素を駆動させるか、駆動させないフレームを設けるようにする。駆動させないフレームを設けたときの黒レベルの調整は、前フレームの黒レベルを用いるか、黒レベルの変更を行わないようにする。 However, when vertical thinning is added as in the second thinning driving example, an OB (Optical Black) region (not shown) of the image sensor 13 may be insufficient. In that case, the same pixel is driven in the vertical direction, or a frame that is not driven is provided. When adjusting the black level when a frame that is not driven is provided, the black level of the previous frame is used or the black level is not changed.
　なお、図１０のＡに示されるように、イメージセンサ１３を構成する画素の色の配列については、図２のＡを参照して説明した第１の間引き駆動の例と同様であるので、その説明は省略する。 As shown in FIG. 10A, the arrangement of the colors of the pixels constituting the image sensor 13 is the same as the first thinning drive example described with reference to FIG. Description is omitted.
「イメージセンサ１３の第３の間引き駆動の例について」
　図１２は、イメージセンサ１３の第３の間引き駆動の例を説明するための、イメージセンサ１３の全領域を示しており、図中の斜線領域が駆動される画素の領域を示している。すなわち、第３の間引き駆動の例は、イメージセンサ１３の全領域を複数の矩形領域に区切り、該矩形領域のうちの所定の矩形領域においてのみ、上述した第１または第２の間引き駆動を実施するようにする。 “Example of third thinning drive of image sensor 13”
FIG. 12 shows the entire area of the image sensor 13 for explaining an example of the third thinning driving of the image sensor 13, and the hatched area in the drawing shows the area of the pixel to be driven. That is, in the third thinning drive example, the entire area of the image sensor 13 is divided into a plurality of rectangular areas, and the first or second thinning driving described above is performed only in a predetermined rectangular area of the rectangular areas. To do.
　具体的には、イメージセンサ１３の全領域１９２０画素×１０８０画素を、４×４の矩形領域に分割した例であり、この場合、１６bitのALS_GRID_ENの各bitが各矩形領域の駆動をONとする設定に利用できる。 Specifically, the entire area 1920 pixels × 1080 pixels of the image sensor 13 is divided into 4 × 4 rectangular areas. In this case, each bit of 16-bit ALS_GRID_EN turns on driving of each rectangular area. Can be used for configuration.
　第３の間引き駆動を実施することにより、第１または第２の間引き駆動を実施する場合に比較してさらに消費電力を抑えることができる。 By performing the third thinning drive, it is possible to further reduce the power consumption compared to the case where the first or second thinning drive is performed.
　画素を駆動させる特定の矩形領域は、外部から指定するようにしてもよいし、フレーム毎に随時変更するようにしてもよい。 The specific rectangular area for driving the pixel may be designated from the outside, or may be changed as needed for each frame.
　なお、当然ながら、第１乃至第３の間引き駆動の例を実施した場合に得られる画像は、通常の全画素を駆動させた画像に比較して画素数が少なく解像度が劣るものの照度の演算、光源の位置特定、動作物の判断には支障がない。 Of course, the image obtained when the first to third thinning driving examples are performed has an illuminance calculation although the number of pixels is small and the resolution is inferior to that of an image in which all normal pixels are driven. There is no problem in identifying the position of the light source and determining the moving object.
「イメージセンサ１３の第１乃至第３の間引き駆動における変形例について」
　図１３のＡは、上述したイメージセンサ１３の第１乃至第３の間引き駆動の例を、図１３のＢは第１乃至第３の間引き駆動の変形例として、Ｒ列の画素とＢ列の画素とを単独駆動させる例を示している。図１３のＢに示された例の場合、得られる画像の空間周波数は保持したままで、駆動させる画素の数を半分にすることができ、その分だけ消費電力を抑えることができる。なお、Ｒ列の画素とＢ列の画素が空間的に離れることにより偽色が発生し得るが、照度計測の用途には問題ない。 “Variation in First to Third Thinning Drive of Image Sensor 13”
13A shows an example of the first to third thinning driving of the image sensor 13, and FIG. 13B shows a modified example of the first to third thinning driving. An example in which pixels are driven independently is shown. In the case of the example shown in FIG. 13B, the number of pixels to be driven can be halved while maintaining the spatial frequency of the obtained image, and power consumption can be reduced by that amount. A false color may be generated when the pixels in the R column and the pixels in the B column are spatially separated, but this is not a problem for use in illuminance measurement.
「イメージセンサ１３に像面位相差検出画素が設けられている場合の照度演算について」
　図１４は、像面位相差ＡＦ(Auto Focus)機能を備えるデジタルカメラなどに搭載されるイメージセンサ１３に像面位相差検出画素を設ける場合の配置例と、その場合に対応する照度演算の例を示している。 "Illuminance calculation when image plane phase difference detection pixel is provided in image sensor 13"
FIG. 14 shows an arrangement example when an image plane phase difference detection pixel is provided in an image sensor 13 mounted on a digital camera or the like having an image plane phase difference AF (Auto Focus) function, and an example of illuminance calculation corresponding to that case. Is shown.
　イメージセンサ１３に像面位相差検出画素を設ける場合、像面位相差検出画素をBayer配列のＧｒ，ｇｒの位置に配置し、照度演算にはＧｒ，ｇｒの出力を用いないようにする。なお、イメージセンサ１３に赤外線検出画素を設けた場合についても同様に、赤外線検出画素の出力を照度演算に用いないようにすればよい。 When the image plane phase difference detection pixel is provided in the image sensor 13, the image plane phase difference detection pixel is arranged at the position of Gr, gr in the Bayer array so that the output of Gr, gr is not used for the illuminance calculation. Similarly, when an infrared detection pixel is provided in the image sensor 13, the output of the infrared detection pixel should not be used for illuminance calculation.
「照度に基づく駆動制御の例」
　制御部１１は、画像処理部１４から入力される照度に基づいて駆動部２２の動作を制御することができる。 "Example of drive control based on illuminance"
The control unit 11 can control the operation of the drive unit 22 based on the illuminance input from the image processing unit 14.
　例えば、該電素装置１０が火災報知機や監視モニタに適用された場合には、急に照度が上がって明るくなった部位をズーム撮影するなど、照度計測モードから通常撮影モードに切換えたり、撮影モードで撮影されたビデオの録画を開始したり、警報を鳴らしたりすることができる。 For example, when the electronic device 10 is applied to a fire alarm or a monitoring monitor, the illuminance measurement mode is switched from the illuminance measurement mode to the normal imaging mode, such as zooming the area where the illuminance suddenly increases and becomes brighter, or imaging. You can start recording a video shot in the mode or sound an alarm.
　例えば、該電素装置１０が車載カメラに適用された場合には、対向車のライトの位置から自車のライトの向きを対向車からずらすようにライトの照射軸を動かすモータを制御することができる。 For example, when the electronic device 10 is applied to an in-vehicle camera, it is possible to control a motor that moves the irradiation axis of the light so as to shift the direction of the light of the own vehicle from the position of the oncoming vehicle. it can.
　例えば、該電素装置１０が家庭電化製品に適用された場合には、計測された照度に合わせて時刻も参照し、時間帯が夜間であって周囲が暗く動く物体が存在しないときには消費電力を抑制する省電力モードで動作するように制御することができる。 For example, when the electric device 10 is applied to a home appliance, the time is also referred to according to the measured illuminance, and the power consumption is reduced when the time zone is night and there is no moving object in the dark. It can be controlled to operate in the power saving mode to be suppressed.
　例えば、該電素装置１０が携帯電話機、スマートフォンなどに適用された場合には、時間帯が昼間であるにも拘らず照度が低いときには該携帯電話などがポケットやカバンに入れられたと判断して、誤動作防止の目的でタッチパネルをオフとしたり、着信音量を上げたりする。反対に、時間帯が夜間であって照度が低いときにはユーザが就寝中である判断して、スピーカの音量を下げたりする。このように、周囲の環境に応じて制御動作を変えることができる。 For example, when the electronic device 10 is applied to a mobile phone, a smart phone, etc., it is determined that the mobile phone has been put in a pocket or a bag when the illumination is low even though the time zone is daytime. To prevent malfunctions, turn off the touch panel or increase the ringtone volume. Conversely, when the time zone is night and the illuminance is low, it is determined that the user is sleeping and the volume of the speaker is lowered. In this way, the control operation can be changed according to the surrounding environment.
　以上に説明したように、本実施の形態である電子装置１０によれば、照度計測のための専用の回路構成（照度計など）を設けることなく、照度を計測することができる。また、照度計測時にはイメージセンサ１３の画素を間引いて駆動するので、イメージセンサ１３の全画素を駆動させる場合に比較して消費電力を抑えることができる。また、全画素を読み出す場合に比較して速やかに照度を演算することができる。 As described above, according to the electronic device 10 of the present embodiment, it is possible to measure illuminance without providing a dedicated circuit configuration (illuminance meter or the like) for illuminance measurement. Further, since the pixels of the image sensor 13 are driven by thinning out during illuminance measurement, power consumption can be suppressed as compared with the case where all the pixels of the image sensor 13 are driven. Further, the illuminance can be calculated more quickly than when all the pixels are read out.
　さらに、間引きして駆動させる画素の領域をフレーム毎などで変更し、フレーム毎の照度をフレーム間で移動平均するので、最終的に得られる照度のばらつきを抑え、精度を保つことができる。間引きして駆動させる画素の領域をフレーム毎などで変更させることにより、画素の劣化を均一化する副次効果も期待できる。 Furthermore, since the pixel area to be driven by thinning is changed for each frame or the like, and the illuminance for each frame is moving averaged between the frames, variation in the finally obtained illuminance can be suppressed and accuracy can be maintained. By changing the pixel area to be thinned and driven for each frame or the like, it is possible to expect a secondary effect to make the deterioration of the pixels uniform.
　また、画像全体のうちの特定の領域の画素だけを駆動させて、その出力から照度を演算することもできる。これにより、撮像範囲のうちの特定の領域（画角）のみの照度を求めることが可能となる。 It is also possible to drive only the pixels in a specific area of the entire image and calculate the illuminance from the output. Thereby, it becomes possible to obtain the illuminance of only a specific region (view angle) in the imaging range.
　照度計測用に得た画素値を低解像度の画像として出力することもできる。 画素 Pixel values obtained for illuminance measurement can be output as low-resolution images.
　なお、本開示の実施の形態は、上述した実施の形態に限定されるものではなく、本開示の要旨を逸脱しない範囲において種々の変更が可能である。 Note that the embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present disclosure.
　なお、本開示は以下のような構成も取ることができる。
（１）
　多数の画素から成るイメージセンサ部と、
　前記イメージセンサ部を成す前記多数の画素を間引き駆動させるとともに、間引き駆動させる画素を周期的に変更する制御部と、
　前記多数の画素が間引き駆動されたときの前記イメージセンサ部の出力を用いて照度を演算する照度演算部と
　を備える電子装置。
（２）
　前記照度演算部は、前記多数の画素が間引き駆動される毎に照度を演算し、さらに演算した前記照度を移動平均した値を出力する
　前記（１）に記載の電子装置。
（３）
　前記照度演算部から出力された前記照度に基づく制御に従って動作する駆動部を
　さらに備える前記（１）または（２）に記載の電子装置。
（４）
　前記制御部は、前記イメージセンサ部を成す前記多数の画素を所定のサイズの領域に区切り、駆動させる前記領域をフレーム毎または起動毎に周期的に変更する
　前記（１）から（３）のいずれかに記載の電子装置。
（５）
　前記イメージセンサ部は、短蓄画素と長蓄画素から成り、
　前記照度演算部は、同時に駆動された前記短蓄画素の出力と前記長蓄画素の出力を用いて照度を演算する
　前記（１）から（４）のいずれかに記載の電子装置。
（６）
　前記イメージセンサ部は、像面位相差検出用途を兼ねる画素を含み、
　前記照度演算部は、駆動された前記像面位相差検出用途を兼ねる画素の出力を用いることなく照度を演算する
　前記（１）から（５）のいずれかに記載の電子装置。 In addition, this indication can also take the following structures.
(1)
An image sensor unit comprising a large number of pixels;
A controller that thinly drives the pixels forming the image sensor unit and periodically changes the pixels to be thinned; and
An illuminance calculation unit that calculates illuminance using an output of the image sensor unit when the plurality of pixels are thinned out.
(2)
The electronic device according to (1), wherein the illuminance calculation unit calculates illuminance every time the large number of pixels are driven to be thinned, and further outputs a value obtained by moving average the calculated illuminance.
(3)
The electronic device according to (1) or (2), further including a drive unit that operates according to control based on the illuminance output from the illuminance calculation unit.
(4)
The control unit divides the large number of pixels forming the image sensor unit into regions of a predetermined size, and periodically changes the region to be driven for each frame or for each activation. An electronic device according to any one of the above.
(5)
The image sensor unit includes a short accumulation pixel and a long accumulation pixel,
The electronic device according to any one of (1) to (4), wherein the illuminance calculation unit calculates illuminance using an output of the short accumulation pixel and an output of the long accumulation pixel that are simultaneously driven.
(6)
The image sensor unit includes a pixel that also serves as an image plane phase difference detection application,
The electronic device according to any one of (1) to (5), wherein the illuminance calculation unit calculates illuminance without using an output of a driven pixel that also serves as an image plane phase difference detection application.
　１０　電子装置，　１１　制御部，　１２　入力部，１３　イメージセンサ，　１４　画像処理部，　１９　設定部，　２２　駆動部 10 electronic device, 11 control unit, 12 input unit, 13 image sensor, 14 image processing unit, 19 setting unit, 22 drive unit

Claims (7)

  1. 　多数の画素から成るイメージセンサ部と、
    　前記イメージセンサ部を成す前記多数の画素を間引き駆動させるとともに、間引き駆動させる画素を周期的に変更する制御部と、
    　前記多数の画素が間引き駆動されたときの前記イメージセンサ部の出力を用いて照度を演算する照度演算部と
    　を備える電子装置。     An image sensor unit comprising a large number of pixels;
    A controller that thinly drives the pixels forming the image sensor unit and periodically changes the pixels to be thinned; and
    An illuminance calculation unit that calculates illuminance using an output of the image sensor unit when the plurality of pixels are thinned out.
  2. 　前記照度演算部は、前記多数の画素が間引き駆動される毎に照度を演算し、さらに演算した前記照度を移動平均した値を出力する
    　請求項１に記載の電子装置。     The electronic device according to claim 1, wherein the illuminance calculation unit calculates an illuminance every time the plurality of pixels are thinned out and outputs a value obtained by moving and averaging the calculated illuminance.
  3. 　前記照度演算部から出力された前記照度に基づく制御に従って動作する駆動部を
    　さらに備える請求項２に記載の電子装置。     The electronic device according to claim 2, further comprising a drive unit that operates according to control based on the illuminance output from the illuminance calculation unit.
  4. 　前記制御部は、前記イメージセンサ部を成す前記多数の画素を所定のサイズの領域に区切り、駆動させる前記領域をフレーム毎または起動毎に周期的に変更する
    　請求項２に記載の電子装置。     The electronic device according to claim 2, wherein the control unit divides the plurality of pixels forming the image sensor unit into regions of a predetermined size, and periodically changes the region to be driven for each frame or for each activation.
  5. 　前記イメージセンサ部は、短蓄画素と長蓄画素から成り、
    　前記照度演算部は、同時に駆動された前記短蓄画素の出力と前記長蓄画素の出力を用いて照度を演算する
    　請求項２に記載の電子装置。     The image sensor unit includes a short accumulation pixel and a long accumulation pixel,
    The electronic device according to claim 2, wherein the illuminance calculation unit calculates illuminance using the output of the short accumulation pixel and the output of the long accumulation pixel that are driven simultaneously.
  6. 　前記イメージセンサ部は、像面位相差検出用途を兼ねる画素を含み、
    　前記照度演算部は、駆動された前記像面位相差検出用途を兼ねる画素の出力を用いることなく照度を演算する
    　請求項５に記載の電子装置。     The image sensor unit includes a pixel that also serves as an image plane phase difference detection application,
    The electronic apparatus according to claim 5, wherein the illuminance calculation unit calculates illuminance without using the output of the driven pixel that also serves as an image plane phase difference detection application.
  7. 　多数の画素から成るイメージセンサにおいて、
    　通常撮像時には、前記多数の画素を同時に駆動し、
    　照度計測時には、前記多数の画素を間引き駆動するとともに、間引き駆動する画素を周期的に変更する
    　イメージセンサ。     In an image sensor consisting of many pixels,
    During normal imaging, the multiple pixels are driven simultaneously,
    An image sensor that, during illuminance measurement, thinly drives the pixels and periodically changes the pixels to be thinned.
PCT/JP2014/073055 2013-09-12 2014-09-02 Electronic device and image sensor WO2015037474A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013189724 2013-09-12
JP2013-189724 2013-09-12

Publications (1)

Publication Number Publication Date
WO2015037474A1 true WO2015037474A1 (en) 2015-03-19

Family

ID=52665584

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2014/073055 WO2015037474A1 (en) 2013-09-12 2014-09-02 Electronic device and image sensor

Country Status (2)

Country Link
TW (1) TWI636558B (en)
WO (1) WO2015037474A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7059284B2 (en) 2017-05-03 2022-04-25 オッポ広東移動通信有限公司 Camera assembly and mobile electronics

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008042846A (en) * 2006-08-10 2008-02-21 Nec Corp Camera module comprising luminance sensor function and mobile terminal using the same
JP2008306254A (en) * 2007-06-05 2008-12-18 Sony Ericsson Mobilecommunications Japan Inc Camera module and portable terminal device
JP2009010697A (en) * 2007-06-28 2009-01-15 Kyocera Corp Portable electronic apparatus

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6201559B1 (en) * 1996-12-19 2001-03-13 Minolta Co., Ltd. Method for measuring the quantity of light emergent from an optical tip array and image forming apparatus provided with an optical tip array
JP3750734B2 (en) * 2001-07-27 2006-03-01 セイコーエプソン株式会社 Scan line driving circuit, electro-optical device, electronic apparatus, and semiconductor device
JP2008172493A (en) * 2007-01-11 2008-07-24 Matsushita Electric Ind Co Ltd Method and apparatus for processing front end signal

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008042846A (en) * 2006-08-10 2008-02-21 Nec Corp Camera module comprising luminance sensor function and mobile terminal using the same
JP2008306254A (en) * 2007-06-05 2008-12-18 Sony Ericsson Mobilecommunications Japan Inc Camera module and portable terminal device
JP2009010697A (en) * 2007-06-28 2009-01-15 Kyocera Corp Portable electronic apparatus

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7059284B2 (en) 2017-05-03 2022-04-25 オッポ広東移動通信有限公司 Camera assembly and mobile electronics

Also Published As

Publication number Publication date
TWI636558B (en) 2018-09-21
TW201511246A (en) 2015-03-16

Similar Documents

Publication Publication Date Title
US8907282B2 (en) Thermal imaging camera with intermittent image capture
CN101622859B (en) Imaging apparatus and flicker detection method
US20140062882A1 (en) Display control device, method, and program
JP2018119942A (en) Imaging device, method of monitoring the same, and program
MX2013003446A (en) System and method for processing image data using an image signal processor having back-end processing logic.
JP2014050042A (en) Image processor and solid-state imaging device
JP7250483B2 (en) Imaging device, computer program, and storage medium
JP2014006759A (en) Information processing apparatus and method, and photoelectric conversion device
JP2016154285A5 (en)
JP2020099055A (en) Imaging apparatus
WO2018225517A1 (en) Information processing device and method
WO2015037474A1 (en) Electronic device and image sensor
JP2011254340A (en) Imaging apparatus
JP2013205434A (en) Electronic apparatus
JP5895426B2 (en) Display device
JP6492588B2 (en) Projector and projector control method
JP2017126964A (en) Imaging control unit, and imaging apparatus
JP2014027587A (en) Imaging device, evaluation frame setting system, and control method for imaging device
JP6924089B2 (en) Imaging device and its control method
WO2020079946A1 (en) Flicker measurement device, flicker measurement method, flicker measurement program, flicker evaluation assistance device, flicker evaluation assistance method, and flicker evaluation assistance program
KR101147327B1 (en) Camera module and surveillance apparatus having the same
JP2007096601A (en) Imaging device
JP2015103944A (en) Imaging apparatus and image signal processing method
US20240056692A1 (en) Flicker detection
JP2005258277A (en) Image movement compensation device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14844137

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14844137

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP