KR20190051371A - Camera module including filter array of complementary colors and electronic device including the camera module - Google Patents

Abstract

According to various embodiments of the present invention, a camera module comprises: a lens module including at least one lens; and an image sensor module coupled to the lens module. The image sensor module includes: a micro lens; at least one photo diode; and a filter array arranged between the micro lens and the at least one photo diode and including a first filter capable of passing light in a first designated band and a second filter capable of passing light in a second designated band corresponding to a complementary color to a color corresponding to the first designated band. Besides, various embodiments are possible.

Description

[0001] CAMERA MODULE INCLUDING FILTER ARRAY OF COMPLEMENTARY COLORS AND ELECTRONIC DEVICE INCLUDING THE CAMERA MODULE [0002]

The present invention relates to a camera module, and more particularly, to a camera module for sensing an image signal, an electronic device including the camera module, and a method for generating image data of the camera module and / or the electronic device.

The camera module senses incident light using unit pixels of the image sensor to generate an analog signal corresponding to the incident light and convert the analog signal into a digital signal to generate an image for an external object. 2. Description of the Related Art Recently, as functions of portable electronic devices (hereinafter, electronic devices) such as smart phones and tablet PCs have diversified, electronic devices have been provided with camera modules and various functions can be performed using camera modules of electronic devices.

In order for a camera device (or an electronic device having a camera module) to sense a more accurate image, it is necessary to improve the sensitivity of the image sensor. Here, the sensitivity of the image sensor can be defined as the number of photons sensed by the image sensor compared to the exposure time of the image sensor.

According to the prior art, there are various methods for improving the sensitivity of the image sensor. However, the conventional techniques have problems in that the processing speed is slowed due to an excessive amount of calculation, the resolution is low, or the size of a unit pixel (e.g., pixel pitch) there is a problem.

It is an object of the present invention to provide a camera module and an electronic device capable of improving the sensitivity of an image sensor and processing an image with a small amount of calculation.

A camera module according to various embodiments of the present invention includes a lens module including at least one lens and an image sensor module coupled with the lens module, the image sensor module including a micro lens, at least one photodiode, A first filter capable of passing light in a first designated band and a second filter capable of passing light in a second designated band corresponding to a color corresponding to the first designated band and a different color having a complementary color relationship A filter array may be disposed between the microlens and the at least one photodiode.

An electronic device according to various embodiments of the present invention includes a camera module and an image signal processor, wherein the camera module includes a microlens, at least one photodiode capable of converting light passing through the microlens into an electrical signal, And a first filter disposed between the microlens and the at least one photodiode and capable of passing light in a first designated band and a second filter disposed in the second band corresponding to another color having a complementary color with a color corresponding to the first designated band And a second filter capable of passing light in a second designated band, the image signal processor being configured to generate image data based on data output from the at least one photodiode .

According to various embodiments of the present invention, there is provided a method of generating image data of a camera module, the camera module comprising: a microlens; at least one photodiode arranged to correspond to the microlens; and at least one photodiode The method comprising: generating first data based on light sensed in a first region of the at least one photodiode, generating a second data in a second region of the at least one photodiode Generating second data based on the sensed light; and generating image data based on the first data and the second data, wherein the color of the first data and the color of the second data are They can be complementary to each other.

According to various embodiments of the present invention, it is possible to provide a camera module and an electronic device capable of improving the sensitivity of an image sensor and processing an image with a small calculation amount. Particularly, Can be provided.

1 is a block diagram of an electronic device in a network environment in accordance with various embodiments.
2 is a block diagram of a camera module according to various embodiments.
Figure 3 illustrates a unit pixel according to various embodiments.
Figure 4 illustrates a lens module, a microlens, and a photodiode according to various embodiments.
FIG. 5 illustrates a segmented region of a photodiode by a floating diffusion region according to various embodiments.
Figures 6A-6C illustrate a method for generating image data in accordance with various embodiments.
Figure 7 shows the wavelength range of image data according to various embodiments.
Figure 8 illustrates a segmented region of a photodiode by a floating diffusion region according to various embodiments.
Figure 9 shows a segmented area of a photodiode by a floating diffusion region according to various embodiments.
10 is a block diagram of an electronic device according to various embodiments.
11 is a block diagram of an electronic device according to various embodiments.
12 is a flowchart of a method of generating image data according to various embodiments.
13 is a flowchart of a method of generating image data according to various embodiments.

1 is a block diagram of an electronic device 101 in a network environment 100, in accordance with various embodiments. 1, an electronic device 101 in a network environment 100 communicates with an electronic device 102 via a first network 198 (e.g., near-field wireless communication) or a second network 199 (E. G., Remote wireless communication). ≪ / RTI > According to one embodiment, the electronic device 101 is capable of communicating with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input device 150, an audio output device 155, a display device 160, an audio module 170, a sensor module 176, an interface 177, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a subscriber identity module 196, and an antenna module 197 ). In some embodiments, at least one (e.g., display 160 or camera module 180) of these components may be omitted from the electronic device 101, or other components may be added. In some embodiments, some components, such as, for example, a sensor module 176 (e.g., a fingerprint sensor, an iris sensor, or an illuminance sensor) embedded in a display device 160 Can be integrated.

Processor 120 may be configured to operate at least one other component (e.g., hardware or software component) of electronic device 101 connected to processor 120 by driving software, e.g., And can perform various data processing and arithmetic operations. Processor 120 loads and processes commands or data received from other components (e.g., sensor module 176 or communication module 190) into volatile memory 132 and processes the resulting data into nonvolatile memory 134. [ Lt; / RTI > According to one embodiment, the processor 120 may operate in conjunction with a main processor 121 (e.g., a central processing unit or an application processor) and, independently, or additionally or alternatively, Or a co-processor 123 (e.g., a graphics processing unit, an image signal processor, a sensor hub processor, or a communications processor) specific to the designated function. Here, the coprocessor 123 may be operated separately from or embedded in the main processor 121.

 In such a case, the coprocessor 123 may be used in place of the main processor 121, for example, while the main processor 121 is in an inactive (e.g., sleep) state, At least one component (e.g., display 160, sensor module 176, or communications module 176) of the components of electronic device 101 (e.g., 190) associated with the function or states. According to one embodiment, the coprocessor 123 (e.g., an image signal processor or communications processor) is implemented as a component of some other functionally related component (e.g., camera module 180 or communication module 190) . Memory 130 may store various data used by at least one component (e.g., processor 120 or sensor module 176) of electronic device 101, e.g., software (e.g., program 140) ), And input data or output data for the associated command. The memory 130 may include a volatile memory 132 or a non-volatile memory 134.

The program 140 may be software stored in the memory 130 and may include, for example, an operating system 142, a middleware 144,

The input device 150 is an apparatus for receiving a command or data to be used for a component (e.g., processor 120) of the electronic device 101 from the outside (e.g., a user) of the electronic device 101, For example, a microphone, a mouse, or a keyboard may be included.

The sound output device 155 is a device for outputting a sound signal to the outside of the electronic device 101. For example, the sound output device 155 may be a speaker for general use such as a multimedia reproduction or a sound reproduction, . According to one embodiment, the receiver may be formed integrally or separately with the speaker.

Display device 160 may be an apparatus for visually providing information to a user of electronic device 101 and may include, for example, a display, a hologram device, or a projector and control circuitry for controlling the projector. According to one embodiment, the display device 160 may include a touch sensor or a pressure sensor capable of measuring the intensity of the pressure on the touch.

The audio module 170 is capable of bi-directionally converting sound and electrical signals. According to one embodiment, the audio module 170 may acquire sound through the input device 150, or may be connected to the audio output device 155, or to an external electronic device (e.g., Electronic device 102 (e.g., a speaker or headphone)).

The sensor module 176 may generate an electrical signal or data value corresponding to an internal operating state (e.g., power or temperature) of the electronic device 101, or an external environmental condition. The sensor module 176 may be a gesture sensor, a gyro sensor, a barometric sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, Or an illuminance sensor.

The interface 177 may support a designated protocol that may be wired or wirelessly connected to an external electronic device (e.g., the electronic device 102). According to one embodiment, the interface 177 may include a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may be a connector such as an HDMI connector, a USB connector, an SD card connector, or an audio connector that can physically connect the electronic device 101 and an external electronic device (e.g., the electronic device 102) (E.g., a headphone connector).

The haptic module 179 may convert electrical signals into mechanical stimuli (e.g., vibrations or movements) or electrical stimuli that the user may perceive through tactile or kinesthetic sensations. The haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture a still image and a moving image. According to one embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

The power management module 188 is a module for managing the power supplied to the electronic device 101, and may be configured as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 is an apparatus for supplying power to at least one component of the electronic device 101 and may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

The communication module 190 is responsible for establishing a wired or wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108) Lt; / RTI > Communication module 190 may include one or more communication processors that support wired communication or wireless communication, operating independently of processor 120 (e.g., an application processor). According to one embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (E.g., a local area network (LAN) communication module, or a power line communication module), and the corresponding communication module may be used to communicate with a first network 198 (e.g., Bluetooth, WiFi direct, Communication network) or a second network 199 (e.g., a telecommunications network such as a cellular network, the Internet, or a computer network (e.g., a LAN or WAN)). The various types of communication modules 190 described above may be implemented as a single chip or may be implemented as separate chips.

According to one embodiment, the wireless communication module 192 may use the user information stored in the subscriber identification module 196 to identify and authenticate the electronic device 101 within the communication network.

The antenna module 197 may include one or more antennas for externally transmitting or receiving signals or power. According to one example, the communication module 190 (e.g., the wireless communication module 192) may transmit signals to or receive signals from an external electronic device via an antenna suitable for the communication method.

Some of the components are connected to each other via a communication method (e.g., bus, general purpose input / output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI) (Such as commands or data) can be exchanged between each other.

 According to one embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic devices 102 and 104 may be the same or a different kind of device as the electronic device 101. [ According to one embodiment, all or a portion of the operations performed in the electronic device 101 may be performed in another or a plurality of external electronic devices. According to one embodiment, in the event that the electronic device 101 has to perform some function or service automatically or upon request, the electronic device 101 may be capable of executing the function or service itself, And may request the external electronic device to perform at least some functions associated therewith. The external electronic device receiving the request can execute the requested function or additional function and transmit the result to the electronic device 101. [ The electronic device 101 can directly or additionally process the received result to provide the requested function or service. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

The electronic device according to the various embodiments disclosed herein can be various types of devices. The electronic device can include, for example, at least one of a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

It should be understood that the various embodiments of the present document and the terms used therein are not intended to limit the techniques described herein to specific embodiments, but rather to include various modifications, equivalents, and / or alternatives of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar components. The singular expressions may include plural expressions unless the context clearly dictates otherwise. In this document, the expressions "A or B," "at least one of A and / or B," "A, B or C," or "at least one of A, B, and / Possible combinations. Expressions such as "first", "second", "first" or "second" may be used to qualify the components, regardless of order or importance, and to distinguish one component from another And does not limit the constituent elements. When it is mentioned that some (e.g., first) component is "(functionally or communicatively) connected" or "connected" to another (second) component, May be connected directly to the component, or may be connected through another component (e.g., a third component).

As used herein, the term " module " includes units comprised of hardware, software, or firmware and may be used interchangeably with terms such as, for example, logic, logic blocks, components, or circuits. A module may be an integrally constructed component or a minimum unit or part thereof that performs one or more functions. For example, the module may be configured as an application-specific integrated circuit (ASIC).

Various embodiments of the present document may include instructions stored on a machine-readable storage medium (e.g., internal memory 136 or external memory 138) readable by a machine (e.g., a computer) Software (e.g., program 140). The device may include an electronic device (e.g., electronic device 101) in accordance with the disclosed embodiments as an apparatus capable of calling stored instructions from the storage medium and operating according to the called instructions. When the instruction is executed by a processor (e.g., processor 120), the processor may perform the function corresponding to the instruction, either directly or using other components under the control of the processor. The instructions may include code generated or executed by the compiler or interpreter. A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' means that the storage medium does not include a signal and is tangible, but does not distinguish whether data is stored semi-permanently or temporarily on the storage medium.

According to a temporal example, the method according to various embodiments disclosed herein may be provided in a computer program product. A computer program product can be traded between a seller and a buyer as a product. A computer program product may be distributed in the form of a machine readable storage medium (eg, compact disc read only memory (CD-ROM)) or distributed online through an application store (eg PlayStore ™). In the case of on-line distribution, at least a portion of the computer program product may be temporarily stored, or temporarily created, on a storage medium such as a manufacturer's server, a server of an application store, or a memory of a relay server.

Each of the components (e.g., modules or programs) according to various embodiments may be comprised of a single entity or a plurality of entities, and some of the subcomponents described above may be omitted, or other subcomponents May be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity to perform the same or similar functions performed by each respective component prior to integration. Operations performed by a module, program, or other component, in accordance with various embodiments, may be performed sequentially, in parallel, repetitively, or heuristically, or at least some operations may be performed in a different order, .

2 is a block diagram of camera module 200, in accordance with various embodiments.

2, the camera module 200 includes a lens module 210, a flash 220, an image sensor module 230, an image stabilizer 240, a memory 250 (e.g., a buffer memory), or an image signal Processor 265, and various embodiments of the invention may be implemented, even if at least some of the depicted arrangements are omitted or substituted in accordance with specific embodiments. Camera module 200 may be provided in an electronic device (e.g., electronic device 101 of FIG. 1) to provide the acquired image data to an electronic device such as a processor (e.g., processor 120 of FIG. 1).

According to various embodiments, the lens module 210 may collect light emitted from a subject, which is an object of image capturing, and may refract light to be incident on the image sensor module 230. The lens module 210 may include one or more lenses. According to one embodiment, the camera module 200 may include a plurality of lens modules 210. In this case, the camera module 200 may be, for example, a dual camera, a 360 degree camera, or a spherical camera. The plurality of lens modules 210 may have the same lens properties (e.g., angle of view, focal length, autofocus, f number, or optical zoom), or at least one lens assembly may have at least one lens lens assembly It can have one other lens attribute. The lens module 210 may include, for example, a wide-angle lens or a telephoto lens.

According to various embodiments, the flash 220 may emit a light source that is used to enhance the light emitted from the subject. Flash 220 may include one or more light emitting diodes (e.g., red-green-blue (RGB) LEDs, white LEDs, infrared LEDs, or ultraviolet LEDs), or xenon lamps.

According to various embodiments, the camera module 200 may include a shutter (not shown) provided on the front surface, the rear surface, or between the front and rear surfaces of the lens module 210 and performing a function of passing or blocking light incident on the lens module 210, . ≪ / RTI > According to one embodiment, the camera module 200 may include an electronic shutter, which does not have a physical shutter, but can implement a shutter function by blocking the sensing value of the image sensor module 230.

According to various embodiments, the image sensor module 230 may acquire an image corresponding to the subject by converting the light transmitted from the subject through the lens module 210 into an electrical signal. According to one embodiment, the image sensor module 230 may include a selected one of the image sensors of different properties, such as, for example, an RGB sensor, a black and white (BW) sensor, an IR sensor, A plurality of image sensors having the same property, or a plurality of image sensors having different properties. Each image sensor included in the image sensor module 230 may be implemented by, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

According to various embodiments, the image sensor module 230 may include a plurality of photodiodes 234 that make up the pixels. The plurality of photodiodes 234 may be provided with the same type of elements that perform photoelectric conversion. According to various embodiments, a plurality of photodiodes 234 may constitute one unit pixel, for example, four photodiodes may constitute one unit pixel, in which case each photodiode Subpixels can be constructed. An embodiment in which the photodiodes constitute unit pixels will be described in more detail with reference to FIG.

According to various embodiments, the photodiode 234 may be divided into a plurality of regions (e.g., a first region and a second region), and light input into one region may not be detected in another region adjacent thereto.

According to one embodiment, the photodiode 234 may be divided into a first region and a second region as the floating diffusion region is activated. For example, a floating diffusion region may be formed in the vertical direction in the center of the photodiode. The floating diffusion region physically isolates the photodiode 234 from the light, but blocks light in both regions separated by an electric potential. Each floating diffusion region is connected to a transistor (not shown), and the image sensor IC 260 (or processor of the electronic device) can control the voltage supplied to the transistor to activate the floating diffusion region. When the floating diffusion region is activated, the passage of light is blocked between the first region and the second region of the photodiode 234, so that the intensity and / or the value of the light sensed in the first and second regions And may be independent of each other. The floating diffusion region may be activated or deactivated according to the control signal of the image sensor IC 260 or the processor of the electronic device. When the floating diffusion region is activated, the photodiode 234 can distinguish the first data corresponding to the light sensed in the first region and the second data corresponding to the light sensed in the second region, and read out the separated data.

According to another embodiment, the photodiode 234 may be divided into a first region and a second region through a metal film. Here, the metal film may be made of various metals capable of blocking the passage of light such as tungsten. Also in this case, the photodiode 234 can distinguish the first data corresponding to the light sensed in the first area and the second data corresponding to the light sensed in the second area, and read out the light. According to another embodiment, the photodiode 234 may include a photodiode corresponding to the first region and a photodiode corresponding to the second region, i.e., two photodiodes. In this case, it is possible to read out the first data and the second data from each photodiode with a structure in which two photodiodes are disposed under one microlens 232. Also in this embodiment, a metal film may be formed between the two photodiodes constituting the photodiode 234.

According to various embodiments, a micro lens 232 may be disposed corresponding to each photodiode 234. The microlens 232 may function to refract light such that the light passing through the lens module 210 is input to the corresponding photodiode 234.

According to various embodiments, a filter array 236 may be provided between the microlens 232 and the photodiode 234. The filter array 236, based on one photodiode 234, may include a first filter and a second filter for passing light of different bands. According to various embodiments, the first filter may pass the light of the first designated band, and the second filter may transmit the light of the second specified band corresponding to another color having a complementary color with the color corresponding to the first designated band . For example, when four photodiodes 234 constitute unit pixels (for example, R pixels, two G pixels, and B pixels), one of the photodiodes is irradiated with light of R (red) And a second filter for passing light in a C (cyan) band in a complementary color relationship with R are provided so as to correspond to the first filter and a third filter (Or a first filter of the second and third photodiodes), a fourth filter (or a second filter of the second and third photodiodes) for passing light in the M (magenta) band in a complementary color relationship with G (Or a first filter of the fourth photodiode) for passing light of the B (blue) band and a light of Y (yellow) band of a complementary color to the B are passed through the other photodiode (Or the second filter of the fourth photodiode) passing through the first filter may be correspondingly provided. By doing so, the image sensor module 230 can obtain the luminance signal obtained from the monochrome image sensor without using the monochrome image sensor as the sum of the primary color signal passing through the first filter and the complementary color signal passing through the second filter through the photodiode It is possible to obtain luminance data corresponding to the luminance data.

According to various embodiments, the filter array 236 includes a first filter and a second filter, and a third specified band (e.g., ultra-violet ray) band higher than the first designated band and the second specified band A third filter for passing light and a fourth filter for passing light of a fourth designated band (e.g., an infra-red ray band) lower than the first designated band and the second specified band. In this embodiment, the photodiode 234 may be divided into four regions through a floating diffusion region or a shielding film, and the first to fourth filters of the filter array may be arranged to correspond to each of the four regions. This embodiment will be described in more detail with reference to FIGS. 8 and 9. FIG.

The specific shape of the image sensor module 230 including the microlens 232, the filter array 236 and the photodiode 234 according to various embodiments and accordingly the data that can be sensed by the photodiode 234 5A and 5B will be described in more detail.

According to various embodiments, the image stabilizer 240 may be configured to generate a negative effect (e.g., image blur) on the image being photographed, in response to movement of the camera module 200 or the electronic device 101 comprising it (E.g., adjust read-out timing, etc.) in at least one lens or image sensor module 230 included in the lens module 210 to compensate, at least in part. According to one embodiment, the image stabilizer 240 may be implemented as an optical image stabilizer, for example, and may include a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module 200 Can be used to detect the motion.

According to various embodiments, the memory 250 may at least temporarily store at least a portion of the image acquired through the image sensor module 230 for subsequent image processing operations. For example, if image acquisition according to the shutter is delayed or a plurality of images are acquired at high speed, the acquired original image (e.g., a high resolution image) is stored in the memory 250, and a corresponding copy An image (e.g., a low resolution image) can be previewed via the display device 160. Thereafter, at least a portion of the original image that was stored in the memory 250 may be acquired and processed by, for example, the image signal processor 265 if the specified condition is satisfied (e.g., user input or system command). According to one embodiment, the memory 250 may be comprised of at least a portion of the memory 130, or a separate memory operated independently thereof.

According to various embodiments, the image sensor IC 260 may include an image signal processor 265 as a configuration for performing control and image processing of each configuration of the camera module 200.

According to various embodiments, image signal processor 265 may perform image processing (e.g., depth map generation, three-dimensional modeling, etc.) on an image acquired through image sensor module 230 or an image stored in memory 250 (E.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening), in addition to or in addition to, The image signal processor 265 may provide control (e.g., exposure time control, or lead-out timing control) to at least one of the components (e.g., image sensor module 230) included in the camera module 200 The image processed by the image signal processor 265 may be stored back in the memory 250 for further processing or may be stored in an external component of the camera module 200, The image signal processor 265 may be coupled to the processor (e. G., The processor 130, the display device 160, the electronic device 102, the electronic device 104, or the server 108) 120, or may be configured as a separate processor running independently of the processor 120. In the case of a separate processor, the images processed by the image signal processor 265 may be provided to the processor 120 Or may be displayed through the display device 160 after additional image processing.

According to one embodiment, the electronic device 101 may include two or more camera modules each having a different attribute or function. In this case, for example, at least one camera module may be a wide angle camera or a front camera, and at least one other camera module may be a telephoto camera or a rear camera.

According to various embodiments, the camera module 200 may generate image data based on first data and second data output from each photodiode 234. For example, the camera module 200 generates luminance data (e.g., y data) by summing the first data and the second data, and generates first color data (e.g., U data or Cb data ), And generate second color difference data (e.g., V data or Cr data) based on the second data. The camera module 200 may generate the image data by summing the generated luminance data, the first color difference data, and the second color difference data. The operation in which the camera module 200 generates image data based on the first data and the second data will be described later in more detail with reference to FIGS. 6A through 6C.

According to various embodiments, the image data generation process described above may be performed by the image signal processor 265 of the image sensor IC 260 of the camera module 200. According to other embodiments, the image sensor IC 260 only functions to transmit the data output from the photodiode 234 to a processor (e.g., processor 1030 of FIG. 10) of an electronic device, The data generation process may be performed by a processor of the electronic device. According to still other embodiments, the image sensor IC 260 provides the data output from the photodiode 234 to a separate companion chip (e. G., The second processor 1150 of FIG. 11) provided in the electronic device, The image data generation process described above may be performed by the companion chip. According to still another embodiment, the image data generation process described above may be performed on an external device other than the electronic device (e.g., the electronic device 1000 of FIG. 10, the electronic device 1100 of FIG. 11) on which the camera module 200 is mounted For example, a server). That is, there is no limitation to the image signal processor that performs the image data generation process described in the present specification. The above-described embodiment will be described in more detail with reference to FIGS. 10 and 11. FIG.

According to various embodiments, the camera module 200 (e.g., image sensor IC 260) may adjust the white balance of the image data based on the first data and the second data. The camera module 200 can sample the center wavelength of the data obtained by the photodiode 234 and estimate the color temperature of the light source based on the sampled value. According to various embodiments, since the first data and the second data may be complementary to each other (e.g., the first data is RGB data and the second data is CMY data), the camera module 200 can simultaneously output six bands So that the accuracy of estimating the color temperature of the light source is improved, and more accurate white balancing can be performed accordingly.

According to various embodiments, the camera module 200 can set the exposure value of the image sensor module 230 by controlling the shutter based on the second data (e.g., CMY data). Since the CMY data has higher sensitivity than the RGB data (e.g., the first data), the camera module 200 sets the exposure value in accordance with the sensitivity of the CMY data, and then synthesizes the RGB data and the CMY data to generate a high dynamic range Can be generated.

According to various embodiments, the camera module 200 may set the exposure value of the image sensor module 230 by controlling the shutter based on the first data (e.g., RGB data). The camera module 200 can obtain the RGB data and the CMY data through the structure for determining the exposure value according to the first data, storing the second data in the memory 250 in such a manner that the output of the second data is not saturated, have.

Figure 3 illustrates a unit pixel according to various embodiments.

FIG. 3 shows a unit pixel in a case where the floating diffusion region is not activated, or a case where a blocking film is not formed between the photodiodes, which is not divided into the first region and the second region, for example.

As shown, the image sensor module (e.g., image sensor module 230 of FIG. 2) may include a plurality of photodiodes 310 (e.g., photodiode 234 of FIG. 2). A microlens (e.g., microlens 232 in FIG. 2) is disposed in each of the plurality of photodiodes 310 and a filter array (e.g., filter array 236 in FIG. 2) is disposed between the photodiodes and the microlenses . Each photodiode and filter array can constitute at least one pixel.

As shown in the figure, the unit pixel is composed of four photodiodes 311, 312, 313 and 314, and the R-band filter and the G-band filter corresponding to the respective photodiodes 311, 312, 313 and 314, , And a filter of the B band can be arranged. Accordingly, each of the photodiodes 311, 312, 313, and 314 can output an R signal, a G signal, or a B signal.

Figure 4 illustrates a lens module, a microlens, and a photodiode according to various embodiments.

4, the lens 410 may be any one of a plurality of lenses 410 included in a lens module (for example, the lens module 210 of FIG. 2), and light passing through the lens 410 may be incident on a micro lens 420 to the photodiode 430. Here, the photodiode 430 may correspond to any one of the subpixels 311, 312, 313, and 314 shown in FIG.

According to various embodiments, the photodiode 430 may be divided into two or more regions (e.g., a first region 431 and a second region 432). According to one embodiment, a floating diffusion region 435 may be formed in the middle of the photodiode 430 and the floating diffusion region 435 may be formed by an image sensor IC (e.g., the image sensor IC 260 of FIG. 3) May be activated or deactivated according to the control signal of a processor (e.g., processor 120 of FIG. 1) of the electronic device. According to another embodiment, a blocking film for blocking light may be formed in the center of the photodiode 430. According to another embodiment, the photodiode 430 may include a photodiode corresponding to the first region 431 and another photodiode corresponding to the second region 432. That is, the photodiode 430 may be formed by coupling two photodiodes.

According to various embodiments, a filter array 440 including a first filter 441 and a second filter 442 may be disposed between each photodiode 430 and the microlens 420.

The light in the right region 414 of the lens 410 is refracted through the microlens 410 and passes through the first filter 441 and then passes through the photo- Is input to the first area 431 of the diode 430 and the light in the left area 412 of the lens 410 is refracted through the microlens 410 and passes through the second filter 442, 430 in the second area 432. [

According to various embodiments, the first filter 441 is configured to pass light in a first designated band, and the second filter 442 is configured to pass light in a second specified band in a complementary color relationship with the first designated band Lt; / RTI > For example, the filter array corresponding to the first photodiode may include a first filter 441 for passing light in the R (red) band, a second filter 441 for passing light in the C (cyan) Filter 442 as shown in FIG. The filter array corresponding to the second photodiode (for example, 312 in FIG. 3) and the third photodiode (for example, 313 in FIG. 3) includes a third filter for passing light in the green band, The filter array corresponding to the fourth photodiode (for example, 314 in FIG. 3) includes a fourth filter for passing light in the B (blue) band, 5 filter, and a sixth filter for passing light in the Y (yellow) band, which is in a complementary relationship with B,

FIG. 5 illustrates a segmented region of a photodiode by a floating diffusion region according to various embodiments.

5, the four photodiodes constituting the unit pixel 550 convert RGB data (or first data), RGB data, and CMY data (or second data) of a complementary color relationship into Can be output. In other words, the first photodiode 551 distinguishes and outputs the R data 551a and the C data 551b, and the second photodiode 552 and the third photodiode 553 separate the G data 552a, 553a and 552b and 553b to be read out and the fourth photodiode 554 can read out B data 554a and Y data 554b separately.

The image sensor IC (for example, the image sensor IC 340 in FIG. 3) includes a first region separated from the floating diffusion region in each photodiode 551, 552, 553, and 554, (E.g., RGB data) 551a, 552a, 553a, and 554a and second data (e.g., CMY data) (e.g., 551b, 552b, 553b, and 554b).

Figures 6A-6C illustrate a method for generating image data in accordance with various embodiments.

In the following description, the method of generating the image data shown is performed in the image sensor IC (e.g., the image sensor IC 260 in FIG. 2) of the camera module, but is not limited thereto. For example, according to other embodiments, the image sensor IC of the camera module performs only the function of transmitting the data output from the photodiode to a processor (e.g., processor 120 of FIG. 1) of the electronic device, The process of generating image data may be performed by a processor of the electronic device. According to still another embodiment, the image sensor IC provides the data output from the photodiode to a separate companion chip (or second processor) provided in the electronic device, and the above- Chip. ≪ / RTI > According to still another embodiment, the image data generation process described above may be performed on an external device other than the electronic device (e.g., the electronic device 1000 of FIG. 10, the electronic device 1100 of FIG. 11) on which the camera module 200 is mounted For example, a server).

According to various embodiments, the image signal processor can generate image data including luminance data and color difference data by performing simple calculations on the first data and the second data in a complementary color relationship without further calculation. The image data may be JPEG data, and the JPEG data may be yUV or yCbCr data.

6A shows color data that can be sensed by four photodiodes constituting a unit pixel. 6A illustrates that the floating diffusion region or the shielding film is formed in the middle of the longitudinal direction of each photodiode, but the number and position of the floating diffusion region or the shielding film are not limited thereto.

According to various embodiments, the image signal processor may obtain the first data and the second data output from each photodiode. 6A, the unit pixel data includes first data (e.g., RGB data) 611a, 612a, 613a, and 614a and second data (e.g., CMY data) 611b, 612b, 613b, and 614b .

According to various embodiments, the image signal processor may sum the first data and the second data to produce the luminance data 620. [ For example, the image signal processor generates the luminance data (y1) 621 of the first sub-pixel by summing the R data 611a and the C data 611b of the first photodiode, The luminance data (y2) 622 of the second sub-pixel is generated by summing the data 612a and the M data 612b, and the G data 613a and the M data 613b of the third photodiode are summed (Y4) 624 of the fourth subpixel by adding the B data 614a and the Y data 614b of the fourth photodiode to generate the luminance data (y3) 623 of the third subpixel and the luminance data can do.

When the color data in the complementary color relationship is summed, the RGB values become mono data having the same value, so that the image signal processor can acquire the luminance data through simple sum calculation without having a separate mono filter.

FIG. 6B shows a process of obtaining color difference data from the first data and the second data.

An image signal processor (e.g., image signal processor 265 of FIG. 2) may generate RGB data 641 of each subpixel from first data 631. In this case, RGB data 641 of each subpixel can be generated by interpolation using RGB data of adjacent subpixels. For example, the R data of the first subpixel, the average value of the G data of the upper / lower / left / right subpixels of the first subpixel and the average value of the G data of the subpixel of the upper left / upper / The RGB data (R ₁ G ₁ B ₁ ) 641a of the first subpixel can be generated as an average value of the data. In the case of the second subpixel, the average value of the R data of the left / right subpixels of the second subpixel, the G data of the second subpixel, the average value of the B data of the upper and lower subpixels of the second subpixel To generate RGB data (R ₂ G ₂ B ₂ ) 641b of the second subpixel.

In addition, the image signal processor may generate CMY data 642 for each subpixel from the second data 632. The method of generating CMY data of each subpixel may be the same as the method of generating RGB data of each subpixel. For example, the C data of the first subpixel, the average value of the M data of the upper / lower / left / right subpixels of the first subpixel and the average value of the Y data of the upper left / upper / The CMY data (C ₁ M ₁ Y ₁ ) 642a of the first subpixel can be generated as an average value of the data. In the case of the second subpixel, the average value of the C data of the left / right subpixels of the second subpixel, the M data of the second subpixel, the average value of the Y data of the upper / (C ₂ M ₂ Y ₂ ) 642 of the second subpixel can be generated.

The image signal processor can generate the first color difference data 651 and 652 based on the RGB data of the obtained unit pixel. Here, the first color difference data may include U data (or Cb data) 651 and V data (or Cr data) 652. The U data 651 can be obtained through the difference between the G value and the R value in each of the subpixels 641a, 641b, 641c, and 641d of the RGB data 641. [ For example, the image signal processor calculates the difference (G ₁ -R ₁ ) between the G value and the R value of the first subpixel 641 a of the RGB data 641 to obtain the U data 651 a of the first subpixel And obtain the U data 651b of the second subpixel by calculating the difference (G ₂ -R ₂ ) between the G value and the R value of the second subpixel 641b. The V data 652 can be obtained through the difference between the G value and the B value in each of the subpixels 641a, 641b, 641c, and 641d of the RGB data 641. [ For example, the image signal processor calculates the difference (G ₁ -B ₁ ) between the G value and the B value of the first subpixel 641a of the RGB data 641 to obtain the V data 652a of the first subpixel And obtains the V data 652b of the second subpixel by calculating the difference (G ₂ -B ₂ ) between the G value and the B value of the second subpixel 641b.

The image signal processor may generate the second color difference data 653 and 654 based on the CMY data 642 of the obtained unit pixel. Here, the second color difference data may include U data (or Cb data) 653 and V data (or Cr data) The U data 653 may be obtained through the difference between the C value and the M value at each of the subpixels 642a, 632b, 642c, and 642d of the CMY data 642. [ For example, the image signal processor is a first sub-pixel difference value of C and M values of (642a) (C ₁ -M ₁₎ a data U (653a) of the calculations to the first sub-pixel in the CMY data 642 And obtain the U data 653b of the second subpixel by calculating the difference (C ₂ -M ₂ ) between the C value and the M value of the second subpixel 642b. The V data 654 may also be obtained through the difference between the Y value and the M value at each of the subpixels 642a, 632b, 642c, and 642d of the CMY data 642. [ For example, the image signal processor may calculate the difference (Y ₁ -M ₁ ) between the Y value and the M value of the first subpixel 642a of the CMY data 642 to obtain the V data 654a of the first subpixel acquired, and the second is able to obtain sub-pixel difference between the Y value and the M value of _{(642b) (Y 2 -M 2} ) the V data (654b) of the second sub-pixel is calculated.

The image signal processor may sum U data 651 of the first color difference data and U data 653 of the second color difference data. As described above, the U data 651 of the first color difference data is a difference between the G value and the R value of each subpixel of the RGB data 641, and the U data 653 of the second color difference data is CMY data ( 642) is obtained by subtracting the C value and the M value of each subpixel. The first color difference data and the U data of the second color difference data are calculated as C = G + B / M = R + B and CM = (G + B) - (R + B) And may be the same color component.

Likewise, the image signal processor may sum the V data 652 of the first color difference data and the V data 654 of the second color difference data. The V data 652 of the first color difference data is obtained by finding the difference between the G value and the B value of each sub pixel of the RGB data 641 and the V data 654 of the second color difference data is the difference The difference between the Y value and the M value of the subpixel is obtained. (R + G) - (R + B) = GB, Y = R + G / M = R + B and YM = May be the same color component.

FIG. 6C shows a process of acquiring image data by summing luminance data acquired through the process of FIG. 6A and color difference data acquired through the process of FIG. 6B.

The image signal processor may generate the image data 670 by summing the calculated luminance data 620 and the first color difference data 661 and the second color difference data 662. As shown, the image data 670 may be yUV data (or yCbCr data).

According to various embodiments, the image signal processor obtains the same result as an existing image sensor having only one pixel under the microlens through the process as described in FIGS. 6A to 6C. The image signal processor may output the luminance data and color difference data generated as described above, and may output Bayer data by forming Bayer pattern. In this case, since the processing stage after the image signal processor can maintain the existing method, the processor of the electronic device that obtains the image data can separately obtain the image data without changing the design or performing additional calculation.

Figure 7 shows the wavelength range of image data according to various embodiments.

According to various embodiments, an image signal processor (image signal processor 265 of FIG. 2) may obtain RGB data (or first data) and CMY data (second data) from photodiodes of a unit pixel. The left part of FIG. 7 shows the wavelength range of the RGB data, and the right part shows the wavelength range of the CMY data.

As shown in the figure, since the wavelength range of each component of the CMY data is wider than the wavelength range of each component of the RGB data, the sensitivity of the CMY data can be about two times higher. Considering this sensitivity difference, camera modules can be designed in various ways.

In the first embodiment, it is possible to design the image sensor such that the transmittance of the filter that transmits RGB and the filter that transmits CMY are the same, and the sensitivity difference is doubled. In this embodiment, since the CMY is saturated first in the bright scene, the camera module can adjust the auto exposure according to the CMY data. In this case, since RGB data is half the light amount of CMY data, synthesizing RGB data and CMY data can realize HDR image of 1stop and adjust the automatic exposure according to CMY data of high sensitivity. Since the shutter speed can be driven at half the speed of the camera module, there is an advantage that the synthesis error is small in the synthesis of the moving subject. In addition, since the amount of light is small in a dark scene, automatic exposure can be adjusted around RGB data. In this case, since the sensitivity of CMY data is good, a dark part can be realized on the image data.

In the second embodiment, the image sensor can be designed so that the transmittance of the filter transmitting CMY is adjusted to be equal to the transmit sensitivity of RGB. In this case, there is an advantage in white balancing, and it is possible to guarantee the function of the existing camera module as it is.

As a third embodiment, it is possible to design a storage diode to be applied to a global shutter in a CMY pixel so that CMY is read at once and stored in a storage diode while controlling automatic exposure based on RGB. Even in this case, there is an advantage that the function of the existing camera module can be assured as it is.

According to various embodiments, the image signal processor may adjust the white balance of the image data based on the first data and the second data. The camera module can sample the light centered at the wavelength of the data obtained at the photodiode and estimate the color temperature of the light source based on the sampled value. According to various embodiments, the RGB data and the CMY data are complementary to each other and include three wavelength bands, so that the image signal processor can sample a total of six bands. Accordingly, as a result of using six bands when performing white balancing, the accuracy of color temperature estimation of the light source can be improved, and more accurate white balancing is possible.

Figure 8 illustrates a segmented region of a photodiode by a floating diffusion region or blocking film in accordance with various embodiments.

According to various embodiments, the photodiode can be divided into a plurality of regions by a floating diffusion region or a shielding film. Hereinafter, the photodiode is divided into a plurality of regions through the floating diffusion region.

According to various embodiments, the photodiode may be divided into four regions (e.g., 811a, 811b, 811c, 811d) through the first floating diffusion region and the second floating diffusion region. According to the present embodiment, the filter array corresponding to each photodiode may include first to fourth filters corresponding to the first to fourth regions of the photodiode, respectively. For example, a first filter and a third filter that respectively pass an R band are disposed in a first area 811a and a third area 811c of a first photodiode constituting a unit pixel, and a second area 811b And the fourth region 811d may be provided with a second filter and a fourth filter that pass the C band, respectively. Similarly, the first and second filters 812a and 812b are disposed in the first and second regions 812a and 812c, respectively. The second and fourth filters 812b and 812d ) May be arranged with a second filter and a fourth filter passing through the M band, respectively.

Each of the photodiodes can independently lead out the first to fourth data 820, 830, 840 and 850 obtained in the first to fourth regions, whereby the image signal processor outputs 4 data can be generated through the output first data and fourth data 820, 830, 840, and 850. [ The image signal processor can generate luminance data and color difference data and generate image data through the method described above with reference to Figs. 6A to 6C based on the obtained data.

According to the present embodiment, since the photodiode can be divided into four regions through two floating diffusion regions and phase AF (auto focusing) can be performed in both the horizontal and vertical directions, it is possible to improve the existing AF performance It is effective.

Figure 9 shows a segmented area of a photodiode by a floating diffusion region according to various embodiments.

According to various embodiments, the photodiode may be divided into four regions (e.g., 911a, 911b, 911c, 911d) through a first floating diffusion region and a second floating diffusion region, May include first to fourth filters corresponding to the first to fourth regions of the photodiode, respectively.

In this embodiment, some filters of at least one photodiode may be configured to pass light outside the visible band. For example, a filter array corresponding to at least one photodiode may have a third specified band (e.g., ultraviolet (UV)) that is higher than a first designated band (e.g., R band) and a fourth filter for passing light of a fourth designated band (e.g., an infra-red ray band) lower than the first designated band and the second specified band, .

9, a filter for passing light in the G band is disposed in the first region 913a and a third region 913c in the third sub-pixel, and light in the UV band is disposed in the second region 913b. And a fourth filter for passing light in the IR band may be disposed in the fourth region 913d.

In this case, the third data 940 output from the photodiode may include UV data 940c in the third subpixel and the fourth data 950 may include IR data 950c in the third subpixel. have.

According to the present embodiment, since the camera module can acquire data of a total of 8 bands in addition to the RGB and CMY, as well as the UV band and the IR band, the performance of white balancing and AF can be further improved.

The camera module 200 according to various embodiments of the present invention includes a lens module 210 including at least one lens and an image sensor module 230 coupled with the lens module 210, Module 230 includes a microlens 232, at least one photodiode 234 and a first filter capable of passing light in a first designated band and a second filter capable of passing light in a second predetermined band, A filter array 236 may be disposed between the microlens 232 and the at least one photodiode 234, including a second filter capable of passing light in a second designated band corresponding to a different color .

According to various embodiments, the at least one photodiode 234 may include a first region for sensing light passing through the first filter and a second region for sensing light passing through the second filter .

According to various embodiments, the at least one photodiode 234 may acquire first data corresponding to light sensed in the first region, acquire second data corresponding to sensed light in the second region, . ≪ / RTI >

According to various embodiments, the image signal processor 265 may be further configured to receive the first data and the second data.

According to various embodiments, the image signal processor 265 may be configured to sum luminance data by summing the first data and the second data.

According to various embodiments, the image signal processor 265 may generate first color difference data based on the first data, generate second color difference data based on the second data, And to generate third color difference data by summing the second color difference data.

According to various embodiments, the image signal processor 265 may be configured to generate image data based on the generated luminance data and the third color difference data.

According to various embodiments, the image signal processor 265 may be configured to transmit the first data and the second data via an interface to a processor external to the camera module 200.

According to various embodiments, the image signal processor 265 may be configured to adjust the white balance of the image data based on the first data and the second data.

According to various embodiments, the filter array 236 may include a third filter capable of passing light of a third specified band higher than the first designated band and the second designated band, And a fourth filter capable of passing light of a fourth specified band lower than the designated band.

According to various embodiments, the image sensor module 230 includes a plurality of photodiodes 234 and a plurality of filter arrays 236 corresponding to each of the plurality of photodiodes 234, Photodiodes 234 and a plurality of filter arrays 236 may be configured to configure at least one pixel.

According to various embodiments, the at least one photodiode 234 may be divided into the first region and the second region as the floating diffusion region is activated, or may be divided into the first region and the second region .

10 is a block diagram of an electronic device according to various embodiments.

As shown, the electronic device 1000 may include a camera module 1010, a display 1020, a processor 1030 and a memory 1040, and if necessary, at least some of the depicted configurations are omitted or replaced Various embodiments of the present invention may be implemented. The electronic device 1000 may further include at least some of the configuration and / or functions of the electronic device 101 of FIG.

Hereinafter, functions of processing image data acquired by the camera module 1010 among various functions of the electronic device 1000 will be described. Descriptions of the technical features described above with reference to FIGS. 1 to 9 will be described It will be omitted.

The display 1020 may be a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) Or the like. Display 1020 may include at least some of the configuration and / or functions of display device 160 of FIG.

Memory 1040 includes a volatile memory (e.g., volatile memory 132 of FIG. 1) and / or a non-volatile memory (e.g., non-volatile memory 134 of FIG. 1) . The memory 1040 may store various instructions that may be executed in the processor 1030. Such instructions may include control commands such as arithmetic and logical operations, data movement, input / output, etc., which may be recognized by the control circuitry. The memory 1040 may also store at least some of the programs 140 of FIG.

Processor 1030 is a configuration capable of performing operations and data processing related to the control and / or communication of each component of electronic device 1000, such that at least some of the configuration and / or functionality of processor 120 of FIG. 1 . ≪ / RTI > The processor 1030 may be electrically connected to each component of the electronic device 1000 (e.g., memory 1040, display 1020, camera module 1010, display 1020, etc.).

The camera module 1010 may include an image sensor module 1012 and an image sensor IC 1014.

The image sensor module 1012 may include a plurality of photodiodes constituting pixels, a microlens arranged corresponding to each photodiode, and a filter array disposed between the photodiodes and the microlenses. The filter array includes a first filter capable of passing light in a first designated band and a second filter capable of passing light in a second designated band corresponding to a color corresponding to the first designated band and another color having a complementary color relationship . ≪ / RTI >

The photodiode is set as a first region and a second region as the floating diffusion region is activated. The photodiode detects light passing through the first filter in the first region to output first data, and passes through the second filter One light can be detected in the second area and the second data can be output.

According to various embodiments, the image sensor IC 1014 may generate image data based on the first data and the second data output from the photodiode. The operation of the image sensor IC 1014 to generate image data has been described above with reference to Figs. 6A to 6C.

According to other embodiments, the image sensor IC 1014 outputs data (e.g., first data and second data) output from the photodiode to the processor 1030, and the processor 1030 is coupled to the image sensor IC 1014 To generate image data. The image sensor IC 1014 and the processor 1030 may be connected through a known interface (e.g., mobile industry processor interface (MIPI)). In addition, the processor 1030 may perform at least some of the operations of the image sensor IC 1014 described above with reference to FIGS. 2-9.

11 is a block diagram of an electronic device according to various embodiments.

As shown, the electronic device 1100 may include a camera module 1110, a display 1120, a first processor 1130, a second processor 1150, and a memory. The camera module 1110, the display 1120 and the memory 1140 may be the same as those of the camera module 1010, the display 1020 and the memory 1040 in Fig. In addition, the first processor 1130 may be the same as the processor 1030 of FIG.

The electronic device 1100 may include a second processor 1150 provided between the camera module 1110 and the first processor 1130. The second processor 1150 may be provided as a separate chip (e.g., a companion chip) capable of performing at least some of the functions of the first processor 1130. According to various embodiments, the image sensor IC 1114 outputs data (e.g., first data and second data) output from the photodiode to a second processor 1150, and the second processor 1150 outputs data It is possible to process the data output from the IC 1114 to generate image data. And the second processor 1150 may output the generated image data to the first processor 1130. [

An electronic device 1000 according to various embodiments of the present invention includes a camera module 1010 and an image signal processor such as a processor 1030 and the camera module 1010 includes a micro lens 232, At least one photodiode 234 capable of converting light passing through the microlens 232 into an electrical signal and at least one photodiode 234 disposed between the microlens 232 and the at least one photodiode 234, A first filter capable of passing light of a designated band and a second filter capable of passing light of a second specified band corresponding to a color corresponding to the first designated band and a different color having a complementary color relationship Array 236, and the image signal processor may be configured to generate image data based on data output from the at least one photodiode 234. [

According to various embodiments, the at least one photodiode 234 may include a first region for sensing light passing through the first filter and a second region for sensing light passing through the second filter .

According to various embodiments, the at least one photodiode 234 may acquire first data corresponding to light sensed in the first region, acquire second data corresponding to sensed light in the second region, And the image signal processor may be configured to generate the image data based on the first data and the second data.

According to various embodiments, the image signal processor may generate luminance data by summing the first data and the second data, generate first color difference data based on the first data, And generating third color difference data by summing the first color difference data and the second color difference data, and generating the second color difference data based on the generated luminance data and the third color difference data, . ≪ / RTI >

According to various embodiments, the image signal processor may include an application processor of the electronic device 1000, an image sensor IC of the camera module 1010, or a companion chip electrically coupled to the application processor and the camera module 1010 It can be configured as any one.

12 is a flowchart of a method of generating image data according to various embodiments.

The illustrated method may be performed by the camera module (e.g., image sensor IC 340) described above with reference to FIG. According to another embodiment, the depicted method may be implemented in an electronic device (e.g., processor 1030) and / or an electronic device (e.g., first processor 1130 and / or second processor 1150) Lt; / RTI >

According to various embodiments, the camera module may include a microlens, a photodiode disposed to correspond to the microlens, and a filter array disposed between the microlens and the photodiode.

In operation 1210, the camera module may generate first data based on light sensed in a first region of the photodiode. For example, the first data may be RGB data.

In operation 1220, the camera module may generate second data based on light sensed in a second region of the photodiode. For example, the second data may be CMY data.

According to various embodiments, the first filter disposed in the first region of the photodiode and the second filter disposed in the second region are configured to pass colors having a complementary relationship with each other, wherein the first data and the second data are And can have a complementary color to each other.

In operation 1230, the camera module may generate image data based on the first data and the second data. Specific embodiments for generating image data based on the first data and the second data have been described above with reference to Figs. 6A to 6C.

13 is a flowchart of a method of generating image data according to various embodiments.

In operation 1310, the camera module may generate the first data based on the sensed light in the first area of the photodiode and generate the second data based on the sensed light in the second area of the photodiode.

In operation 1320, the camera module may sum the first data and the second data to generate luminance data. As described above, since the first data and the second data have a complementary color relationship, when the first data and the second data are summed, the RGB components have the same size, and thus can have a luminance component.

In operation 1330, the camera module may generate the first color difference data based on the first data. Concurrently with operation 1330 and at least in part, in operation 1340, the camera module may generate second color difference data based on the second data.

In operation 1340, the camera module may generate the third color difference data by summing the first color difference data and the second color difference data.

In operation 1350, the camera module may generate the third color difference data by summing the luminance data and the second color difference data.

In operation 1360, the camera module may synthesize luminance data and third color difference data to generate image data.

According to various embodiments of the present invention, there is provided a method of generating image data of a camera module, the camera module comprising: a microlens; at least one photodiode arranged to correspond to the microlens; and at least one photodiode , The method comprising: generating (1210) first data based on light sensed in a first region of the at least one photodiode (1210), performing a second operation on the at least one photodiode Generating (1220) second data based on the sensed light in the first and second regions, generating (1230) image data based on the first data and the second data, The color and the color of the second data may be complementary to each other.

According to various embodiments, the at least one photodiode includes a first region for sensing light passing through the first filter and a second region for sensing light passing through the second filter, A first filter provided corresponding to the first area and capable of passing light in a first designated band and a second filter arranged in a different color having a complementary color relationship with the color corresponding to the first designated band, And a second filter capable of passing light of a corresponding second specified band.

According to various embodiments, the operation of generating the image data may include generating the luminance data by summing the first data and the second data, generating the first color difference data based on the first data, An operation of generating second color difference data based on the second data, an operation of generating third color difference data by summing the first color difference data and the second color difference data, and an operation of adding the generated luminance data and the third color difference data And generating the image data based on the data.

Claims (20)

A camera module comprising:
A lens module including at least one lens; And
And an image sensor module coupled with the lens module,
A micro lens;
At least one photodiode; And
A first filter capable of passing light in a first designated band and a second filter capable of passing light in a second designated band corresponding to a color corresponding to the first designated band and a different color having a complementary color relationship Wherein a filter array is disposed between the microlens and the at least one photodiode.
The method according to claim 1,
Wherein the at least one photodiode comprises a first region for sensing light passing through the first filter and a second region for sensing light passing through the second filter.
3. The method of claim 2,
Wherein the at least one photodiode obtains first data corresponding to light sensed in the first region; And
And to obtain second data corresponding to light sensed in the second region.
The method of claim 3,
And an image signal processor configured to receive the first data and the second data.
5. The method of claim 4,
The image signal processor comprising:
And to generate luminance data by summing the first data and the second data.
6. The method of claim 5,
The image signal processor comprising:
Generating first color difference data based on the first data, generating second color difference data based on the second data,
And to generate third color difference data by summing the first color difference data and the second color difference data.
The method according to claim 6,
And the image signal processor is configured to generate image data based on the generated luminance data and the third color difference data.
5. The method of claim 4,
Wherein the image signal processor is configured to transmit the first data and the second data via an interface to a processor external to the camera module.
5. The method of claim 4,
And the image signal processor is configured to adjust the white balance of the image data based on the first data and the second data.
The method according to claim 1,
Wherein the filter array includes a third filter capable of passing light of a third specified band higher than the first designated band and the second designated band and a third filter capable of passing light of a fourth specified band lower than the first designated band and the second specified band And a fourth filter capable of passing light therethrough.
The method according to claim 1,
The image sensor module includes:
A plurality of photodiodes; And
A plurality of filter arrays corresponding to each of the plurality of photodiodes,
Wherein the plurality of photodiodes and the plurality of filter arrays are configured to configure at least one pixel.
3. The method of claim 2,
Wherein the at least one photodiode comprises:
The floating diffusion region is divided into the first region and the second region according to activation of the floating diffusion region, or
And the first region and the second region through the metal film.
In an electronic device,
A camera module; And
An image signal processor,
The camera module includes:
A micro lens;
At least one photodiode capable of converting light passing through the microlens into an electrical signal; And
A first filter disposed between the microlens and the at least one photodiode and capable of passing light in a first designated band, and a second filter disposed in the other of the first band and the second band, 2 a filter array including a second filter capable of passing light of a specified band,
Wherein the image signal processor is configured to generate image data based on data output from the at least one photodiode.
14. The method of claim 13,
Wherein the at least one photodiode comprises a first region for sensing light passing through the first filter and a second region for sensing light passing through the second filter.
15. The method of claim 14,
Wherein the at least one photodiode is configured to acquire first data corresponding to light sensed in the first region and acquire second data corresponding to sensed light in the second region,
And the image signal processor is configured to generate the image data based on the first data and the second data.
15. The method of claim 14,
The image signal processor comprising:
Summing the first data and the second data to generate luminance data,
Generating first color difference data based on the first data,
Generating second color difference data based on the second data,
Summing the first color difference data and the second color difference data to generate third color difference data, and
And generating the image data based on the generated luminance data and the third color difference data.
14. The method of claim 13,
The image signal processor comprising:
An application processor of the electronic device, an image sensor IC of the camera module, or a companion chip electrically connected to the application processor and the camera module.
A method for generating image data of a camera module,
The camera module includes a microlens, at least one photodiode arranged to correspond to the microlens, and a filter array disposed between the microlens and the at least one photodiode,
The method comprises:
Generating first data based on light sensed in a first region of the at least one photodiode;
Generating second data based on light sensed in a second region of the at least one photodiode;
Generating image data based on the first data and the second data,
Wherein the color of the first data and the color of the second data are complementary to each other.
19. The method of claim 18,
Wherein the at least one photodiode includes a first region for sensing light passing through the first filter and a second region for sensing light passing through the second filter,
Wherein the filter array includes a first filter that is provided to correspond to the first area and is capable of passing light of a first designated band, and a second filter that is provided to correspond to the second area and has a color and a complementary color relationship corresponding to the first designated band And a second filter capable of passing light of a second specified band corresponding to another color having the second color.
19. The method of claim 18,
Wherein the generating of the image data comprises:
Summing the first data and the second data to generate luminance data;
Generating first color difference data based on the first data;
Generating second color difference data based on the second data;
Generating third color difference data by summing the first color difference data and the second color difference data; And
And generating the image data based on the generated luminance data and the third color difference data.

Images