KR20170047150A - Method and device for generating an image - Google Patents

Abstract

A device according to an aspect of the present invention includes: a first camera which generates a first monochrome image; a second camera which generates a first color image; a first electronic circuit which generates a second monochrome image by using the first color image; a second electronic circuit which obtains information indicating a positional relationship between pixels included in the first and the second monochrome image; and a third electronic circuit which generates a third monochrome image based on the first monochrome image, the second monochrome image, and the obtained information. Accordingly, the present invention can generate an image with high quality by removing brightness noise and color noise.

Description

Method and device for generating images. {Method and device for generating an image}

A method and device for generating an image.

As a mobile device such as a smart phone with a camera is popularized, a camera included in a smart phone is increasingly used for shooting a subject, compared to a DSLR or a general digital camera. However, the size or thickness of the camera included in the smartphone is limited by the size or thickness of the smartphone itself. Therefore, a camera included in a smart phone is forced to use a small-sized image sensor and a lens, which causes deterioration of image quality. On the other hand, as demand for thinner and lighter mobile devices has increased in recent years, there is a growing demand for a technology capable of generating high-quality images even by a smart phone having a thin camera.

And to provide a method and device for generating an image. The present invention also provides a computer-readable recording medium on which a program for causing the computer to execute the method is provided. The technical problem to be solved is not limited to the technical problems as described above, and other technical problems may exist.

A device according to one aspect, comprising: a first camera for generating a first monochrome image; A second camera for generating a first color image; A first electronic circuit for generating a second monochrome image using the first color image; A second electronic circuit for obtaining information indicating a positional relationship between the pixels included in the first monochrome image and the pixels included in the second monochrome image; And a third electronic circuit for generating the third monochrome image, the second monochrome image, and the third monochrome image based on the obtained information.

A method of generating an image according to another aspect includes generating a first monochrome image and a first color image; Generating a second monochrome image using the first color image; Obtaining information indicating a positional relationship between the pixels included in the first monochrome image and the pixels included in the second monochrome image; And generating a third monochrome image based on the first monochrome image, the second monochrome image, and the obtained information.

According to another aspect, a computer-readable recording medium includes a recording medium on which a program for causing a computer to execute the above-described method is recorded.

1 is a view for explaining an example of a method of generating an image according to an embodiment.
2 is a configuration diagram showing an example of a device according to an embodiment.
3 is a configuration diagram showing another example of a device according to an embodiment.
4A and 4B are views for explaining an example in which a device according to an embodiment includes a plurality of cameras.
5A and 5B are views for explaining an example of an image sensor included in cameras according to an embodiment.
6 is a block diagram illustrating an example of a processor according to an embodiment.
7 is a diagram for explaining an example in which a first electronic circuit according to an embodiment generates a second monochrome image.
8 is a diagram for explaining an example in which a second electronic circuit according to an embodiment operates.
9 is a configuration diagram showing another example of a processor according to an embodiment.
10 is a flowchart showing an example of a method of generating an image according to an embodiment.
11 is a flowchart illustrating another example of a method of generating an image according to an embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following examples of the present invention are intended only to illustrate the present invention and do not limit or limit the scope of the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements, without departing from the spirit or scope of the present invention. Also, as used herein, the term "part " refers to a hardware component such as software, FPGA or ASIC, and" part " However, "part" is not meant to be limited to software or hardware. "Part" may be configured to reside on an addressable storage medium and may be configured to play back one or more processors. Thus, by way of example, and not limitation, "part (s) " refers to components such as software components, object oriented software components, class components and task components, and processes, Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and "parts " may be combined into a smaller number of components and" parts " or further separated into additional components and "parts ".

1 is a view for explaining an example of a method of generating an image according to an embodiment.

FIG. 1 shows an example in which the user 10 generates an image using the device 100. FIG.

The device 100 may include a plurality of cameras (or camera modules). For example, the device 100 may include a first camera for generating a monochrome image and a second camera for generating a color image. Here, a color image means an image in which a subject is represented through various colors, and a monochrome image means an image in which a subject is expressed through a difference in brightness between black and white.

When a photographing signal is input by the user 10, the device 100 generates the first color image 20 and the first black and white image 30 representing the subject. Thereafter, the device 100 generates the second color image 40 using the first color image 20 and the first monochrome image 30. For example, the device 100 may generate a second monochrome image using the first color image 20. Then, the device 100 compares the first black-and-white image 30 with the second black-and-white image, and displays the positional relationship between the pixels included in the first black-and-white image 30 and the pixels included in the second black- Information can be obtained. Then, the device 100 may generate the third monochrome image based on the first monochrome image 30, the second monochrome image, and the obtained information. Then, the device 100 may generate the second color image 40 using the first color image 20 and the third black and white image.

Generally, a color image produced by a color camera (i.e., a camera including a color image sensor) may have a lower quality than a monochrome image produced by a monochrome camera (i.e., a camera including a monochrome image sensor). In other words, the subject displayed on the color image may be less sharp than the subject displayed on the black and white image. Generally, color images are more sensitive to noise than black and white images.

The device 100 generates the second color image 40 using the first color image 20 generated through the first camera and the first black and white image 30 generated through the second camera. Thus, the device 100 can generate a second color image 40 of higher quality than the first color image 20.

Hereinafter, with reference to Figs. 2 to 3, examples of the device 100 will be described in detail.

2 is a configuration diagram showing an example of a device according to an embodiment.

The device 100a includes a photographing unit 110, an analog signal processing unit 120, a memory 130, a storage / reading control unit 140, a data storage unit 142, a program storage unit 150, a display driving unit 162, A display unit 164, a CPU / DSP 170, and an operation unit 180. [

The overall operation of the device 100a is governed by the CPU / DSP 170. 4 to 11, the CPU / DSP 170 is referred to as a processor. The CPU / DSP 170 controls the operation of each component included in the device 100a such as the lens driving unit 112, the diaphragm driving unit 115, the imaging device control unit 119, the display driving unit 162, the operation unit 180, Lt; / RTI >

The photographing unit 110 is a component for generating an image of an electrical signal from incident light and includes a lens 111, a lens driving unit 112, a diaphragm 113, a diaphragm driving unit 115, an image pickup device 118, And a device control unit 119.

The lens 111 may include a plurality of lenses and a plurality of lenses. When the device 100a includes a plurality of lenses, each lens may be a lens having a different angle of view or a lens having the same angle of view. The position of the lens 111 is adjusted by the lens driving unit 112. The lens driver 112 adjusts the position of the lens 111 according to a control signal provided from the CPU / DSP 170.

The diaphragm 113 is controlled by the diaphragm driver 115 to adjust the amount of light incident on the image sensor 118.

The optical signal transmitted through the lens 111 and the diaphragm 113 reaches the light-receiving surface of the image pickup element 118 and forms an image of the object. The image pickup device 118 may be a CCD (Charge Coupled Device) image sensor or a CIS (Complementary Metal Oxide Semiconductor Image Sensor) for converting an optical signal into an electric signal. The sensitivity of the image pickup element 118 can be adjusted by the image pickup element controller 119. The image pickup element control section 119 can control the image pickup element 118 in accordance with a control signal automatically generated by the video signal inputted in real time or a control signal manually inputted by the user's operation.

The exposure time of the imaging element 118 is adjusted by a shutter (not shown). A shutter (not shown) has a mechanical shutter for moving the screen to adjust the incidence of light, and an electronic shutter for controlling exposure by supplying an electric signal to the imaging element 118.

The analog signal processing unit 120 performs noise reduction processing, gain adjustment, waveform shaping, analog-to-digital conversion processing, and the like on the analog signal supplied from the image pickup element 118.

The signal processed by the analog signal processing unit 120 may be input to the CPU / DSP 170 via the memory 130 or may be input to the CPU / DSP 170 without going through the memory 130. [ Here, the memory 130 operates as the main memory of the device 100a, and temporarily stores information necessary for the CPU / DSP 170 during operation. The program storage unit 130 stores programs such as an operating system and an application system for driving the device 100a.

In addition, the device 100a includes a display unit 164 for displaying its operation state or image information photographed by the device 100a. The display unit 164 can provide visual and / or auditory information to the user. In order to provide visual information, the display unit 164 may be composed of, for example, a liquid crystal display panel (LCD), an organic light emitting display panel, or the like.

In addition, the device 100a may include two or more display units 164, or may be a touch screen capable of recognizing a touch input. For example, the device 100a may include a display unit for displaying a live view image representing an object to be photographed and a display unit for displaying an image representing the state of the device 100a.

The display driver 162 provides a driving signal to the display unit 164.

The CPU / DSP 170 processes the input video signal and controls the components according to the input video signal or the external input signal. The CPU / DSP 170 reduces noise on the input image data and performs a gamma correction, a color filter array interpolation, a color matrix, a color correction, And image signal processing for improving image quality such as color enhancement can be performed. Also, the image data generated by the image signal processing for image quality improvement may be compressed to generate an image file, or the image data may be recovered from the image file. The compression format of the image may be reversible or irreversible. As an example of a proper format, the still image can be converted into a JPEG (Joint Photographic Experts Group) format or a JPEG 2000 format. In addition, when recording a moving image, a moving image file can be generated by compressing a plurality of frames according to the MPEG (Moving Picture Experts Group) standard. The image file can be created according to the Exif (Exchangeable image file format) standard, for example.

The image data output from the CPU / DSP 170 is input to the memory 130 or directly to the storage / read control unit 140. The storage / In the data storage unit 142. The storage / read control unit 140 also reads data relating to an image from the image file stored in the data storage unit 142 and inputs it to the display driver through the memory 130 or another path, Images may be displayed. The data storage unit 142 may be removable or permanently mounted to the device 100a.

In addition, the CPU / DSP 170 can perform blurring processing, color processing, blur processing, edge emphasis processing, image analysis processing, image recognition processing, image effect processing, and the like. The face recognizing process, the scene recognizing process, and the like can be performed by the image recognizing process. In addition, the CPU / DSP 170 can perform display video signal processing for display on the display unit 164. For example, brightness level adjustment, color correction, contrast adjustment, contour enhancement adjustment, screen division processing, character image generation, and image synthesis processing can be performed. The CPU / DSP 170 may be connected to an external monitor to process a predetermined image signal to be displayed on the external monitor, and transmit the processed image data to display the corresponding image on the external monitor.

The CPU / DSP 170 may execute a program stored in the program storage unit 130 or may include a separate module to generate a control signal for controlling autofocus, zoom change, focus change, automatic exposure correction, To the diaphragm driving unit 115, the lens driving unit 112 and the imaging device control unit 119 so that the operations of the components provided in the device 100a such as the shutter and the strobe can be collectively controlled.

The operation unit 180 is a configuration in which a user can input a control signal. The operation unit 180 includes a shutter-release button for inputting a shutter-release signal for exposing the image pickup device 118 to light for a predetermined period of time to take a picture, a power source for inputting a control signal for controlling on- Button, a zoom button for widening the angle of view according to the input or narrowing the angle of view, a mode selection button, and other shooting setting value adjustment buttons. The operation unit 180 may be implemented in any form in which a user can input a control signal, such as a button, a keyboard, a touch pad, a touch screen, a remote controller, or the like.

The sensor 190 may measure the physical quantity or sense the operating state of the device 100a and convert the measured or sensed information into an electrical signal. One example of a sensor 190 that may be included in the device 100a will be described below with reference to FIG. The sensor 190 may further include a control circuit for controlling at least one or more sensors belonging thereto. In some embodiments, the device 100a further includes a processor configured to control the sensor 190, either as part of or separate from the CPU / DSP 170, such that the CPU / DSP 170 is in a sleep state The sensor 190 can be controlled.

It is needless to say that the device 100a shown in Fig. 2 is an example showing the configurations necessary for the photographing to be performed, and the device 100a according to some embodiments is not limited to the device 100a shown in Fig.

Hereinafter, another example of the device 100 will be described in detail with reference to FIG.

3 is a configuration diagram showing another example of a device according to an embodiment.

For example, the device 100b may include all or part of the device 100 shown in Fig. 1 or the device 100a shown in Fig. The device 100b may include one or more processors (e.g., a CPU / DSP or AP (application processor) 210), a communication module 220, a subscriber identification module 224, a memory 230, a sensor module 240, An input device 250, a display 260, an interface 270, an audio module 280, a camera module 291, a power management module 295, a battery 296, an indicator 297, and a motor 298 ).

The processor 210 may control a plurality of hardware or software components connected to the processor 210, for example, by driving an operating system or an application program, and may perform various data processing and calculations. The processor 210 may be implemented with, for example, a system on chip (SoC). According to one embodiment, the processor 210 may further include a graphics processing unit (GPU) and / or an image signal processor. Processor 2010 may include at least some of the components shown in FIG. 3 (e.g., cellular module 221). Processor 210 may load or process instructions or data received from at least one of the other components (e.g., non-volatile memory) into volatile memory and store the various data in non-volatile memory have.

The communication module 220 includes a cellular module 221, a WiFi module 223, a Bluetooth module 225, a GNSS module 227 (e.g., a GPS module, a Glonass module, a Beidou module, or a Galileo module) An NFC module 228, and a radio frequency (RF) module 229.

The cellular module 221 can provide voice calls, video calls, text services, or Internet services, for example, over a communication network. As an example, the cellular module 221 may utilize a subscriber identity module (e.g., a SIM card) 224 to perform the identification and authentication of the device 100b within the communication network. As another example, the cellular module 221 may perform at least some of the functions that the processor 210 may provide. As another example, the cellular module 221 may include a communication processor (CP).

Each of the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228 may include a processor for processing data transmitted and received through the corresponding module, for example. At least some (e.g., two or more) of the cellular module 221, the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228 may be integrated into one integrated chip Lt; / RTI >

The RF module 229 can, for example, send and receive communication signals (e.g., RF signals). The RF module 229 may include, for example, a transceiver, a power amplifier module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. As another example, at least one of the cellular module 221, the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228 may transmit and receive RF signals through separate RF modules .

The subscriber identity module 224 may include, for example, a card containing a subscriber identity module and / or an embedded SIM and may include unique identification information (e.g., an integrated circuit card identifier (ICCID) Subscriber information (e.g., international mobile subscriber identity (IMSI)).

The memory 230 may include, for example, an internal memory 232 or an external memory 234. The built-in memory 232 may be implemented as, for example, a volatile memory (e.g., dynamic RAM, SRAM, or synchronous dynamic RAM), a non-volatile memory Programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash) A hard drive, or a solid state drive (SSD).

The external memory 234 may be a flash drive such as a compact flash (CF), a secure digital (SD), a micro secure digital (SD), a mini secure digital (SD) digital, a multi-media card (MMC), a memory stick, and the like. The external memory 234 may be functionally and / or physically connected to the device 100b via various interfaces.

The sensor module 240 may, for example, measure a physical quantity or sense the operating state of the device 100b to convert the measured or sensed information into an electrical signal. The sensor module 240 includes a gesture sensor 240A, a gyro sensor 240B, an air pressure sensor 240C, a magnetic sensor 240D, an acceleration sensor 240E, a grip sensor 240F, A temperature sensor 240G, a UV sensor 240G, a color sensor 240H (e.g., an RGB (red, green, blue) sensor), a living body sensor 240I, And a sensor 240M. Additionally or alternatively, the sensor module 240 may include, for example, an E-nose sensor, an electromyography sensor, an electroencephalogram sensor, an electrocardiogram sensor, , An infrared (IR) sensor, an iris sensor, and / or a fingerprint sensor. The sensor module 240 may further include a control circuit for controlling at least one or more sensors belonging to the sensor module 240. For example, the device 100b may further include a processor configured to control the sensor module 240, either as part of the processor 210 or separately, so that while the processor 210 is in a sleep state, (240).

The input device 250 may include a touch panel 252, a (digital) pen sensor 254, a key 256, or an ultrasonic input device 258). As the touch panel 252, for example, at least one of an electrostatic type, a pressure sensitive type, an infrared type, and an ultrasonic type can be used. Further, the touch panel 252 may further include a control circuit. The touch panel 252 may further include a tactile layer to provide a tactile response to the user.

(Digital) pen sensor 254 may be part of, for example, a touch panel or may include a separate recognition sheet. Key 256 may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device 258 can sense the ultrasonic wave generated by the input tool through the microphone (e.g., the microphone 288) and confirm the data corresponding to the ultrasonic wave detected.

The display 260 (e.g., display portion 164) may include a panel 262, a hologram device 264, or a projector 266. The panel 262 may be embodied, for example, flexible, transparent, or wearable. The panel 262 may be composed of one module with the touch panel 252. [ The hologram device 264 can display a stereoscopic image in the air using interference of light. The projector 266 can display an image by projecting light onto a screen. The screen may be located, for example, inside or outside the device 100b. According to one embodiment, the display 260 may further comprise control circuitry for controlling the panel 262, the hologram device 264, or the projector 266.

The interface 270 may be implemented using a variety of interfaces including, for example, a high-definition multimedia interface (HDMI) 272, a universal serial bus (USB) 274, an optical interface 276, or a D- ) ≪ / RTI > Additionally or alternatively, the interface 270 may be, for example, a mobile high-definition link (MHL) interface, a secure digital (SD) card / multi-media card (MMC) data association standard interface.

The audio module 280 can, for example, convert sound and electrical signals in both directions. The audio module 280 may process sound information input or output through, for example, a speaker 282, a receiver 284, an earphone 286, a microphone 288, or the like.

The camera module 291 is, for example, a device capable of capturing a still image and a moving image. According to an embodiment, the camera module 291 may include at least one image sensor (e.g., a front sensor or a rear sensor) , Or a flash (e.g., an LED or xenon lamp, etc.). When the camera module 291 includes a plurality of lenses, each lens may be a lens having a different angle of view or a lens having the same angle of view.

The power management module 295 can, for example, manage the power of the device 100b. According to one embodiment, the power management module 295 may include a power management integrated circuit (PMIC), a charger integrated circuit (PWM), or a battery or fuel gauge. The PMIC may have a wired and / or wireless charging scheme. The wireless charging scheme may include, for example, a magnetic resonance scheme, a magnetic induction scheme, or an electromagnetic wave scheme, and may further include an additional circuit for wireless charging, for example, a coil loop, a resonant circuit, have. The battery gauge can measure, for example, the remaining amount of the battery 296, the voltage during charging, the current, or the temperature. The battery 296 may include, for example, a rechargeable battery and / or a solar battery.

Indicator 297 may indicate a particular state of device 100b or a portion thereof (e.g., processor 210), such as a boot state, a message state, or a state of charge. The motor 298 can convert electrical signals to mechanical vibration and can generate vibration, haptic effects, and the like. Although not shown in FIG. 3, the device 100b may include a processing unit (e.g., a GPU) for mobile TV support. The processing unit for supporting the mobile TV can process media data conforming to standards such as digital multimedia broadcasting (DMB), digital video broadcasting (DVB), or media flow ( ^TM ).

Each of the components described in this document may be composed of one or more components, and the name of the component may be changed according to the type of the device. In various embodiments, the device may comprise at least one of the components described herein, and some components may be omitted or further include additional components. In addition, some of the components of the device according to various embodiments may be combined to form an entity, so that the functions of the corresponding components before being combined can be performed in the same manner.

As described above with reference to Figs. 1 to 3, the devices 100, 100a, and 100b may include a plurality of cameras. Hereinafter, an example in which the device 100, 100a, 100b includes a plurality of cameras will be described with reference to Figs. 4A to 5B.

4A and 4B are views for explaining an example in which a device according to an embodiment includes a plurality of cameras.

4A shows an example of a device 1000 including a first camera 1010 and a second camera 1020. In FIG. The device 1000 may correspond to any one of the devices 100a and 100b described above with reference to FIGS. For example, the device 1000 may be a camera, a smart phone, a tablet PC, or a wearable device.

The first camera 1010 and the second camera 1020 respectively generate an image representing a subject. For example, the first camera 1010 may be a camera that generates a monochrome image, and the second camera 1020 may be a camera that generates a color image.

The first camera 1010 and the second camera 1020 may be disposed on the same side of the device 1000 to produce a similar image representing the subject. In other words, the first camera 1010 and the second camera 1020 may be arranged to face the same direction on the device 1000. [ In addition, the first camera 1010 and the second camera 1020 can be disposed at adjacent positions in the device 1000, and thus the image generated by the first camera 1010 and the image generated by the second camera 1020 One image can represent the same subject.

The device 1000 may have previously stored the positional relationship between the first camera 1010 and the second camera 1020 in addition to the specifications of the first camera 1010 and the second camera 1020. In other words, the device 1000 is provided with a relative position between the first camera 1010 and the second camera 1020, a focal length of the lens included in each of the cameras 1010 and 1020, The size of the image sensor included in the image sensor may be stored in advance. In addition, the first camera 1010 and the second camera 1020 may be arranged such that the horizontal axis or the vertical axis is the same.

FIG. 4B shows an example of the configurations included in the camera 1030. FIG. The camera 1030 shown in FIG. 4B may correspond to the first camera 1010 or the second camera 1020 in FIG. 4A. Meanwhile, only the main components included in the camera 1030 are shown in FIG. 4B. Accordingly, it is understood by those skilled in the art that the camera 1030 may further include other configurations necessary for generating an image.

The camera 1030 may be stacked with the image sensor 1034 on the substrate 1035 and the optical filter 1033 may be located on the image sensor 1034. [ For example, the image sensor 1034 may include a complementary metal-oxide semiconductor (CMOS) or a charge coupled device (CCD). The lens 1031 is positioned on the optical filter 1033 and the positions of the image sensor 1034 and the lens 1031 can be fixed by the cover 1032. [

Generally, the thickness T of the camera 1030 is determined by the thickness and position of the configurations 1031, 1032, 1033, 1034, and 1035 described above. In particular, the thickness T of the camera 1030 is determined according to the size and thickness of the lens 1031 and the image sensor 1034. Therefore, in order to reduce the thickness of the device 1000, the size of the lens 1031 and the image sensor 1034 must be reduced and the thickness thereof must be reduced. However, the smaller the size of the lens 1031 and the image sensor 1034 and the thinner the thickness, the lower the quality of the image generated by the camera 1030. [

The device 1000 generates a new image representing a subject using the first image generated by the first camera 1010 and the second image generated by the second camera 1020. [ At this time, the newly generated image may be a higher quality image than the first image and the second image. Therefore, even if the lens 1031 and the image sensor 1034 included in the device 1000 are small in size and thin in thickness, the device 1000 can generate and display a high-quality image.

Meanwhile, the camera 1030 may be a light field camera. The light field camera refers to a camera to which a plurality of photodiodes are connected to each of a plurality of microlenses included in the image sensor 1034. Generally, the image sensor 1034 included in the camera 1030 includes a plurality of microlenses, and one photodiode is connected to each microlens. However, in the case of a light field camera, a plurality of photodiodes are connected to one microlens, so that a plurality of images of a subject are generated, and the viewpoints of the respective images may be different.

On the other hand, assuming that the first camera 1010 generates a monochrome image, the image sensor included in the first camera 1010 may be a monochrome image sensor. Also, assuming that the second camera 1020 is a camera that generates a color image, the image sensor included in the second camera 1020 may be a color image sensor. In this case, the image sensor included in the first camera 1010 may be a sensor having a higher pixel count than the image sensor included in the second camera 1020. In this case, the monochrome image photographed by the first camera 1010 may include a larger number of pixels than the color image photographed by the second camera 1020. [ Meanwhile, the image sensor included in the first camera 1010 and the image sensor included in the second camera 1020 may have the same number of pixels. In this case, the monochrome image photographed by the first camera 1010 and the color image photographed by the second camera 1020 may include the same number of pixels. Hereinafter, an example of the image sensor included in the first camera 1010 and the second camera 1020 will be described with reference to FIGS. 5A to 5B. FIG.

5A and 5B are views for explaining an example of an image sensor included in cameras according to an embodiment.

FIG. 5A shows an example of a monochrome image sensor 1110, and FIG. 5B shows an example of a color image sensor 1120. FIG. The pixel 1111 included in the monochrome image sensor 1110 is used to determine the brightness of white based on the magnitude of the incident light. On the other hand, the pixel 1121 included in the color image sensor 1120 can be predetermined to represent any color of R (red), G (green), or B (blue). Thus, the pixel 1121 is used to determine that the brightness of a predetermined color (e.g., green) based on the magnitude of the incident light is determined.

6 to 11, the device 1000 generates a new image using the first image generated through the first camera 1010 and the second image generated through the second camera 1020, The following description will be given.

6 is a block diagram illustrating an example of a processor according to an embodiment.

Referring to FIG. 6, the processor 1200 includes a first electronic circuit 1210, a second electronic circuit 1220, and a third electronic circuit 1230. The processor 1200 may be implemented as an array of a plurality of logic gates, or may be implemented as a combination of a general purpose microprocessor and a memory in which a program executable in the microprocessor is stored. It will be appreciated by those skilled in the art that the present invention may be implemented in other forms of hardware.

6, the processor 1200 is shown as being divided into a plurality of electronic circuits 1210, 1220, and 1230, but a single electronic circuit may be included in the electronic circuits 1210 and 1220 , 1230).

The first electronic circuit 1210 generates a second monochrome image using the first color image. The first color image refers to a color image generated by the cameras 1010 and 1020 included in the device 1000. For example, assuming that the second camera 1020 included in the device 1000 is a camera for capturing a color image, the first color image refers to an image captured by the second camera 1020. The second monochrome image is not an image captured by the cameras 1010 and 1020 but an image generated by the first electronic circuit 1210 using the first color image. For example, assuming that the first camera 1010 included in the device 1000 is a camera that captures a monochrome image, the first monochrome image is an image photographed by the first camera 1010, and the second monochrome image Means an image newly generated based on the image photographed by the second camera 1020.

The second monochrome image generated by the first electronic circuit 1210 may have the same number of pixels as the first color image and may have the same number of pixels as the first monochrome image. The first electronic circuit 1210 may generate a second monochrome image using a color value of a specific pixel included in the first color image, and may generate a color value of a specific pixel included in the first color image and adjacent pixels May be used to generate the second monochrome image. Here, the color value of the pixel means a value corresponding to the chroma of the pixel. Hereinafter, an example in which the first electronic circuit 1210 generates the second monochrome image will be described with reference to Fig.

7 is a diagram for explaining an example in which a first electronic circuit according to an embodiment generates a second monochrome image.

7, the first electronic circuit 1210 converts the pixel value of the pixel 1321 included in the second monochrome image 1320 to the pixel value of the pixel 1321 included in the second monochrome image 1320 using the pixel value of the pixel 1311 included in the first color image 1310. [ Value can be calculated. Here, the pixel value of the pixel 1311 means the color value of the pixel 1311. [ In addition, the pixel 1321 refers to a pixel corresponding to the pixel 1311 of the first color image 1310 among the pixels included in the second monochrome image 1320. For example, the pixel 1321 may be a pixel representing the same portion as the portion of the subject shown in the pixel 1311. [

The first electronic circuit 1210 uses the color value of the pixel 1311 to obtain information on the color of the pixel 1311 and the brightness value. For example, the first electronic circuit 1210 can obtain the color of the pixel 1311 and the brightness value. The first electronic circuit 1210 uses the brightness value of the pixel 1311 to determine the pixel value of the pixel 1321. [ For example, the first electronic circuit 1210 may determine the pixel value of the pixel 1321 to the same degree as the brightness of the pixel 1311.

On the other hand, the first electronic circuit 1210 calculates the pixel value of the pixel 1321 by taking into consideration not only the pixel value of the pixel 1311 but also the pixel value of each of the pixels 1312 adjacent to the pixel 1311 . The first electronic circuit 1210 may calculate a pixel value of the pixel 1321 by applying a predetermined weight according to the distance between each of the pixels 1312 and the pixel 1311. For example, the first electronic circuit 1210 may calculate the pixel value of the pixel 1321 by applying a smaller weight to the pixel 1321 as the distance from the pixel 1311 increases.

Although FIG. 7 shows eight pixels 1321 adjacent to the pixel 1311, the present invention is not limited thereto. In other words, the adjacent pixels 1321 used to compute the pixel value of the pixel 1321 may correspond not only to the eight pixels that are in contact with the pixel 1311, but also to more pixels. In addition, adjacent pixels 1321 used to calculate the pixel value of the pixel 1321 may correspond to some of the eight pixels that are in contact with the pixel 1311.

Referring again to FIG. 6, the second electronic circuit 1220 obtains information indicating a positional relationship between the pixels included in the first monochrome image and the pixels included in the second monochrome image. For example, the second electronic circuit 1220 may identify a pixel of the first monochrome image corresponding to each pixel included in the second monochrome image. Hereinafter, an example in which the second electronic circuit 1220 operates will be described with reference to FIG.

8 is a diagram for explaining an example in which a second electronic circuit according to an embodiment operates.

Referring to FIG. 8, the second electronic circuit 1220 obtains information indicating the positional relationship between the pixels included in the first black and white image 1410 and the pixels included in the second black and white image 1420. In other words, the second electronic circuit 1220 identifies the pixel 1421 included in the first monochrome image 1410 and the corresponding pixel 1421 in the second monochrome image 1420.

As an example, the second electronic circuit 1220 may use the specifications of the first camera 1010, the specifications of the second camera 1020, and the positional relationship between the first camera 1010 and the second camera 1020 It is possible to identify the pixel 1421 in the second monochrome image 1420. Specifically, the second electronic circuit 1220 includes a distance between the first camera 1010 and the second camera 120, a focal length of the lens of the first camera 1010, a focus of the lens of the second camera 1020, The same information as the information represented by the pixel 1411 in the first black and white image 1410 is displayed in consideration of the distance, the size of the image sensor of the first camera 1010, the size of the image sensor of the second camera 1020, Pixels 1421 may be identified in the second monochrome image 1420.

As another example, the second electronic circuit 1220 may generate a second black image 1420 based on a correlation between pixel values of pixels included in the first monochrome image 1410 and pixel values of pixels included in the second monochrome image 1420 It is possible to identify the pixel 1421 in the second monochrome image 1420. Specifically, the second electronic circuit 1220 can calculate the correlation between the pixel value of the pixel 1411 and the pixel value of each of the pixels included in the second monochrome image 1420. Then, the second electronic circuit 1220 can identify the pixel 1421 having the highest correlation among the pixels included in the second monochrome image 1420.

As another example, the second electronic circuit 1220 may extract the feature points from the first black and white image 1410 and the second black and white image 1420, and may use the matching results of the feature points to identify the pixel 1421 . A technique of extracting a feature point from images and matching the corresponding points by matching the feature points can be easily understood by a person skilled in the art.

Alternatively, the second electronic circuit 1220 may identify pixels 1421 corresponding to pixels 1411 by combining two or more of the above examples. For example, the second electronic circuit 1220 may use the specifications of the first camera 1010, the specifications of the second camera 1020, and the positional relationship between the first camera 1010 and the second camera 1020 By selecting the candidate region 1422 in the second black and white image 1420 and calculating the correlation between the pixel value of each of the pixels included in the candidate region 1422 and the pixel value of the pixel 1411, . ≪ / RTI >

Referring again to FIG. 6, the third electronic circuit 1230 generates a third monochrome image based on the first monochrome image, the second monochrome image, and the information obtained by the second electronic circuit 1220. At this time, the third monochrome image may have the same number of pixels as the first monochrome image, or may have more pixels than the first monochrome image.

For example, the third electronic circuit 1230 may use the information obtained by the second electronic circuit 1220 to generate a pixel (hereinafter, referred to as a 'first pixel') included in the first monochrome image Quot; second pixel ") is searched from the second monochrome image. The third electronic circuit 1230 calculates a pixel value of a pixel included in the third monochrome image (hereinafter, referred to as a 'third pixel') using the pixel value of the first pixel and the pixel value of the second pixel do.

In addition, the third electronic circuit 1230 may further include not only the pixel value of the first pixel and the pixel value of the second pixel but also the pixel value of the pixels adjacent to the first pixel and the pixel value of the pixels adjacent to the second pixel, The pixel value of 3 pixels can be calculated. For example, the third electronic circuit 1230 applies a predetermined weight to the pixel values of the first pixel, the pixel values of the second pixel, and the pixel values of the pixels adjacent to the first pixel and the second pixel, The pixel value can be calculated.

The first and second monochrome images may differ in terms of average brightness, contrast, blur, and degree of noise.

If the first and second monochrome images differ only in the degree of noise, the third electronic circuit 1230 may calculate a patch pi containing the fifth pixel according to Equation (1) .

In Equation 1, pi 'denotes a patch in which pixels adjacent to the first pixel and the first pixel are combined, and pi' 'denotes a patch in which pixels adjacent to the second pixel and the second pixel are combined. If the number of pixels of the first monochrome image and the second monochrome image is different from each other or the number of pixels of the third monochrome image is different from the number of pixels of the first monochrome image or the second monochrome image, May be defined by applying interpolation to pixel values.

In addition, w1 denotes a weight determined according to noise of the first image (or patch pi '), and w2 denotes a weight determined according to noise of the second image (or patch pi' '). Further, a relationship of w1 + w2 = 1 is established between w1 and w2. If the patch pi 'and the patch pi' 'do not match due to the parallax between the first camera 1010 and the second camera 1020 and the subject, the third electronic circuit 1230 sets w1 to 0 or 0 Apply close values. Therefore, when pi is computed, the specific gravity of pi 'may be lost or its specific gravity may be smaller than pi' '.

As another example, when the first and second monochrome images and the second monochrome image differ in not only the degree of noise but also the average brightness and contrast, the third electronic circuit 1230 may calculate the following equation (2) The pixel pi can be computed according to the following equation.

In equation (2), ai and bi may be scalar values, which are variables defined to make the average brightness and contrast of patch pi " equal to patch pi '. For example, the third electronic circuit 1230 may calculate ai and bi according to the following equations (3) and (4).

는 패치 pi'에 포함된 픽셀들의 픽셀 값들의 평균을 의미하고,

는 패치 pi'에 포함된 픽셀들의 픽셀 값들의 표준 편차를 의미한다. 또한,

는 패치 pi''에 포함된 픽셀들의 픽셀 값들의 평균을 의미하고,

는 패치 pi''에 포함된 픽셀들의 픽셀 값들의 표준 편차를 의미한다.In Equations (3) and (4)

Means an average of pixel values of pixels included in patch pi '

Denotes the standard deviation of the pixel values of the pixels included in the patch pi '. Also,

Quot; means an average of pixel values of pixels included in the patch pi ", and "

Denotes the standard deviation of the pixel values of the pixels included in the patch pi ".

According to the above description, the third electronic circuit 1230 can generate a third monochrome image of higher quality than the first monochrome image.

9 is a configuration diagram showing another example of a processor according to an embodiment.

9, the processor 1201 includes a first electronic circuit 1210, a second electronic circuit 1220, a third electronic circuit 1230, and a fourth electronic circuit 1240. The processor 1201 may be implemented as an array of a plurality of logic gates, or may be implemented as a combination of a general purpose microprocessor and a memory in which a program executable in the microprocessor is stored. It will be appreciated by those skilled in the art that the present invention may be implemented in other forms of hardware.

9, the processor 1201 is shown as being divided into a plurality of electronic circuits 1210, 1220, 1230, and 1240, but a single electronic circuit may be used as the electronic circuits 1210 , 1220, 1230, 1240). The functions of the first electronic circuit 1210, the second electronic circuit 1220, and the third electronic circuit 1230 are as described above with reference to FIGS. Therefore, the detailed description of the first electronic circuit 1210, the second electronic circuit 1220, and the third electronic circuit 1230 will be omitted below.

The fourth electronic circuit 1240 generates the second color image using the first color image and the third monochrome image. Here, the number of pixels included in the second color image may be equal to the number of pixels included in the third monochrome image.

The fourth electronic circuit 1240 may calculate the color value of each of the pixels included in the third monochrome image using the color values of the pixels included in the first color image. For example, the fourth electronic circuit 1240 may combine at least one weight value with a color value of a pixel included in the first color image (hereinafter, referred to as a 'fourth pixel') to be included in the second color image (Hereinafter, referred to as " fifth pixel "). Here, at least one weight value may be determined based on the first color image or the third monochrome image. Further, the fifth pixel means a pixel corresponding to the fourth pixel among the pixels included in the second color image.

For example, the fourth electronic circuit 1240 may calculate the color value of the fifth pixel based on Equation (5) below.

로 정의될 수 있다.In Equation (5), ci 'denotes a chrominance component of the fifth pixel (i.e., the i-th pixel among the pixels included in the second color image). For example, ci 'may mean a saturation component of any one of R (red), G (green), and B (blue). Also, N (i) means a set of pixels adjacent to the fifth pixel. The number of pixels included in N (i) may be predetermined. Further, wi means a normalization term,

. ≪ / RTI >

Further, cj means a chrominance component of the fourth pixel (i.e., the jth pixel among the pixels included in the first color image). If the size of the first color image and the size of the second color image or the number of pixels included in the image are different, cj is not the jth pixel of the first color image but the first color image corresponding to the jth pixel of the second color image Lt; RTI ID = 0.0 >

는 픽셀들 사이의 거리에 대한 가중치를 의미하고, 예를 들어 아래의 수학식 6에 의하여 연산될 수 있다.

Is a weight for a distance between pixels, and can be calculated, for example, by the following equation (6).

는 영상의 픽셀들 사이의 거리에 따른 가중치의 변화를 조절하는 파라미터이며, 픽셀들 사이의 거리에 따라 미리 결정되어 있을 수 있다.In Equation (6), d (i, j) denotes the distance between the i-th pixel and the j-th pixel of the image and may be, for example, an Euclidean distance between the i- have. Also,

Is a parameter for adjusting the change in the weight according to the distance between the pixels of the image, and may be predetermined according to the distance between the pixels.

는 제 3 흑백 영상에 기초한 가중치를 의미하고, 예를 들어 아래의 수학식 7에 의하여 연산될 수 있다.

Means a weight based on the third monochrome image, and can be calculated by Equation (7), for example.

는 제 3 흑백 영상에 포함된 픽셀들의 픽셀 값의 차이에 따른 가중치의 변화를 조절하는 파라미터이며, 픽셀 값의 차이에 따라 미리 결정되어 있을 수 있다.

는, 제 3 흑백 영상의 i번째 픽셀의 픽셀 값과 j번째 픽셀의 픽셀 값의 차이가 작을 경우, i번째 픽셀 및 j번째 픽셀이 포함된 평면은 색 변화가 작은 평면으로 간주되어 i번째 픽셀의 컬러 값과 및 j번째 픽셀의 컬러 값을 비슷하게 하는 역할을 한다. 한편,

는, 제 3 흑백 영상의 i번째 픽셀의 픽셀 값과 j번째 픽셀의 픽셀 값의 차이가 클 경우, i번째 픽셀과 j번째 픽셀의 컬러 차이를 유지하게 하는 역할을 한다.In Equation (7), mi denotes the pixel value of the i-th pixel of the third monochrome image, and mj denotes the pixel value of the j-th pixel of the third monochrome image. Also,

Is a parameter for adjusting a change in weight according to a difference between pixel values of pixels included in the third monochrome image, and may be predetermined according to a difference between pixel values.

When the difference between the pixel value of the i-th pixel of the third monochrome image and the pixel value of the j-th pixel is small, the plane containing the i-th pixel and the j- The color value and the color value of the j-th pixel are similar to each other. Meanwhile,

Th pixel and the jth pixel when the difference between the pixel value of the i-th pixel and the pixel value of the j-th pixel is large, the color difference between the i-th pixel and the j-th pixel is maintained.

는 제 1 컬러 영상에 기초한 가중치를 의미하고, 예를 들어 아래의 수학식 8 또는 수학식 9에 의하여 연산될 수 있다.

Means a weight based on the first color image, and can be calculated, for example, by the following equation (8) or (9).

를 의미한다. 또한,

는 제 1 컬러 영상에 포함된 픽셀들의 픽셀 값의 차이에 따른 가중치의 변화를 조절하는 파라미터이며, 픽셀 값의 차이에 따라 미리 결정되어 있을 수 있다.Equation (8) represents the case where both ci and cj exist

. Also,

Is a parameter for adjusting a change in weight according to a difference between pixel values of pixels included in the first color image, and may be predetermined according to a difference in pixel values.

를 의미한다. 도 4b를 참조하여 상술한 바와 같이, 카메라(1030)의 영상 센서(1034)의 상부에는 광학 필터(1033)가 위치한다. 만약, 카메라(1030)가 컬러 카메라인 경우, 광학 필터(1033)는 컬러 필터에 해당되고, 영상 센서(1034)는 컬러 영상 센서에 해당된다. 따라서, 영상 센서(1034)에 포함된 픽셀들 각각에는 특정한 컬러 성분이 존재할 수도 있으나, 존재하지 않을 수도 있다. 예를 들어, 영상 센서(1034)의 특정 픽셀이 R(red)을 나타내는 픽셀이라고 가정하면, 그 픽셀에 대응하는 컬러 성분에는 R(red)이 존재할 수도 있으나, 그렇지 않을 수도 있다. 따라서, ci와 cj 중 어느 하나라도 존재하지 않는 경우, 수학식 8에 따른 연산은 불가능하며,

는 미리 정해진 상수인 C로 결정될 수 있다.Equation (9) represents the case where neither ci nor cj exists

. As described above with reference to Fig. 4B, an optical filter 1033 is positioned above the image sensor 1034 of the camera 1030. Fig. If the camera 1030 is a color camera, the optical filter 1033 corresponds to a color filter, and the image sensor 1034 corresponds to a color image sensor. Thus, each of the pixels included in the image sensor 1034 may have a specific color component, but may not exist. For example, assuming that a particular pixel of the image sensor 1034 is a pixel representing R (red), R (red) may be present in the color component corresponding to that pixel, but it may not. Therefore, when either ci or cj is not present, the calculation according to Equation (8) is impossible,

Can be determined to be a predetermined constant C.

According to the above description, the fourth electronic circuit 1240 can generate the second color image of higher quality than the first color image by using the first color image and the third monochrome image.

10 is a flowchart showing an example of a method of generating an image according to an embodiment.

Referring to FIG. 10, a method of generating an image is performed in a time-series manner in the devices 100, 100a, 100b, and 1000 or the processors 1200 and 1201 shown in FIGS. 1 to 4, 6 and 9 . Therefore, the contents described above with respect to the devices 100, 100a, 100b, and 1000 or the processors 1200 and 1201 shown in Figs. 1 to 4, 6 and 9, The method of generating the image of FIG.

In step 2110, the devices 100, 100a, 100b, and 1000 capture the first black-and-white image and the first color image. For example, the device 100 may include a first camera for generating a first monochrome image and a second camera for generating a first color image. Also, the first camera and the second camera may be light field cameras. A light field camera means a camera in which a plurality of photodiodes are connected to each of a plurality of microlens included in an image sensor.

In step 2120, the first electronic circuit 1210 generates a second monochrome image using the first color image. The second monochrome image generated by the first electronic circuit 1210 may have the same number of pixels as the first color image and may have the same number of pixels as the first monochrome image.

In step 2130, the second electronic circuit 1220 obtains information indicating a positional relationship between the pixels included in the first monochrome image and the pixels included in the second monochrome image. In other words, the second electronic circuit 1220 identifies, in the second monochrome image, the pixel 1421 corresponding to each of the pixels included in the first monochrome image.

In step 2140, the third electronic circuit 1230 generates a third monochrome image based on the first monochrome image, the second monochrome image, and the information obtained by the second electronic circuit 1220. At this time, the third monochrome image may have the same number of pixels as the first monochrome image, or may have more pixels than the first monochrome image.

11 is a flowchart illustrating another example of a method of generating an image according to an embodiment.

Referring to FIG. 11, a method of generating an image is performed in a time-series manner in the devices 100, 100a, 100b, and 1000 or the processors 1200 and 1201 shown in FIGS. 1 to 4, 6 and 9 . Therefore, the contents described above with respect to the devices 100, 100a, 100b, and 1000 or the processors 1200 and 1201 shown in Figs. 1 to 4, 6 and 9, The method of generating the image of FIG.

In addition, steps 2210 to 2240 of FIG. 11 are the same as steps 2110 to 2140 of FIG. Therefore, detailed description of steps 2210 to 2240 will be omitted.

In step 2250, the fourth electronic circuit 1240 generates the second color image using the first color image and the third monochrome image. Here, the number of pixels included in the second color image may be equal to the number of pixels included in the third monochrome image.

As described above, the device can simultaneously use a color camera and a monochrome camera to generate a high-quality image by removing both brightness noise and color noise.

Meanwhile, the above-described method can be implemented in a general-purpose digital computer that can be created as a program that can be executed by a computer and operates the program using a computer-readable recording medium. In addition, the structure of the data used in the above-described method can be recorded on a computer-readable recording medium through various means. The computer-readable recording medium includes a storage medium such as a magnetic storage medium (e.g., ROM, RAM, USB, floppy disk, hard disk, etc.), optical reading medium (e.g. CD ROM, do.

Further, the above-described method may be performed through execution of instructions contained in at least one of programs stored in a computer-readable recording medium. When the instruction is executed by a computer, the at least one computer may perform a function corresponding to the instruction. Here, the instructions may include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like. In the present disclosure, an example of a computer may be a processor, and an example of a recording medium may be a memory.

It will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the essential characteristics thereof. Therefore, the disclosed methods should be considered from an illustrative point of view, not from a restrictive point of view. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

10: User
20: First color image
30: First black and white image
40: Second color image
100: User

Claims (19)

A first camera for generating a first monochrome image;
A second camera for generating a first color image;
A first electronic circuit for generating a second monochrome image using the first color image;
A second electronic circuit for obtaining information indicating a positional relationship between the pixels included in the first monochrome image and the pixels included in the second monochrome image; And
A third monochrome image, a second monochrome image, and a third electronic circuit for generating a third monochrome image based on the obtained information. The method according to claim 1,
Wherein the number of pixels of the sensor included in the first camera is greater than or equal to the number of pixels of the sensor included in the second camera. The method according to claim 1,
The first electronic circuit
Calculating a pixel value of a second pixel included in the second monochrome image using the pixel value of the first pixel included in the first color image,
Wherein the second pixel is a pixel corresponding to the first pixel among the pixels included in the second monochrome image. The method of claim 3,
The first electronic circuit
And calculating a pixel value of the second pixel by further considering the pixel value of each of the pixels adjacent to the first pixel. The method according to claim 1,
The third electronic circuit
Searching the fourth pixel included in the second monochrome image corresponding to the third pixel included in the first monochrome image using the obtained information,
And calculates a pixel value of a fifth pixel included in the third monochrome image using the pixel value of the third pixel and the pixel value of the fourth pixel. 6. The method of claim 5,
The third electronic circuit
The pixel value of each of the pixels adjacent to the third pixel and the pixel value of each of the pixels adjacent to the fourth pixel. The method according to claim 1,
And a fourth electronic circuit for generating a second color image using the first color image and the third black and white image. 8. The method of claim 7,
The fourth electronic circuit
Combining the color value of the sixth pixel included in the first color image and at least one weight value based on the first color image or the third monochrome image to obtain a color value of the seventh pixel included in the second color image And,
Wherein the seventh pixel is a pixel corresponding to the sixth pixel among the pixels included in the second color image. The method according to claim 1,
Wherein the first camera and the second camera comprise a light field camera. Generating a first monochrome image and a first color image;
Generating a second monochrome image using the first color image;
Obtaining information indicating a positional relationship between the pixels included in the first monochrome image and the pixels included in the second monochrome image; And
And generating a third monochrome image based on the first monochrome image, the second monochrome image, and the obtained information. 11. The method of claim 10,
Wherein the number of pixels included in the first monochrome image is greater than or equal to the number of pixels included in the first color image. 11. The method of claim 10,
The step of generating the second monochrome image
Calculating a pixel value of a second pixel included in the second monochrome image using the pixel value of the first pixel included in the first color image,
Wherein the second pixel is a pixel corresponding to the first pixel among the pixels included in the second monochrome image. 13. The method of claim 12,
The step of generating the second monochrome image
And calculating a pixel value of the second pixel by further considering the pixel value of each of the pixels adjacent to the first pixel. 11. The method of claim 10,
The step of generating the third monochrome image
Searching for a fourth pixel included in a second monochrome image corresponding to a third pixel included in the first monochrome image using the obtained information; And
Computing a pixel value of a fifth pixel included in the third monochrome image using the pixel value of the third pixel and the pixel value of the fourth pixel. 15. The method of claim 14,
The step of calculating the pixel value of the fifth pixel
Calculating a pixel value of the fifth pixel by considering the pixel value of each of the pixels adjacent to the third pixel and the pixel value of each of the pixels adjacent to the fourth pixel. 11. The method of claim 10,
And generating a second color image using the first color image and the third black and white image. 17. The method of claim 16,
The step of generating the second color image
Combining the color value of the sixth pixel included in the first color image and at least one weight value based on the first color image or the third monochrome image to obtain a color value of the seventh pixel included in the second color image And,
Wherein the seventh pixel is a pixel corresponding to the sixth pixel among the pixels included in the second color image. 11. The method of claim 10,
A first camera for generating the first monochrome image and a second camera for generating the first color image comprise a light field camera. A computer-readable recording medium recording a program for causing a computer to execute the method of claim 10.

Images