WO2020196934A1

WO2020196934A1 - Image sensor and control method therefor

Info

Publication number: WO2020196934A1
Application number: PCT/KR2019/003372
Authority: WO
Inventors: 김정기; 황정환
Original assignee: 엘지전자 주식회사
Priority date: 2019-03-22
Filing date: 2019-03-22
Publication date: 2020-10-01

Abstract

The present invention relates to an image sensor and a control method therefor, whereby: pixel data is divided into n x m regions (here, n and m are natural numbers); an image synthesis unit is controlled so that same generates binning data by means of, in a specific region including at least some of the divided regions, applying binning differently according to the attribute of the specific region; and new pixel data is generated by super resolution image processing the binning data outputted in the specific region.

Description

Image sensor and its control method

The present invention relates to an image sensor and a method for controlling the same, and to an image sensor and a method for controlling the same, which improves the brightness of a dark original image by applying binning to a specific area of the entire image and provides a clearer image.

An image sensor is a semiconductor that converts photons into electrons and allows them to be displayed on a display or stored in a storage device. A light-receiving element that converts a light-receiving signal into an electrical signal, and processes an image signal by converting the converted electrical signal into digital It is composed of ASIC part, and there are types such as CCD, CMOS, and CIS (Contact Image Sensor).

More specifically, the CCD image sensor moves electrons generated by light to the output unit using the gate pulse as it is, and converts electrons generated by light into voltages within each pixel and then outputs them through several CMOS switches. It is a CMOS image sensor. The field of application of these image sensors is very wide ranging from not only household products such as digital cameras and mobile phones, but also to endoscopes used in hospitals and telescopes of satellites orbiting the earth.

In the case of the prior art, when the original image data is low in sensitivity, the screen is dark and it is inconvenient for the user to see it, and to solve this problem, several images with different brightnesses must be synthesized. In this case, it takes a long time and the ghost phenomenon Due to this, there was a problem in which it was inconvenient for the user to view the image.

An embodiment of the present invention is an image sensor for generating image data with improved brightness by dividing pixel data into a plurality of areas and applying binning only to a specific area when a specific area among the divided areas is dark, and a control method thereof Its purpose is to provide.

Another embodiment of the present invention is to divide the pixel data into a plurality of areas, identify the brightness of the divided areas and classify them by brightness level, and apply different binning based on the brightness level to obtain image data with improved brightness. An object thereof is to provide an image sensor to be generated and a method for controlling the same.

The technical problem to be achieved in the present invention is not limited to the above-mentioned technical problem, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. .

According to an embodiment of the present invention, an image sensor includes: a pixel unit configured to convert an optical signal of an object into an electrical signal and output pixel data (original image) arranged in a matrix of a plurality of rows and columns; A row selection unit receiving a row address and generating a selection signal for selecting at least two rows from among the plurality of rows; An image synthesizing unit for generating binning data by applying binning to the pixel data (original image data); An analog-to-digital converter configured to convert the pixel data selected by the row selector and the generated binning data into a digital image signal and output the converted pixel data; And dividing the pixel data into nxm regions (where n and m are natural numbers), and applying different binning according to the attribute of the specific region in a specific region including at least a portion of the divided regions, the image combining unit And a controller configured to generate new pixel data by controlling to generate binning data, processing the binning data output to the specific area into a super-resolution image.

According to another embodiment of the present invention, a method of controlling an image sensor includes: converting an optical signal of a subject into an electrical signal, and outputting pixel data arranged in a matrix of a plurality of rows and columns; Generating a selection signal for selecting at least two rows from among the plurality of rows by receiving a row address; Generating binning data by applying binning to pixel data (original image data); Converting the pixel data and the generated binning data selected by a row selector into a digital image signal and outputting a digital image signal; And dividing the pixel data into n x m regions (where n and m are natural numbers). Controlling the image synthesizing unit to generate binning data by applying different binning according to a property of the specific area in a specific area including at least a portion of the divided areas; And generating new pixel data by processing the binning data output to the specific area as a super-resolution image.

According to an embodiment of the present invention, when pixel data is divided into a plurality of areas, and when a specific area among the divided areas is dark, binning is applied only to a specific area to generate image data with improved brightness. Time can be shortened, ghosting can be prevented, and since the brightness of an image is improved, user convenience can be improved.

According to another embodiment of the present invention, image data with improved brightness by dividing pixel data into a plurality of areas, identifying the brightness of the divided areas and classifying them by brightness level, and applying different binning based on the brightness level. Can be generated, the image processing time can be shortened and the ghost phenomenon can be prevented, so user convenience can always be improved.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.

1 is a diagram showing the configuration of an image sensor according to an embodiment of the present invention.

2 is a flowchart illustrating a method of controlling an image sensor according to an exemplary embodiment of the present invention.

3 is a diagram illustrating generating binning data by applying binning to pixel data and pixel data in a sensor according to an embodiment of the present invention.

4 is a diagram illustrating a flowchart of searching for a dark area in an entire image and applying binning to the dark area according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating generating an image that improves brightness by searching for a dark area in the entire image and applying binning to the dark area according to an embodiment of the present invention.

6 is a diagram illustrating a flow chart of dividing each region by brightness level and applying binning based on the brightness level according to an embodiment of the present invention.

FIG. 7 is a diagram illustrating an example in which each area is divided into brightness levels according to an embodiment of the present invention.

FIG. 8 is a diagram illustrating an example in which binning and weights are applied to regions divided by brightness levels according to an embodiment of the present invention.

9 is a diagram illustrating an example in which an exposure value is differently applied according to a weight when binning is applied according to an embodiment of the present invention.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but identical or similar elements are denoted by the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have meanings or roles that are distinguished from each other.

In addition, in describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, the detailed description thereof will be omitted.

In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all modifications included in the spirit and scope of the present invention , It should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various elements, but the elements are not limited by the terms. These terms are used only for the purpose of distinguishing one component from another component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In the present application, terms such as "comprises" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

Referring to FIG. 1, the image sensor 100 includes a pixel unit 110, a row selector 120, an image synthesis unit 130, an analog-to-digital conversion unit 140, a control unit 150, and an amplifier unit 160. , And a memory 170.

The pixel unit 110 converts an optical signal of a subject into an electrical signal and outputs pixel data (original image) arranged in a matrix of a plurality of rows and columns. The pixel unit 110 is arranged in a Bayer pattern composed of a first line in which red pixels and green pixels are alternately arranged, and a second line in which blue pixels and green pixels are alternately arranged.

The row selector 120 receives a row address and generates a selection signal for selecting at least two rows from among the plurality of rows.

The image synthesis unit 130 generates binning data by applying binning to pixel data (original image data).

The analog-to-digital converter 140 converts the pixel data selected by the row selector 120 and the generated binning data into a digital image signal and outputs the converted image signal.

The control unit 150 divides the pixel data into an nxm area (where n and m are natural numbers), and in a specific area including at least a part of the divided areas, differently applies binning according to the properties of the specific area to synthesize an image. The unit 130 controls to generate binning data, and processes the binning data output to a specific area as a super-resolution image to generate new pixel data.

The controller 150 generates new pixel data by fusing the output pixel data and the output binning data.

The amplifier unit 160 amplifies pixel data.

The memory 170 stores pixel data. The memory 170 is a flash memory type, a hard disk type, a solid state disk type, an SDD type, a multimedia card micro type. ), card-type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read (EEPROM) -only memory), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk.

Referring to FIG. 2, the pixel unit 110 converts an optical signal of a subject into an electrical signal and outputs pixel data (original image) arranged in a matrix of a plurality of rows and columns (S210). Here, the pixel data corresponds to the original image data.

The row selector 120 receives a row address and generates a selection signal for selecting at least two rows from among a plurality of rows (S220).

The image synthesis unit 130 generates binning data by applying binning to the pixel data (S230).

The analog-to-digital converter 140 converts the pixel data selected by the row selector 120 and the generated binning data into a digital image signal and outputs it (S240).

The controller 150 divides the pixel data into n x m regions (here, n and m are natural numbers) (S250).

The control unit 150 controls the image synthesis unit 130 to generate binning data by applying different binning according to a property of the specific region in a specific region including at least a portion of the divided regions (S260).

The control unit 150 generates new pixel data by processing the binning data output to a specific area as a super-resolution image (S270). Here, processing of the super-resolution image refers to generating new pixel data by fusing pixel data output to a specific area and binning data output to a specific area.

3 is a diagram illustrating generating binning data by applying binning to pixel data and pixel data in an image sensor according to an embodiment of the present invention. Figure 3 includes Figures 3(a) and 3(b).

3(a) is a diagram illustrating the image sensor 100 and the AP 200. The image sensor 100 outputs a full-size image result. In addition, the AP 200 processes image signal processing. That is, the AP 200 processes RAW data from the image sensor 100. After processing, the AP 200 displays the resulting image. Here, the resulting image may be jpeg. According to an embodiment of the present invention, the AP 200 may be the control unit 150.

3B is a diagram illustrating generating binning data by applying binning to pixel data and pixel data inside the image sensor 100.

Referring to FIG. 3B, the image synthesis unit 130 generates binning data 20 by applying binning to the pixel data 10. Here, the pixel data 10 means original image data. The image sensor 100 outputs a full-size image result.

For example, if the size of the pixel data is 48 Mb, the size of binning data is 12 Mb, and the size of the full-size output is 12 Mb. That is, when binning (total binning) is applied to pixel data corresponding to the original image data, the file size is reduced to 1/4.

Referring to FIG. 4, first, an image input is received through an image sensor (S410).

The preview screen is displayed (S412). The preview screen can be in YUV or other format. YUV is a video data format and refers to a color format that thrives in TV as a color face. Color face refers to sending a two-dimensional color using the frequency and amplitude of the color subcarriers. YUV is a format in which color and light are separately composed. The YUV method has the advantage of being able to transmit data with a small bandwidth compared to the RGB method.

The screen area is divided into N x N (S414).

The snapshot screen is displayed (S416). The snapshot screen can be in RAW/BAYER or other formats.

The screen area is divided into N x N (S418).

The two regions are mapped 1:1 (S420).

A dark area is searched (S430). Here, the dark area means an area in which the brightness of the area is darker than the preset first brightness.

When the dark area is searched (S440), binning is applied to the area found on the snapshot side (S450). Specifically, when the brightness of a specific area is darker than the preset first brightness, the control unit 150 controls the image synthesis unit 130 to generate binning data with improved brightness.

The controller 150 controls the image synthesis unit 130 not to generate binning data when the brightness of a specific region is in a saturation state.

This is because when the binning data is generated in a specific area, the brightness of the specific area is brightened. When the brightness of the specific area is saturated, the meaning of generating the binning data decreases.

If binning is applied to a bright area, it becomes brighter and the quality of the image is lowered due to the saturation effect. For example, a cloud image was visible before binning was applied, but when binning was applied, the cloud image was not visible.

If the dark area is not searched (S440), the dark area is searched again (S430).

SR-process the selected area (S460). Here, SR processing means Super Resolution processing. Specifically, the controller 150 generates new pixel data by performing super-resolution image processing in which the resolution information of the pixel data and brightness information of the binning data are combined. Therefore, the resolution of the new pixel data maintains the level of the original image data and the brightness becomes the brightness level of the binning data.

After SR processing, before encoding, the control unit 150 may modify the gain. The controller 150 modifies a gain of new pixel data located in a specific area and a gain of pixel data located in a different area than the specific area.

According to the present invention, since the gain can be corrected in the step before encoding, a sharper image can be obtained.

The result is encoded (S470).

The resulting image is displayed (S480). The resulting image can be a jpeg or other file format.

FIG. 5 is a diagram illustrating generating an image that improves brightness by searching for a dark area in the entire image and applying binning to the dark area according to an embodiment of the present invention. FIG. 5 includes FIGS. 5(a), 5(b), 5(c), and 5(d).

Fig. 5(a) is a diagram showing the entire input image. Referring to FIG. 5(a), some areas of the entire image are dark and are difficult to see.

Fig. 5(b) is a diagram showing that the entire image is divided and a dark area is specified. Referring to FIG. 5B, the specific area refers to a specific area of the entire image to which binning is to be applied. The

specific areas

52, 54, and 56 mean areas whose brightness is darker than a preset brightness.

Next, division of pixel data based on the resolution of the cell data will be described. The controller 150 divides the pixel data into n x m regions based on the resolution of the pixel data.

For example, when the resolution of the image is FHD (1920 x 1080), the controller 150 may divide the pixel data into 100 x 50 regions. When the resolution of the image is HD (1280 x 720), pixel data can be divided into 66 x 33 areas.

Accordingly, when the resolution is large, the control unit may divide the pixel data into more areas than when the resolution is small.

Also, pixel data may be divided into n x m regions based on the resolution of the image sensor 100.

For example, when the sensor has a resolution of 48 Mb, an 80 x 60 area, that is, pixel data may be divided into 4,800 areas. When the sensor has a resolution of 16 Mb, a 46 x 35 area, that is, pixel data can be divided into 1,600 areas.

5(c) is a diagram showing an entire image converted to a Bayer image. Referring to FIG. 5C, the specific area refers to a specific area of a partial area to which binning is applied among the entire image converted into a Bayer image. The

specific areas

62, 64, and 66 mean areas whose brightness is darker than a preset brightness.

5(d) is a diagram showing an entire image to which binning is applied. Referring to FIG. 5D, the controller 150 may improve brightness of the entire image by applying binning to a dark partial area of the entire image.

Next, generation of binning data having the same exposure and different brightness will be described. The image synthesizing unit 150 generates binning data having the same exposure as pixel data and different brightness from the pixel data. For example, the image synthesis unit 150 generates binning data having the same exposure and brighter than pixel data. Here, the pixel data corresponds to the original image data.

6 is a diagram illustrating a flow chart of dividing individual regions by brightness levels and applying binning based on brightness levels according to an embodiment of the present invention.

Referring to FIG. 6, first, an image input is received through an image sensor (S610).

The preview screen is displayed (S612). The preview screen can be in YUV or other format. YUV is a video data format and refers to a color format that thrives in TV as a color face. Color face refers to sending a two-dimensional color using the frequency and amplitude of the color subcarriers. YUV is a format in which color and light are separately composed. The YUV method has the advantage of being able to transmit data with a small bandwidth compared to the RGB method.

The screen area is divided into N x N (S614).

The snapshot screen is displayed (S616). The snapshot screen can be in RAW/BAYER or other formats.

The screen area is divided into N x N (S618).

The two regions are mapped 1:1 (S620).

Each area is divided into brightness levels, and binning is applied to a specific area based on the brightness level (S630). Here, the dark area means an area in which the brightness of the area is darker than the preset first brightness.

SR-process the selected area (S640). Here, SR processing means Super Resolution processing. Specifically, the controller 150 generates new pixel data by performing super-resolution image processing in which the resolution information of the pixel data and brightness information of the binning data are combined.

After SR processing, before encoding, the control unit 150 may modify the gain. The controller 150 modifies a gain of new pixel data located in a specific area and a gain of the pixel data located in a different area than the specific area.

According to the present invention, since the gain can be modified, a clearer image can be obtained.

The result is encoded (S650).

The resulting image is displayed (S660). The resulting image can be a jpeg or other file format.

FIG. 7 is a diagram illustrating an example in which each area is divided into brightness levels according to an embodiment of the present invention. 7 includes FIGS. 7(a), 7(b) and 7(c).

7(a) is a diagram in which the entire image is divided and the brightness of individual regions is sensed.

Referring to FIG. 7A, the control unit 150 divides the entire image into 11 x 6 regions and senses the brightness of individual regions.

7(b) is a diagram illustrating a table divided into stages according to the brightness range of individual sections.

Referring to FIG. 7B, the controller 150 senses the brightness of an individual region, and classifies the individual region by brightness level based on the sensing result. For example, there are 256 steps from 0 to 255 based on an 8-bit image (general JPEG), and if divided into 8 steps, it can be divided as shown in Fig. 7(b).

For level 1, the brightness range is 0-31.

For the second stage, the brightness range is 32-63.

For the 3rd level, the brightness range is 64-95.

For level 4, the brightness range is 96-127.

For the 5th level, the brightness range is 128-159.

In the case of step 6, the brightness range is 160-191.

For step 7, the brightness range is 192-223.

For level 8, the brightness range is 224-255.

The brightness of the image gets darker as you go to step 1, and the brightness of the image gets brighter as you go to step 8.

7(c) is a diagram illustrating a display of brightness levels corresponding to brightness of divided individual regions.

Referring to FIG. 7(c), a 5×6 area of 11×6 areas of the entire image will be described. First, the area is divided into 5 x 6 areas. The controller 150 maps a brightness level corresponding to the brightness of the divided individual regions to the individual regions.

In the case of the first line, the brightness levels are 3, 3, 4, 4, and 5.

In the case of the second line, the brightness levels are 4, 4, 5, 6, and 4.

In the case of the third line, the brightness levels are 2, 1, 2, 3, 4.

In the case of the fourth line, the brightness levels are 3, 3, 2, 3, 2.

For the fifth line, the brightness levels are 1, 1, 1, 2, and 2.

In the case of the sixth line, the brightness levels are 1, 1, 1, 1, 1.

FIG. 8 is a diagram illustrating an example in which binning and weights are applied to regions divided by brightness levels according to an embodiment of the present invention. 8 includes FIGS. 8(a) and 8(b).

FIG. 8A is a diagram illustrating a display of brightness levels corresponding to brightness of divided individual regions. Since a description of this has been described in FIG. 7(c), a detailed description will be omitted.

FIG. 8(b) is a diagram illustrating a table in which brightness levels are divided according to brightness ranges of individual sections, and binning and weights are applied based on brightness levels.

First, different application of binning based on the brightness level will be described. The controller 150 determines whether to apply binning based on the brightness level, and controls the image synthesis unit 130 to generate binning data based on the determination result.

For level 1, the brightness range is 0-31. Binning is applied.

For the second stage, the brightness range is 32-63. Binning is applied.

For the 3rd level, the brightness range is 64-95. Binning is applied.

For level 4, the brightness range is 96-127. Binning is applied.

For the 5th level, the brightness range is 128-159. Binning is applied.

In the case of step 6, the brightness range is 160-191. Binning is applied.

For the 7th level, the brightness range is 192-223. Do not apply binning.

For level 8, the brightness range is 224-255. Do not apply binning.

In the case of steps 7 and 8, since the brightness is sufficiently bright, specific contents of the image can be grasped, and in the case of steps 7 and 8, the controller 150 determines that binning is not applied. Accordingly, the image synthesizing unit 130 generates binning data by applying binning to a specific area only when the brightness level is 1 to 6 of the entire area.

Next, a description will be given of applying different weights based on the brightness level.

The control unit 150 determines a weight based on the brightness level, and controls the image synthesis unit 130 to generate binning data according to the determined weight.

For level 1, the brightness range is 0-31. The weight becomes +4.

For the second stage, the brightness range is 32-63. The weight becomes +3.

For the 3rd level, the brightness range is 64-95. The weight becomes +2.

For level 4, the brightness range is 96-127. The weight becomes +1.

For the 5th level, the brightness range is 128-159. The weight becomes -1.

In the case of step 6, the brightness range is 160-191. The weight becomes -2.

For the 7th level, the brightness range is 192-223. The weight becomes -2.

For level 8, the brightness range is 224-255. The weight becomes -3.

The brightness of the image gets darker as you go to step 1, and the brightness of the image gets brighter as you go to step 8. Accordingly, the closer to step 1, the control unit 150 sets the weight higher, and the closer to step 8, the control unit 150 sets the weight lower. The weight has a positive value, and the larger its size, the brighter the image.

Therefore, in the case of step 1, since the brightness of the image is the darkest, the control unit 150 determines the weight to be high, and controls the image synthesis unit 130 to generate binning data according to the determined weight.

In the case of step 8, since the brightness of the image is the brightest, the control unit 150 determines the weight to be low, and controls the image synthesis unit 130 to generate binning data according to the determined weight. Additional description of the weights will be described later in FIG. 9.

Next, different application of binning and weight based on the brightness level will be described.

The controller 150 determines binning application and weight based on the brightness level, and controls the image synthesis unit 130 to generate binning data according to the binning application result and the determined weight.

For level 1, the brightness range is 0-31. Binning is applied, and the weight is +4.

For the second stage, the brightness range is 32-63. Binning is applied, and the weight is +3.

For the 3rd level, the brightness range is 64-95. Binning is applied, and the weight is +2.

For level 4, the brightness range is 96-127. Binning is applied, and the weight is +1.

For the 5th level, the brightness range is 128-159. Binning is applied, and the weight is -1.

In the case of step 6, the brightness range is 160-191. Binning is applied, and the weight is -2.

For the 7th level, the brightness range is 192-223. No binning is applied, and the weight is -2.

For level 8, the brightness range is 224-255. No binning is applied, and the weight is -3.

In addition, in step 1, since the brightness of the image is the darkest, the controller 150 determines the weight to be high, and controls the image synthesis unit 130 to generate binning data according to the determined weight.

In the case of step 8, since the brightness of the image is the brightest, the control unit 150 determines the weight to be low, and controls the image synthesis unit 130 to generate binning data according to the determined weight.

That is, in the first step, the controller 150 applies binning to a specific area, determines a high weight, and controls the image synthesis unit 130 to generate binning data according to the determined weight.

In the case of step 8, the control unit 150 does not apply binning to a specific region, determines a low weight, and controls the image synthesis unit 130 to generate binning data according to the determination. Here, an additional description of the weight will be described later in FIG. 9.

According to the present invention, the entire image is divided into a plurality of regions,

9 is a diagram illustrating an example in which an exposure value is differently applied according to a weight when binning is applied according to an embodiment of the present invention. 9 includes FIGS. 9(a), 9(b), 9(c), 9(d), and 9(e).

9(a) is a diagram showing a case where the exposure value is -1.0. This is the case when the weight is low. The brightness of the image is the darkest.

9(b) is a diagram showing a case where the exposure value is -0.3.

9(c) is a diagram illustrating a case where the exposure value is 0.

9(d) is a diagram showing a case where the exposure value is +0.3.

9(e) is a diagram showing a case where the exposure value is +1.0. This is the case when the weight is high. The brightness of the image is the brightest.

9(a) to 9(e), the greater the weight has a positive value and the greater the absolute value, the brighter the brightness of the image. As the weight has a negative value and the magnitude of the absolute value is larger, the brightness of the image is darker.

The control unit 150 processes the weight by increasing the exposure value (EV) as the weight has a positive value.

The image display device and its operation method according to the present invention are not limited to the configuration and method of the embodiments described as described above, but the embodiments are all or part of each embodiment so that various modifications can be made. May be configured by selectively combining.

Meanwhile, the method of operating a video display device of the present invention may be implemented as a code that can be read by a processor on a recording medium that can be read by a processor provided in the video display device. The processor-readable recording medium includes all types of recording devices that store data that can be read by the processor. Examples of recording media that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and also include those implemented in the form of carrier waves such as transmission through the Internet. . In addition, the recording medium readable by the processor may be distributed over a computer system connected through a network, so that code readable by the processor may be stored and executed in a distributed manner.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

Various embodiments have been described in the best mode for carrying out the present invention.

The present invention is used in an image sensor related field that improves the brightness of a dark original image by applying binning to a specific area of the entire image and provides a clearer image.

It is apparent to those skilled in the art that various changes and modifications are possible in the present invention without departing from the spirit or scope of the present invention. Accordingly, the present invention is intended to cover modifications and variations of the present invention provided within the appended claims and their equivalents.

Claims

In the image sensor,

A pixel unit configured to convert an optical signal of a subject into an electrical signal and output pixel data arranged in a matrix of a plurality of rows and columns;

A row selection unit receiving a row address and generating a selection signal for selecting at least two rows from among the plurality of rows;

An image synthesizing unit 130 for generating binning data by applying binning to the pixel data;

An analog-to-digital converter configured to convert the pixel data selected by the row selector and the generated binning data into a digital image signal and output the converted pixel data; And

Dividing the pixel data into n x m regions (where n and m are natural numbers),

In a specific area including at least a portion of the divided areas, binning is applied differently according to the property of the specific area, and the image combining unit generates binning data,

A control unit that generates new pixel data by processing the binning data output to the specific area as a super resolution image

An image sensor comprising a.
The method of claim 1, wherein the control unit

When the brightness of the specific area is darker than a preset first brightness, controlling the image combining unit to generate binning data with improved brightness,

Image sensor.
The method of claim 1, wherein the control unit

When the brightness of the specific region is in a saturation state, controlling the image synthesis unit not to generate binning data,

Image sensor.
The method of claim 1, wherein the control unit

Generating new pixel data by performing super-resolution image processing in which the resolution information of the pixel data and brightness information of the binning data are fused,

Image sensor.
The method of claim 1, wherein the control unit

Modifying a gain of the new pixel data located in the specific area and a gain of the pixel data located in a different area from the specific area,

Image sensor.
The method of claim 1, wherein the control unit

Dividing the pixel data into n x m regions based on the resolution of the pixel data,

Image sensor.
The method of claim 1, wherein the control unit,

Sensing the brightness of individual areas,

Dividing the individual regions by brightness level based on the sensing result

Image sensor.
The method of claim 7, wherein the control unit

Decide whether to apply binning based on the brightness level,

Controlling the image synthesis unit to generate binning data based on the determination result,

Image sensor.
The method of claim 7, wherein the control unit

Determine the weights based on the brightness level,

Controlling the image synthesis unit to generate binning data according to the determined weight,

Image sensor.
The method of claim 1, wherein the image combining unit

The exposure is the same as the pixel data, and the brightness generates binning data different from the pixel data,

Image sensor.
In the control method of the image sensor,

Converting an optical signal of an object into an electrical signal and outputting pixel data arranged in a matrix of a plurality of rows and columns;

Generating a selection signal for selecting at least two rows from among the plurality of rows by receiving a row address;

Generating binning data by applying binning to the pixel data;

Converting the pixel data selected by the row selection unit and the generated binning data into a digital image signal and outputting a digital image signal; And

Dividing the pixel data into n x m regions (where n and m are natural numbers);

Controlling the image synthesizing unit to generate binning data by applying different binning according to a property of the specific area in a specific area including at least a portion of the divided areas; And

Generating new pixel data by processing the binning data output to the specific area as a super resolution image

Control method of an image sensor comprising a.
The method of claim 11,

When the brightness of the specific area is darker than a preset first brightness, controlling the image combining unit to generate binning data with improved brightness,

How to control the image sensor.
The method of claim 11,

When the brightness of the specific region is in a saturation state, controlling the image synthesis unit not to generate binning data,

How to control the image sensor.
The method of claim 11,

Generating new pixel data by performing a super-resolution image processing in which the resolution information of the pixel data and the brightness information of the binning data are fused,

How to control the image sensor.
The method of claim 11,

Generating new pixel data by performing a super-resolution image processing in which the resolution information of the pixel data and the brightness information of the binning data are fused,

How to control the image sensor.