KR20220009843A - Camera module, image processing system and image processing method - Google Patents

Abstract

According to the technical idea of the present disclosure, provided is an image compression method. The image compression method for compressing each of a plurality of pixel groups constituting image data includes the steps of: detecting bad pixels in a group of pixels; generating a flag indicating location information of the bad pixel; calculating difference values between pixel values of pixels other than the bad pixel in the pixel group and a reference pixel value; and generating a bitstream including the flag and the difference values.

Description

Camera module, image processing system and image compression method {CAMERA MODULE, IMAGE PROCESSING SYSTEM AND IMAGE PROCESSING METHOD}

The technical idea of the present disclosure relates to a camera module, an image processing system, and an image compression method, and more particularly, to a camera module, an image processing system, and an image compression method for compressing image data in consideration of a bad pixel.

Recently, as the demand for high-quality and high-definition pictures and images increases, the size of the pixels of the image sensor is reduced and the number of pixels of the image sensor is increased. Accordingly, due to a process issue, pixels of the image sensor may include bad pixels constituting an arbitrary shape at an arbitrary location. Since a large number of bad pixels of arbitrary shapes are not used to generate photos, images, and the like, there is a problem in degrading the performance of the image sensor. Accordingly, there is a need for a technique for correcting image data output from the bad pixels.

The technical idea of the present disclosure provides a camera module, an image processing apparatus, and an image compression method for compressing image data in consideration of a bad pixel.

In order to achieve the above object, an image compression method according to an aspect of the technical idea of the present disclosure includes detecting a bad pixel in a pixel group, generating a flag indicating position information of the bad pixel, and the pixel The method may include calculating difference values between pixel values of pixels other than the bad pixel in the group and a reference pixel value, and generating a bitstream including the flag and the difference values.

A camera module according to an aspect of the technical concept of the present disclosure includes an image sensor that generates image data including a plurality of pixels, divides the plurality of pixels into a plurality of pixel groups, and divides the image data into a pixel group unit An encoder for sequentially compressing to generate compressed data including a plurality of bitstreams, and a memory for storing reference information including pixel values corresponding to pixels compressed by the encoder, wherein the encoder includes: A first pixel corresponding to the first pixel group is detected by detecting a bad pixel included in the pixel group, and compressing pixel values of the first pixels included in the first pixel group based on the detection result of the bad pixel and the reference information. A bitstream may be generated and the reference information may be updated based on pixel values corresponding to the first pixels.

An image processing system according to an aspect of the inventive concept generates a plurality of bitstreams by sequentially compressing an image sensor generating image data including a plurality of pixels and a plurality of pixel groups constituting the image data. and a decoder that decompresses the plurality of bitstreams and reconstructs the image data, wherein the encoder, for each of the plurality of pixel groups, detecting a bad pixel in a corresponding pixel group, the corresponding pixel Compressing pixel values of a corresponding pixel group based on reference information generated based on pixel values corresponding to pixels compressed before a group, and updating the reference information based on a result of detecting the bad pixel can do.

According to the image compression method, the camera module, and the image processing system according to an embodiment of the present disclosure, reference information used for compression or decompression is generated in consideration of a bad pixel, and image data is compressed or By decompressing the compressed data, it is possible to remarkably reduce the number of bad pixel detection operations. Accordingly, it is possible to reduce the amount of computation and power consumption.

1 is a diagram illustrating an image processing system according to an exemplary embodiment of the present disclosure.
2 is a diagram illustrating an encoder according to an exemplary embodiment of the present disclosure.
3 is a conceptual diagram illustrating an image compression method according to an exemplary embodiment of the present disclosure.
4 is a conceptual diagram illustrating an image compression method according to an exemplary embodiment of the present disclosure.
5A and 5B are diagrams illustrating a bitstream according to an exemplary embodiment of the present disclosure.
6 is a diagram illustrating a bitstream according to an exemplary embodiment of the present disclosure.
7 is a flowchart illustrating an image compression method according to an exemplary embodiment of the present disclosure.
8 is a diagram illustrating a decoder according to an exemplary embodiment of the present disclosure.
9A and 9B are conceptual diagrams illustrating an image decompression method according to an exemplary embodiment of the present disclosure.
10 is a diagram illustrating an image processing system according to an exemplary embodiment of the present disclosure.
11 is a diagram illustrating an encoder according to an exemplary embodiment of the present disclosure.
12A and 12B are tables for explaining compressed information according to an exemplary embodiment of the present disclosure.
13 is a diagram illustrating an image processing system according to an exemplary embodiment of the present disclosure.
14 is a diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure.
15 is a diagram illustrating a part of an electronic device according to an exemplary embodiment of the present disclosure.
16 is a diagram illustrating a detailed configuration of a camera module according to an exemplary embodiment of the present disclosure.

1 is a diagram illustrating an image processing system according to an exemplary embodiment of the present disclosure. Referring to FIG. 1 , the image processing system 10 may include a camera module 100 and an image processing apparatus 200 . In some embodiments, the camera module 100 may include an image sensor 110 , an encoder 120 , a memory 130 , and an interface 140 . In some embodiments, the image processing apparatus 200 may include an interface 210 , a memory 220 , a decoder 230 , and an image signal processor (ISP) 240 .

For example, the image processing system 10 may be implemented as a personal computer (PC), an Internet of Things (IoT) device, or a portable electronic device. Portable electronic devices include laptop computers, mobile phones, smart phones, tablet PCs, personal digital assistants (PDAs), enterprise digital assistants (EDAs), digital still cameras, digital video cameras, audio devices, PMPs (portable multimedia players), PNDs. (personal navigation device), an MP3 player, a handheld game console, an e-book, a wearable device, and the like. In addition, the image processing system 10 is an electronic device, such as a drone, an advanced driver assistance system (ADAS), etc. or an electronic device provided as a component in a vehicle, furniture, manufacturing facility, door, various measuring devices, etc. can be mounted on

The camera module 100 may photograph an external subject (or object) and generate image data IDT. In some embodiments, the camera module 100 may include an image sensor 110 capable of converting an optical signal of a subject into an electrical signal. For example, the image sensor 110 may include a pixel array 111 in which a plurality of pixels are two-dimensionally arranged. One color among a plurality of reference colors may be assigned to each of the plurality of pixels. For example, the plurality of reference colors may include red, green, blue (RGB) or red, green, blue, and white (RGBW), and may include other colors. For example, the plurality of reference colors may include cyan, yellow, green, and magenta. The pixel array 111 may generate pixel signals including information on a reference color of each of the plurality of pixels.

The pixel array 111 has a plurality of row lines, a plurality of column lines, and a plurality of pixels each connected to a row line and a column line and arranged in a matrix form, and arranged to correspond to each of the plurality of pixels It may include a plurality of color filters. For example, referring to FIG. 1 , in the color filter, cells (Cells) having a size of 2 X 2 including a red pixel (R), a blue pixel (B), and two green pixels (Gr, Gb) are repeatedly performed. It may be a deployed configuration. Such a pattern may be referred to as a Bayer pattern.

In another example, the color filter may have a configuration in which pixel groups corresponding to each of the reference colors are repeatedly disposed. For example, the color filter may include a red pixel group including red pixels R arranged in 2 X 2, a first green pixel group including first green pixels Gr arranged in 2 X 2, and 2 X 2 The blue pixel group including the blue pixels B arranged as , and the second green pixel group including the green pixels Gb arranged as 2×2 may be repeatedly arranged. Such a pattern may be referred to as a TETRA pattern.

In another example, the color filter includes a red pixel group including red pixels (R) arranged in 3 X 3, a first green pixel group including first green pixels (Gr) arranged in 3 X 3, 3 X A blue pixel group including blue pixels B arranged in 3 and a second green pixel group including green pixels Gb arranged in 3×3 may be repeatedly arranged. Such a pattern may be referred to as a NONA pattern.

Meanwhile, the present disclosure is not limited thereto, and the color filter includes a red pixel group, a blue pixel group, a first green pixel group, and a second Pixel groups may be repeatedly arranged.

As a non-limiting example, the image sensor 110 may be implemented using a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS), and in addition to various types of photoelectric It may be implemented as a conversion element. In some embodiments, the image sensor 110 may output image data IDT on which pre-processing of the pixel signal generated by the pixel array 111 is performed.

Meanwhile, in the above example, it has been described that the image data IDT includes information about the reference color (eg, RGB information), but the present disclosure is not limited thereto. Specifically, the image sensor 110 may convert RGB information of each pixel into YUV information including information on luminance and color difference through color space conversion, and thus the image data IDT is It may include YUV information. Even when the image data IDT includes YUV information, the technical spirit of the present disclosure may be applied substantially the same.

The camera module 100 may compress the image data IDT by using the encoder 120 in order to reduce power consumption according to data transmission and to increase the efficiency of data storage space. Specifically, the encoder 120 may receive the image data IDT from the image sensor 110 , and compress the image data IDT to generate compressed data CDT. Compressed data CDT may be implemented in the form of an encoded bitstream. Hereinafter, the encoded bitstream is simply referred to as a bitstream.

In some embodiments, the encoder 120 may compress the image data IDT in units of pixel groups. Here, the pixel group may be set to include a predetermined number of pixels sequentially arranged according to the pattern of the image data IDT, or may be set to include pixels corresponding to the same reference color and adjacent to each other. For example, when the image data IDT is a Bayer pattern, the pixel group may be set to include a predetermined number (eg, four) of pixels sequentially arranged horizontally or vertically. As another example, when the image data IDT is a tetra pattern (or nona pattern), a pixel group corresponds to the same reference color (eg, red, blue, green, etc.) and is adjacent to each other 4 (or 9) pixels may be set to include The encoder 120 may generate one bitstream by compressing one pixel group, and may generate compressed data CDT based on bitstreams of all pixel groups in the image data IDT.

When compressing a specific pixel group, the encoder 120 may perform compression using the first reference map RM1 generated based on pixel values corresponding to pixels compressed before the corresponding pixel group. Specifically, the encoder 120 determines a reference value based on the pixel value of at least one pixel adjacent to the target pixel to be compressed in the pixel group in the first reference map RM1, and the reference value and the pixel value of the target pixel A pixel value of the target pixel may be compressed based on .

When the compression of the pixel group is completed, the encoder 120 may generate (or update) a new first reference map RM1 by adding the pixel values of the compressed pixel group to the existing first reference map RM1. . In addition, the encoder 120 may compress the pixel group of the next order by using the new first reference map RM1.

The pixel value of the target pixel and the pixel value of the adjacent pixel are highly likely to have similar values. Accordingly, when the image data IDT is compressed according to the above-described method to generate the compressed data CDT, compression efficiency may be increased and data loss may be reduced. Hereinafter, for convenience of description, information including pixel values corresponding to pixels that have been compressed first is referred to as the first reference map RM1, but the present disclosure is not limited thereto, and may be referred to by various names such as reference information. Of course you can.

Meanwhile, the image data IDT may include a pixel value of a bad pixel. Here, the bad pixel may include a static bad pixel continuously turned on or turned off and a dynamic bad pixel randomly turned on or turned off. Since the position of the static bad pixel is fixed, the pixel value of the static bad pixel may be corrected through a pre-processing operation of the image sensor 110 capable of simple arithmetic processing. On the other hand, since the dynamic bad pixel is not fixed in position and requires a lot of calculations for correction, it may be somewhat difficult to be corrected through a pre-processing operation of the image sensor 110 . Accordingly, the dynamic bad pixel may remain in the image data IDT.

Since the encoder 120 is configured for compression rather than correction of the image data IDT, the image data IDT may be compressed even if bad pixels are included. Accordingly, when the encoder 120 performs compression according to the above-described method, the pixel value of the bad pixel may be included in the first reference map RM1 . Since the pixel value of the bad pixel has a large difference from the pixel value of the neighboring pixels, compression efficiency may decrease and data loss may increase during a compression operation using the first reference map RM1 .

As a method to prevent this, the encoder 120 may perform a bad pixel detection operation on pixels included in the first reference map RM1 . If the bad pixel is included in the first reference map RM1 , the reference value may be determined based on pixel values of pixels other than the bad pixel (ie, normal pixels). Meanwhile, since the above-described bad pixel detection operation requires a large amount of computation and is performed for each compression operation of all pixels, power consumption may increase.

Accordingly, the encoder 120 according to the technical spirit of the present disclosure corrects the pixel values of the bad pixels (particularly, the dynamic bad pixels) further than the method of performing the compression in the above-described manner, and sets the corrected pixel values to the first It can be included in the reference map RM1. Accordingly, since the pixel value of the bad pixel is not included in the first reference map RM1 , the bad pixel detection operation for the first reference map RM1 may be omitted.

The memory 130 is a storage location for storing data, and may store, for example, the image data IDT or the first reference map RM1 . The memory 130 may be a volatile memory such as a dynamic random access memory (DRAM) or a static RAM (SRAM), or a non-volatile memory such as a phase change RAM (PRAM), a resistive RAM (ReRAM), or a flash memory.

The encoder 120 may provide the generated compressed data CDT to the image processing apparatus 200 through the interface 140 . For example, the interface 140 may be implemented as a Camera Serial Interface (CSI) based on a Mobile Industry Processor Interface (MIPI). Meanwhile, the type of the interface 140 is not limited thereto, and may be implemented according to various protocol standards.

The image processing apparatus 200 may convert image data received from the camera module 100 to generate an image to be displayed on a display (not shown). Specifically, the image processing apparatus 200 receives the compressed data CDT from the camera module 100 , decompresses the compressed data CDT to generate decompressed data DDT, and generates decompressed data DDT. A final image may be generated by performing an image processing operation based on .

In some embodiments, the image processing apparatus 200 may receive the compressed data CDT from the camera module 100 through the interface 210 . The interface 210 may be implemented in MIPI like the interface 140 , but is not limited thereto. The image processing apparatus 200 may store the received compressed data CDT in the memory 220 .

The memory 220 is a storage location for storing data, and may store, for example, an operating system (OS), various programs, and various data (eg, compressed data (CDT)). The memory 220 may be a volatile memory such as DRAM or SRAM, or a non-volatile memory such as PRAM, ReRAM, or flash memory.

The decoder 230 may read the compressed data CDT from the memory 220 and decompress the compressed data CDT to generate decompressed data DDT. The decoder 230 may output the generated decompressed data DDT to the image signal processor 240 .

In some embodiments, the decoder 230 may decompress the compressed data CDT in units of pixel groups. In this case, the decoder 230 may decompress the compression using the second reference map RM2 generated based on pixel values corresponding to the pixels decompressed before the pixel group.

According to an exemplary embodiment of the present disclosure, the second reference map RM2 may include corrected pixel values of bad pixels (particularly, dynamic bad pixels), like the above-described first reference map RM1 . In other words, if there is a bad pixel in the decompressed pixel group, the decoder 230 corrects the pixel value of the bad pixel and generates (or updates) a new second reference map RM2 including the corrected pixel value. can In addition, the decoder 230 may decompress the next-order pixel group based on the new second reference map RM2 . The second reference map RM2 may be stored in the memory 220 . Hereinafter, for convenience of explanation, information including pixel values corresponding to pixels decompressed before a pixel group to be decompressed is referred to as a second reference map RM2, but the present disclosure is not limited thereto, and various Of course, it may be referred to as a name.

Meanwhile, each of the encoder 120 and the decoder 230 may be implemented as software, hardware, or a combination of software and hardware such as firmware. When the encoder 120 and the decoder 230 are implemented in software, each of the above-described functions is implemented as a programmed source code and loaded into a storage medium provided in each of the camera module 100 and the image processing apparatus 200, respectively. The functions of the encoder 120 and the decoder 230 may be implemented by a processor (eg, an image processing processor) provided in each of the camera module 100 and the image processing apparatus 200 executing software. . When the encoder 120 and the decoder 230 are implemented in hardware, the encoder 120 and the decoder 230 may include a logic circuit and a register, and may perform each of the above-described functions based on the register setting. have.

The image signal processor 240 may perform various image processing on the received decompressed data DDT. In a non-limiting example, the image signal processor 240 performs a bad pixel correction, an offset correction, a lens distortion correction, a color gain correction, a shading correction, and a gamma with respect to the decompressed data DDT. ) correction, denoising, and sharpening may be performed. In some embodiments, depending on the performance of the camera module 100, some of the above-described image processing may be omitted. For example, when the camera module 100 includes the high-quality image sensor 110, bad pixel correction (particularly, static bad pixel correction) or offset correction among image processing may be omitted.

Meanwhile, although FIG. 1 illustrates that the image processing system 10 includes the camera module 100 and the image processing apparatus 200 , the present disclosure is not limited thereto. For example, the image processing system 10 may be implemented to include only a part of the camera module 100 and the image processing apparatus 200 , or to include a plurality of camera modules 100 . In addition, although the decoder 230 and the image signal processor 240 are illustrated as separate components in FIG. 1 , the present disclosure is not limited thereto. For example, the image signal processor 240 may be implemented to include a decoder 230 .

In addition, although the memory 130 and the memory 220 are illustrated and described as being included in the camera module 100 and the image processing apparatus 200 with reference to FIG. 1 , the present disclosure is not limited thereto. For example, each of the memory 130 and the memory 220 may be implemented to be located outside the camera module 100 or the image processing apparatus 200 .

An image processing system according to the technical spirit of the present disclosure generates reference maps used for compression or decompression in consideration of a bad pixel, and compresses or decompresses image data based on the generated reference maps, thereby providing a bad pixel The number of detection operations can be drastically reduced. Accordingly, power consumption of the image processing system may be reduced.

2 is a diagram illustrating an encoder according to an exemplary embodiment of the present disclosure. Specifically, FIG. 2 is a diagram illustrating the encoder 120 of FIG. 1 . 1 and 2 , the encoder 120 may include a bad pixel detector 121 , a compressor 123 , and a first reference map generator 125 .

The bad pixel detector 121 may receive image data IDT from the image sensor 110 . The bad pixel detector 121 may detect a bad pixel with respect to a pixel group to be compressed based on the received image data IDT. A detailed description thereof will be described later with reference to FIGS. 3 to 4 . The bad pixel detector 121 may transmit the bad pixel information BP including the detection result to the compressor 123 and/or the first reference map generator 125 .

The compressor 123 may generate the compressed data CDT by compressing the image data IDT using the bad pixel information BP and the first reference map RM1 . Specifically, the compressor 123 may identify a bad pixel in a pixel group to be compressed of the image data IDT based on the bad pixel information BP received from the bad pixel detector 121 . In addition, the compressor 123 may compress pixel values of the remaining pixels (ie, normal pixels) except for the bad pixels in the pixel group by using the first reference map RM1 .

For example, the compressor 123 may determine a reference value based on a pixel value of at least one pixel adjacent to a pixel group or a normal pixel in the first reference map RM1 . In addition, the compressor 123 may generate the bitstream BS corresponding to the pixel group by compressing it based on the reference value and the pixel value of a normal pixel that is not a bad pixel. A detailed description thereof will be described later with reference to FIGS. 3 and 4 .

The compressor 123 may generate bitstreams BS corresponding to the pixel groups by repeating the above-described operation for each of the pixel groups of the image data IDT. In addition, the compressor 123 may generate compressed data CDT based on the bitstreams BS.

The compressor 123 may transmit the generated compressed data CDT to the interface 140 . The interface 140 may transmit compressed data CDT to the image processing apparatus 200 . Also, the compressor 123 may transmit the bitstream BS to the first reference map generator 125 whenever each of the bitstreams BS is generated.

The first reference map generator 125 may generate a new first reference map RM1 (NEW) based on the received bad pixel information BP and the bitstream BS. Specifically, the first reference map generator 125 may restore the pixel values of the pixel group by first decoding the bitstream BS. In addition, the first reference map generator 125 may identify a bad pixel in a pixel group based on the bad pixel information BP. In addition, the first reference map generator 125 may correct the pixel value of the bad pixel among the restored pixel values to be similar to the pixel values of the neighboring pixels. In addition, the first reference map generator 125 adds the reconstructed pixel values of pixels other than the bad pixels in the pixel group and the corrected pixel values of the bad pixels to the existing first reference map RM1 to create a new first reference map ( RM1(NEW)) can be created. A detailed description thereof will be described later with reference to FIGS. 3 and 4 .

In this way, when one pixel group is compressed, the first reference map generator 125 generates a pixel value corresponding to the compressed pixel group (ie, the reconstructed pixel values of the pixel group and the / or correction pixel values of bad pixels) may be added to generate a new first reference map RM1(NEW). The first reference map generator 125 may store the generated first reference map RM1(NEW) in the memory 130 . The compressor 123 reads the new first reference map RM1 (NEW) stored in the memory 130 and performs compression on the pixel group in the next order based on the read first reference map RM1. have.

Meanwhile, each of the bad pixel detector 121 , the compressor 123 , and the first reference map generator 125 may be implemented as software, hardware, or a combination of software and hardware such as firmware. When the bad pixel detector 121 , the compressor 123 , and the first reference map generator 125 are implemented as software, each of the above-described functions is implemented as a programmed source code and a storage medium provided in the camera module 100 . may be loaded into each, and a processor (eg, a microprocessor) provided in each camera module 100 executes software, so that the bad pixel detector 121 , the compressor 123 and the first reference map generator 125 are Functions may be implemented. When the bad pixel detector 121, the compressor 123, and the first reference map generator 125 are implemented in hardware, the bad pixel detector 121, the compressor 123, and the first reference map generator 125 are logic circuits. and a register, and may perform each of the above-described functions based on the register setting.

3 is a conceptual diagram illustrating an image compression method according to an exemplary embodiment of the present disclosure. Specifically, FIG. 3 is a diagram illustrating a compression method for image data IDT of a Bayer pattern of the encoder 120 . Hereinafter, description will be made on the assumption that one pixel group is set to include four sequentially arranged pixels. Also, an embodiment in which the encoder 120 compresses the pixel group PG including the bad pixels R7 and the normal pixels Gr7, R8 and Gr8 will be described. Also, for convenience of description, a pixel to be processed among pixels in the pixel group PG is referred to as a target pixel.

The bad pixel detector 121 may perform a bad pixel detection operation on pixels in the pixel group PG. Specifically, the bad pixel detector 121 may detect whether the target pixel is a bad pixel based on a pixel value of at least one first pixel adjacent to the target pixel in the image data IDT. In some embodiments, the bad pixel detector 121 may detect whether the target pixel is a bad pixel based on a pixel value of at least one first pixel that corresponds to the same reference color as that of the target pixel. In this case, various criteria for determining a pixel adjacent to the target pixel may be set. For example, a pixel in direct contact with the target pixel or a pixel located within a predetermined distance from the target pixel may be determined to be adjacent.

Specifically, when there are a plurality of first pixels, the bad pixel detector 121 may calculate an average pixel value of the first pixels and calculate a difference value between the average pixel value and the pixel value of the target pixel. When the calculated difference value exceeds the threshold value, the bad pixel detector 121 may detect the target pixel as a bad pixel. Here, the threshold value may be preset by a user or a manufacturer, and may have a different value for each image data IDT. The bad pixel detector 121 may generate bad pixel information by repeating the above-described operation for each of the pixels in the pixel group PG. In this case, the bad pixel information may include position information of the bad pixel.

For example, referring to FIG. 3 , the bad pixel detector 121 corresponds to the same red color as the pixel R7 which is the target pixel in the image data IDT, and the first pixels adjacent to the pixel R7 ( For example, the average pixel value of R3, R6, R8, and R11) may be calculated. In addition, the bad pixel detector 121 may calculate a difference value between the calculated average pixel value and the pixel value of the pixel R7 . When the calculated difference value exceeds a threshold value, the bad pixel detector 121 may detect the pixel R7 as a bad pixel.

Meanwhile, the bad pixel detection operation of the bad pixel detector 121 is not limited to the above-described example, and various methods may be applied. For example, the bad pixel detector 121 may detect whether the target pixel is a bad pixel based on a pixel value of at least one first pixel adjacent to the target pixel regardless of the reference color.

For example, referring to FIG. 3 , the bad pixel detector 121 determines the pixel values of first pixels (eg, R6 , Gr6 , Gr7 , and R8 ) horizontally adjacent to the pixel R7 in the image data IDT. Whether the pixel R7 is a bad pixel may be detected based on the values. As another example, the bad pixel detector 121 detects whether the pixel R7 is a bad pixel based on pixel values of the first pixels (eg, R3 , Gb3 , Gb7 , and R11 ) vertically adjacent to the pixel R7 . can do. As another example, the bad pixel detector 121 may detect whether the pixel R7 is a bad pixel based on the pixel values of the remaining pixels (eg, Gr7, R8, Gr8) of the pixel group including the pixel R7. can

The compressor 123 may identify a bad pixel in a pixel group based on the bad pixel information received from the bad pixel detector 121 , and may compress pixel values of pixels other than the bad pixel.

Specifically, the compressor 123 identifies at least one second pixel that corresponds to the same reference color as a target pixel that is not a bad pixel in the first reference map RM1 and is adjacent to the target pixel (or pixel group PG). can In addition, the compressor 123 may determine a reference value based on the checked pixel value of the second pixel. In addition, the compressor 123 may compress the pixel value of the target pixel based on the reference value and the pixel value of the target pixel. In this case, various criteria for determining the adjacent second pixel may be set.

In some embodiments, when the number of second pixels is plural, the compressor 123 may determine a pixel at a specific position (eg, on the left, upper, or diagonal side of the target pixel) among the second pixels as a reference value. Alternatively, the compressor 123 may calculate an average pixel value of the checked second pixels, and determine the calculated average pixel value as a reference value. In addition, the compressor 123 may calculate a difference value RES between the reference value and the pixel value of the target pixel.

The compressor 123 may calculate a plurality of difference values RES by repeating the above-described operation for all pixels in the pixel group PG, and the bitstream BS including the plurality of difference values RES can create As such, the compression method performed based on the difference value RES may be referred to as a differential pulse code modulation (DPCM) method.

For example, referring to FIG. 3 , the compressor 123 determines that the target pixel in the first reference map RM1 corresponds to the same reference color as the pixel Gr7 , and the second pixels adjacent to the pixel Gr7 (eg, , Gb3, Gb4, Gb7, Gb8, Gr3, Gr6, Gr8). Then, the compressor 123 uses the pixel value of the pixel (eg, Gr6) located to the left of the pixel Gr7 among the checked second pixels (eg, Gb3, Gb4, Gb7, Gb8, Gr3, Gr6, Gr8) as a reference value. can decide In addition, the compressor 123 may calculate a difference value RES between the reference value and the pixel value of the pixel Gr7 and generate a bitstream BS including the calculated difference value RES.

When the compression of the pixel group PG is completed, the first reference map generator 125 adds pixel values corresponding to the compressed pixel group PG to the existing first reference map RM1 to create a new first reference. A map RM1 (NEW) can be created. For example, the first reference map generator 125 restores pixel values of the pixel group PG by decoding the bitstream BS of the pixel group PG received from the compressor 123 , and the restored pixel values are added to the existing first reference map RM1 to generate a new first reference map RM1 (NEW). Meanwhile, a method for the first reference map generator 125 to reconstruct pixel values based on the bitstream BS may be substantially the same as a decompression method for the decoder 230 to be described later. As another example, the first reference map generator 125 may add pixel values corresponding to the pixel group PG in the image data IDT to the existing first reference map RM1 .

Meanwhile, according to the technical idea of the present disclosure, when a bad pixel exists in the pixel group PG, the first reference map generator 125 first corrects the pixel value of the bad pixel and then uses the existing first reference map RM1 can be added to

In some embodiments, the first reference map generator 125 may correct the pixel value of the bad pixel according to various methods based on the pixel values of the third pixels adjacent to the bad pixel. Specifically, the first reference map generator 125 may identify third pixels adjacent to the bad pixel by corresponding to the same reference color as the bad pixel. In addition, the first reference map generator 125 may correct the pixel value of the bad pixel based on the checked pixel values of the third pixels.

For example, referring to FIG. 3 , the first reference map generator 125 corresponds to the same red color as that of the pixel R7, which is a bad pixel, and a third pixel (eg, R3) located above the pixel R7. The pixel value of the pixel R7 may be corrected with the pixel value of . As another example, the first reference map generator 125 may correct the pixel value of the pixel R7 with the pixel value of the third pixel (eg, R6 ) located to the left of the pixel R7 . As another example, the first reference map generator 125 may correct the pixel value of the pixel R7 with the pixel value of a third pixel (eg, R8 ) located to the right of the pixel R7 . As another example, the first reference map generator 125 may correct the pixel value of the pixel R7 with the average pixel value of the third pixels (eg, R2 and R4 ) located on the diagonal side of the pixel R7 .

In addition, the first reference map generator 125 may add the corrected pixel value R7b of the pixel R7 according to any one of the above-described methods to the existing first reference map RM1 . In addition, the first reference map generator 125 adds the pixel values of the normal pixels Gr7, R8, and Gr8 of the pixel group PG to the existing first reference map RM1 to create a new first reference map RM1 ( NEW)) can be created.

4 is a conceptual diagram illustrating an image compression method according to an exemplary embodiment of the present disclosure. Specifically, FIG. 4 is a diagram illustrating a compression method for image data IDT of a TETRA pattern of the encoder 120 . Hereinafter, description will be made on the assumption that one pixel group is set to include pixels corresponding to the same reference color and adjacent to each other. Also, an embodiment in which the encoder 120 compresses the pixel group PG including the bad pixels Gb5 and the normal pixels Gb6, Gb7, and Gb8 will be described. Meanwhile, in the description of FIG. 4 , content overlapping with the description of FIG. 3 will be omitted.

The bad pixel detector 121 may detect whether the target pixel is a bad pixel based on a pixel value of at least one first pixel adjacent to the target pixel in the image data IDT. In some embodiments, the bad pixel detector 121 may detect whether the target pixel is a bad pixel based on pixel values of at least one first pixel that corresponds to the same reference color as that of the target pixel and is adjacent to the target pixel.

For example, referring to FIG. 4 , the bad pixel detector 121 corresponds to the same green color as the pixel Gr5 serving as the target pixel in the image data IDT, and corresponds to the first pixels adjacent to the target pixel (eg, The average pixel value of Gb6, Gb7, Gb8) can be calculated. In addition, the bad pixel detector 121 may calculate a difference value between the calculated average pixel value and the pixel value of the target pixel. When the calculated difference value exceeds the threshold value, the bad pixel detector 121 may detect the target pixel as a bad pixel. Meanwhile, the bad pixel detection operation of the bad pixel detector 121 is not limited to the above-described example, and various methods may be applied.

The compressor 123 may identify a bad pixel in a pixel group based on the bad pixel information received from the bad pixel detector 121 , and may compress pixel values of pixels other than the bad pixel.

Specifically, the compressor 123 identifies at least one second pixel that corresponds to the same reference color as a target pixel that is not a bad pixel in the first reference map RM1 and is adjacent to the target pixel (or pixel group PG). can In addition, the compressor 123 may determine a reference value based on the checked pixel values of the second pixel. In addition, the compressor 123 may compress the pixel value of the target pixel based on the reference value and the pixel value of the target pixel.

In some embodiments, when the number of second pixels is plural, the compressor 123 may determine a pixel value of a pixel at a specific position (eg, the left, upper, or diagonal side of the target pixel) among the second pixels as a reference value. . Alternatively, the compressor 123 may calculate an average pixel value of pixel values of a pixel group compressed before the currently compressed pixel group PG, and determine the calculated average pixel value as a reference value. In addition, the compressor 123 may calculate a difference value RES between the reference value and the pixel value of the target pixel, and generate a bitstream BS including the calculated difference value RES.

The compressor 123 may calculate a plurality of difference values RES by repeating the above-described operation for all pixels in the pixel group PG, and the bitstream BS including the plurality of difference values RES can create

For example, referring to FIG. 4 , the compressor 123 corresponds to the same reference color as the pixel Gb6 that is not a bad pixel in the first reference map RM1, and the second pixels adjacent to the pixel Gb6 ( For example, Gb1, Gb2, Gb3, Gb4) can be identified. In addition, the compressor 123 may determine an average pixel value of the checked second pixels (eg, Gb2, Gb4, and Gr7) as a reference value. In addition, the compressor 123 may calculate a difference value RES between the reference value and the pixel value of the pixel Gb6 and generate a bitstream BS including the calculated difference value RES.

When the compression of the pixel group PG is completed, the first reference map generator 125 adds pixel values corresponding to the compressed pixel group PG to the existing first reference map RM1 to create a new first reference. A map RM1 (NEW) can be created. For example, the first reference map generator 125 restores the pixel values of the pixel group PG by decoding the bitstream BS received from the compressor 123, and uses the restored pixel values as an existing first reference. In addition to the map RM1, a new first reference map RM1(NEW) may be generated.

In this case, the first reference map generator 125 may add the pixel values of the bad pixels in the pixel group PG to the existing first reference map RM1 after first performing correction.

In some embodiments, the first reference map generator 125 may correct the pixel value of the bad pixel according to various methods based on the pixel values of the third pixels adjacent to the bad pixel. Specifically, the first reference map generator 125 may identify at least one third pixel adjacent to the bad pixel by corresponding to the same reference color as the bad pixel. In addition, the first reference map generator 125 may correct the pixel value of the bad pixel based on the checked pixel value of the third pixel.

For example, referring to FIG. 4 , the first reference map generator 125 corresponds to the same green color as that of a pixel Gb5 that is a bad pixel, and a third pixel (eg, Gb2) located to the left of the pixel Gb5. The pixel value of the pixel Gb5 may be corrected with the pixel value of . As another example, the first reference map generator 125 may correct the pixel value of the pixel Gb5 with the pixel value of the third pixel positioned above the pixel Gb5 . As another example, the first reference map generator 125 may correct the pixel value of the pixel Gb5 with the pixel value of the third pixel (eg, Gb6) located to the right of the pixel Gb5. As another example, the first reference map generator 125 may correct the pixel value of the pixel Gb5 with an average pixel value of pixels (eg, Gr4 and Gr7) located diagonally to the pixel Gb5. As another example, the first reference map generator 125 may correct the pixel value of the pixel Gb5 with the average pixel value of the remaining pixels (eg, Gb6, Gb7, Gb8) that are not bad pixels in the pixel group PG. can

In addition, the first reference map generator 125 may add the corrected pixel value Gb5b of the target pixel according to any one of the above-described examples to the existing first reference map RM1 . In addition, the first reference map generator 125 adds the pixel values of the normal pixels Gb6, Gb7, and Gb8 of the pixel group PG to the existing first reference map RM1 to create a new first reference map RM1 ( NEW)) can be created.

Meanwhile, in the illustration and description of FIG. 4 , it is assumed that the image data IDT is a TETRA pattern, but the present disclosure is not limited thereto. For example, even when the image data IDT is a NONA pattern, substantially the same method as the above-described method may be applied.

5A and 5B are diagrams illustrating an encoded bitstream according to an exemplary embodiment of the present disclosure. Specifically, FIGS. 5A and 5B are diagrams illustrating a bitstream BS generated by compressing pixel values of pixels included in one pixel group PG. Hereinafter, for easy understanding, an embodiment in which one pixel group PG includes four pixels P1 to P4 and one pixel is configured by 10 bits will be described.

5A and 5B , the bitstream BS may consist of 20 bits, 4 bits indicating the header HEADER, 4 bits indicating the bad pixel flag BP FLAG, and a difference value RES may include 12 bits representing The header may include information related to compression of the image data IDT. For example, the header may include information such as a compression method, a compression mode, a compression rate, and loss information. The decoder 230 may check the header HEADER and decompress the bitstream BS according to a method corresponding to the header HEADER.

The bad pixel flag BP FLAG may include information indicating a pixel that is a bad pixel in the pixel group PG. Specifically, each of the 4 bits constituting the bad pixel flag BP FLAG may correspond to each of the pixels P1 to P4 of the pixel group PG. For example, the first bit of the bad pixel flag BP FLAG corresponds to the first pixel P1, the second bit corresponds to the second pixel P2, and the third bit corresponds to the third pixel P3, and , the fourth bit may correspond to the fourth pixel P4 . In addition, when each of the 4 bits of the bad pixel flag BP FLAG has a specific value (eg, 1 or 0), it may mean that a pixel corresponding to the corresponding bit is a bad pixel.

The difference value RES may include difference values between normal pixels in the pixel group PG. Here, the difference value of the normal pixel indicates a difference value between the reference value determined based on the first reference map RM1 described above with reference to FIGS. 3 and 4 and the pixel value of the target pixel (ie, the normal pixel). If the difference value between the reference value and the pixel value of the target pixel is not included in the range that can be expressed by bits allocated in the bitstream BS, the LSB (Least Significant Bit) of the difference value may be removed. When the difference value falls within the expressible range, the difference value may be included in the bitstream BS.

According to an exemplary embodiment of the present disclosure, as the number of bad pixels in the pixel group PG increases, the number of bits allocated to a difference value between normal pixels may also increase. For example, referring to FIG. 5A , since the fourth pixel P4 has one bad pixel in the pixel group PG, the difference values RES1, RES2, and P3 between the remaining three normal pixels P1, P2, and P3 are RES3) may be composed of 12 bits. For example, 4 bits may be allocated to each of the difference values RES1 , RES2 , and RES3 . Meanwhile, referring to FIG. 5B , since there are two bad pixels in the pixel group PG, the third pixel P3 and the fourth pixel P4, the difference values RES1 between the remaining two normal pixels P1 and P2. , RES2) may consist of 12 bits. For example, 6 bits may be allocated to each of the difference values RES1 and RES2. Since the range of the difference value increases as the number of allocated bits increases, data loss can be reduced. Meanwhile, the number of bits allocated to each of the difference values of the normal pixels may be the same or different from each other.

The decoder 230 may restore pixel values of the pixel group PG based on the bitstream BS. Specifically, first, the decoder 230 may restore pixel values of normal pixels based on the difference value RES. In addition, the decoder 230 may restore the pixel values of the bad pixels based on the restored pixel values of the normal pixels. A detailed description of the restoration operation of the decoder 230 will be described later with reference to FIG. 9 .

Meanwhile, the number of bits constituting each of the header HEADER, the bad pixel flag BP FLAG and the difference value RES is not limited to the above-described example, and the number of bits may be set differently depending on the embodiment. is of course

6 is a diagram illustrating an encoded bitstream according to an exemplary embodiment of the present disclosure. Specifically, FIG. 6 is a view showing a deformable embodiment of FIG. 5 . In the description of FIG. 6 , content overlapping with the description of FIG. 5 will be omitted.

Referring to FIG. 6 , the bitstream BS may consist of 20 bits, 4 bits indicating the header HEADER, 2 bits indicating the bad pixel flag BP FLAG, and 14 bits indicating the difference value RES It may contain bits.

The bitstream BS of FIG. 6 may be applied when one bad pixel is included in the pixel group PG, and 2 bits constituting the bad pixel flag BP FLAG are 4 bits in the pixel group PG. It may have a value indicating a bad pixel among the pixels P1 to P4 . For example, if the first pixel P1 is a bad pixel, the bad pixel flag BP FLAG may have a value of 00, and if the second pixel P2 is a bad pixel, the bad pixel flag BP FLAG may have a value of 01. may have a value, and if the third pixel P3 is a bad pixel, the bad pixel flag BP FLAG may have a value of 10, and if the fourth pixel P4 is a bad pixel, the bad pixel flag BP FLAG is It can have a value of 11.

According to the present embodiment, since only 2 bits are allocated to the bad pixel flag BP FLAG, more bits may be allocated to the difference value RES than in the embodiment of FIG. 5A . Accordingly, data loss of the difference value RES may be reduced. Meanwhile, in FIG. 6 , the difference value RES1 of the first pixel P1 is 5 bits, the difference value RES2 of the second pixel P2 is 5 bits, and the difference value RES3 of the third pixel P3 is 5 bits. Although it is shown that 4 bits are allocated, the present disclosure is not limited thereto, and it goes without saying that the number of bits may be set differently according to embodiments.

5A to 6 , since 40 bits constituting the pixel group PG are compressed to 20 bits of the bitstream BS, the image data IDT may be compressed at a compression rate of 50%. Meanwhile, the present disclosure is not limited thereto, and the number of bits of the bitstream BS may be set to be less than or greater than 20 bits, and accordingly, the compression rate may be increased or decreased.

7 is a flowchart illustrating an image compression method according to an exemplary embodiment of the present disclosure. Specifically, FIG. 7 is a flowchart illustrating an image compression method of the image processing system 10 of FIG. 1 . At least one of the steps of FIG. 7 may be performed by the encoder 120 .

1 and 7 , the encoder 120 may detect a bad pixel in the pixel group for each of a plurality of pixel groups constituting the image data IDT ( S110 ). Specifically, the encoder 120 may detect a bad pixel based on at least one of pixel values of pixels within a pixel group and pixels adjacent to a pixel group in the image data IDT. For example, the encoder 120 may calculate average pixel values of pixels adjacent to the target pixel with respect to a target pixel to be detected as a bad pixel among pixels included in the pixel group. In addition, the encoder 120 may detect whether the target pixel is a bad pixel based on a difference value between the calculated average pixel value and the pixel value of the target pixel. For example, when the difference value between the average pixel value and the pixel value of the target pixel exceeds a threshold value, the encoder 120 may detect the target pixel as a bad pixel.

In addition, the encoder 120 may generate a flag indicating location information of the bad pixel ( S120 ). Here, the flag may be implemented to have a value corresponding to the position of at least one bad pixel included in the pixel group. In addition, the encoder 120 may calculate difference values between the reference pixel values and the remaining pixel values except for the bad pixels in the pixel group ( S130 ). Specifically, the encoder 120 may determine the reference pixel value based on reference information (eg, the first reference map RM1 ) including pixel values corresponding to pixels compressed before the pixel group. For example, the encoder 120 may determine the reference pixel value based on the pixel value of at least one pixel adjacent to the pixel group in the reference information. A detailed description of the method of determining the reference pixel value has been described above with reference to FIGS. 3 and 4 , and thus a redundant description thereof will be omitted.

And the encoder 120 may generate a bitstream including the flag and difference values (S140). In this case, the encoder 120 may generate compression information indicating the compression method applied to the pixel group. And the encoder 120 may include the generated compression information in the bitstream. For example, the encoder 120 may include the compression information in a header of the bitstream.

In addition, after generating the bitstream, the encoder 120 may update reference information based on pixel values corresponding to pixels in a pixel group. Specifically, the encoder 120 may correct a pixel value of a bad pixel in a pixel group. For example, the encoder 120 may correct a pixel value of a bad pixel based on a pixel value of a pixel adjacent to the bad pixel. A detailed description of the method of correcting the pixel value of the bad pixel has been described with reference to FIGS. 3 and 4 , and thus a redundant description thereof will be omitted. In addition, the encoder 120 may update the reference information by adding the pixel values of pixels other than the bad pixels in the pixel group and the correction pixel values of the bad pixels to the reference information.

8 is a diagram illustrating a decoder according to an exemplary embodiment of the present disclosure. Specifically, FIG. 8 is a diagram illustrating the decoder 230 of FIG. 1 . 1 and 8 , the decoder 230 may include a decompressor 231 and a second reference map generator 233 .

The decompressor 231 receives the compressed data CDT and the second reference map RM2 from the memory 220 , and decompresses the compressed data CDT by using the second reference map RM2 , thereby compressing the compressed data CDT. Release data DDT may be generated. Specifically, the decompressor 231 sequentially converts the plurality of bitstreams BS included in the compressed data CDT using the second reference map RM2 generated based on the pixel values of the first decompressed pixels. can be decompressed. A detailed description thereof will be described later with reference to FIGS. 9A and 9B .

When the decompression of one bitstream BS is completed, the decompressor 231 may provide the restored pixel information DP including pixel values of the decompressed pixels to the second reference map generator 233 . have. In addition, the decompressor 231 identifies a bad pixel through the bad pixel flag BP FLAG of the bitstream BS, and returns bad pixel information BP for the checked bad pixel to the second reference map generator 233 . can be provided to

The second reference map generator 233 may generate a new second reference map RM2(NEW) based on the received reconstructed pixel information DP and the bad pixel information BP. Specifically, the second reference map generator 233 may identify a bad pixel in a reconstructed pixel group based on the bad pixel information BP. In addition, the second reference map generator 233 may correct the pixel value of the bad pixel among the reconstructed pixel values to be similar to the pixel values of the neighboring pixels. In addition, the second reference map generator 233 adds the restored pixel values of the normal pixels and the corrected pixel values of the bad pixels in the pixel group to the existing second reference map RM2 to create a new second reference map RM2(NEW). )) can be created. A detailed description thereof will be described later with reference to FIGS. 9A and 9B .

The second reference map generator 233 may store the generated second reference map RM2(NEW) in the memory 220 . The decompressor 231 reads the new second reference map RM2(NEW) stored in the memory 220, and based on the read second reference map RM2(NEW)) Decompression can be performed.

Meanwhile, each of the decompressor 231 and the second reference map generator 233 may be implemented as software, hardware, or a combination of software and hardware such as firmware. When the decompressor 231 and the second reference map generator 233 are implemented as software, each of the above-described functions is implemented as a programmed source code and loaded into a storage medium provided in the image processing apparatus 200, respectively. The functions of the decompressor 231 and the second reference map generator 233 may be implemented as a processor (eg, a microprocessor) provided in each of the image processing apparatus 200 executes software. When the decompressor 231 and the second reference map generator 233 are implemented in hardware, the decompressor 231 and the second reference map generator 233 may include logic circuits and registers, and register setting Based on the above, each of the above-described functions may be performed.

9A and 9B are conceptual diagrams illustrating an image decompression method according to an exemplary embodiment of the present disclosure. Specifically, FIGS. 9A and 9B are diagrams illustrating a decompression method for image data IDT of a BAYER pattern of the decoder 230 . Hereinafter, description will be made on the assumption that one pixel group is set to include four sequentially arranged pixels. In addition, the decoder 230 decompresses the sixth bitstream BS6 corresponding to the pixel group PG including the pixels R7, Gr7, R8, and Gr8, and the pixel R7 is a bad pixel. Explain.

Referring to FIG. 9A , the decompressor 231 may decompress the sixth bitstream BS6 using the second reference map RM2 . In this case, the second reference map RM2 may include pixel values of the first to fifth bitstreams BS1 to BS5 decompressed before the sixth bitstream BS6 .

Specifically, the decompressor 231 may identify a bad pixel based on the bad pixel flag BP FLAG of the sixth bitstream BS6. In addition, the decompressor 231 may perform decompression from normal pixels other than the bad pixels.

In some embodiments, the decompressor 231 may select at least one pixel adjacent to a target pixel that is a normal pixel of the pixel group PG or the pixel group PG from the second reference map RM2 . In addition, the decompressor 231 may determine a reference value based on the pixel value of the selected pixel. In this case, the method of determining the reference value may correspond to the method in which the compressor 123 determines the reference value in the process of generating the sixth bitstream BS6. For example, when the compressor 123 determines a pixel at a specific position (eg, to the left of the target pixel) among pixels adjacent to the target pixel in the first reference map RM1 as a reference value, the decompressor 231 is The reference value may be determined by referring to a pixel at a specific position (eg, a left side of the target pixel) among pixels adjacent to the target pixel in the second reference map RM2 . According to an embodiment, information on a method in which the compressor 123 determines the reference value may be implemented to be included in the header HEADER of the bitstream BS.

In addition, the decompressor 231 may restore the pixel value of the target pixel based on a difference value RES between the determined reference value and the target pixel included in the sixth bitstream BS6. Specifically, the decompressor 231 may restore the pixel value of the target pixel by adding the difference value RES of the target pixel to the determined reference value. For example, when the target pixel is the pixel Gr7, the decompressor 231 may restore the pixel value of the target pixel Gr7 by adding the difference value RES of the target pixel Gr7 to the reference value. can

When restoration of normal pixels is completed, the decompressor 231 may restore pixel values of bad pixels. Meanwhile, the difference value RES of the bad pixel may not be included in the sixth bitstream BS6. Accordingly, the decoder 230 may restore the pixel value of the bad pixel by calculating an average pixel value of the pixel values of the restored normal pixels and adding a threshold value to the calculated average pixel value. Here, the threshold value may be the same as the threshold value used in the bad pixel detection operation described above in FIG. 3 , but the present disclosure is not limited thereto, and various values may be set as the threshold value.

For example, when the bad pixel is the pixel R7, the decoder 230 calculates an average pixel value of pixel values of the restored normal pixels Gr7, R8, and Gr8, and uses the calculated average pixel value as a threshold value. By adding , the pixel value of the pixel R7 that is a bad pixel may be restored. Meanwhile, the method in which the decoder 230 restores the bad pixel is not limited to the above-described example, and the bad pixel may be restored in various ways.

When the decompression of the pixel group PG is completed, the second reference map generator 233 adds the pixel values of the decompressed pixel group PG to the existing second reference map RM2 for a new second reference. A map RM2(NEW) may be created.

Specifically, if a bad pixel is not included in the pixel group, the second reference map generator 233 adds decompressed pixel values to the existing second reference map RM2 to create a new second reference map RM2 ( NEW)) can be created.

On the other hand, when a bad pixel is included in the pixel group, the second reference map generator 233 may correct the pixel value of the bad pixel among the restored pixel values based on the pixel values of the pixels adjacent to the bad pixel. The method of correcting the pixel value of the bad pixel may be performed similarly to the method of correcting the pixel value of the bad pixel described above with reference to FIG. 3 .

For example, referring to FIG. 9B , the second reference map generator 233 corresponds to the same red color as the pixel R7, which is the target pixel, which is a bad pixel, and a pixel (eg, R3) located above the pixel R7. ), the pixel value of the pixel R7 may be corrected. As another example, the second reference map generator 233 may correct the pixel value of the pixel R7 with the pixel value of the pixel (eg, R6 ) located to the left of the pixel R7 . As another example, the second reference map generator 233 may correct the pixel value of the pixel R7 with the pixel value of the pixel (eg, R8 ) located to the right of the pixel R7 . As another example, the second reference map generator 233 may correct the pixel value of the pixel R7 with an average pixel value of pixels (eg, R2 and R4 ) located on a diagonal side of the pixel R7 .

In addition, the second reference map generator 233 generates a new second reference map RM2(NEW) by adding the pixel values of the restored normal pixels and the corrected pixel values of the bad pixels to the existing second reference map RM2. can do.

As such, when one pixel group PG is decompressed, the second reference map generator 233 sets a pixel value corresponding to the pixel group PG (ie, the pixel group ( A new second reference map RM2(NEW) may be generated by adding the pixel values of the restored normal pixel and the corrected pixel value of the bad pixel) of PG).

10 is a diagram illustrating an image processing system according to an exemplary embodiment of the present disclosure. Specifically, FIG. 10 is a diagram illustrating a deformable embodiment of the image processing system 10 of FIG. 1 .

Referring to FIG. 10 , the image processing system 10a may include a camera module 100a and an image processing apparatus 200a. In some embodiments, the camera module 100a may include an image sensor 110a, an encoder 120a, a memory 130a, and an interface 140a. In some embodiments, the image processing apparatus 200a may include an interface 210a, a memory 220a, a decoder 230a, and an image signal processor 240a.

Compared with the image system 10 of FIG. 1 , in the image system 10b of FIG. 10 , a mode selector 127a is added to the encoder 120a, and the remaining components may be substantially the same. A description of components overlapping those of the image processing system 10 of FIG. 1 among the components of the image processing system 10a will be omitted.

According to an exemplary embodiment of the present disclosure, the encoder 120a may generate a plurality of compressed data CDTs by compressing the image data IDT in a plurality of compression modes. Here, each of the plurality of compression modes may be set based on a compression target, a compression rate, an error rate, and/or loss information. In addition, the encoder 120a may include mode information in the header of the bitstream during compression. A detailed description of the plurality of compression modes will be described later with reference to FIGS. 12A and 12B .

For example, the encoder 120 generates a first reference map RM1 including pixel values of uncorrected original bad pixels, and during a compression operation using the first reference map RM1, the first reference map The image data IDT may be compressed in a first mode in which a bad pixel detection operation for RM1 is performed. In this case, information indicating the first mode may be included in the header of the bitstream.

In addition, the encoder 120a generates the first reference map RM1 including the corrected pixel values of the bad pixels according to the method described above in FIGS. 1 to 9 , and detects the bad pixels for the first reference map RM1 . The image data IDT may be compressed in the second mode in which a compression operation is performed without an operation. In this case, information indicating the second mode may be included in the header of the bitstream.

Meanwhile, the present disclosure is not limited thereto, and the encoder 120a may compress the image data IDT in three or more modes. Also, it goes without saying that the compression method of each of the plurality of modes may be set differently from the above-described example.

The mode selector 127a of the encoder 120a may select one of a plurality of modes and transmit compressed data CDT corresponding to the selected mode to the image processing apparatus 200a through the interface 140a. In some embodiments, the mode selector 127a may select one of the plurality of modes based on error data of the plurality of modes. Here, the error data means data representing a difference between data obtained by reconstructing compressed data CDT of a specific mode and image data IDT. The mode selector 127a may select a mode having the smallest error based on the error data of the plurality of modes, and transmit compressed data CDT corresponding to the selected mode to the image processing apparatus 200a.

According to an exemplary embodiment of the present disclosure, the decoder 230a may generate a plurality of decompressed data DDT by decompressing the compressed data CDT in a plurality of modes. Specifically, the decoder 230a may check the mode information based on the header of the bitstream included in the compressed data CDT. In addition, the decoder 230a may decompress the bitstream according to a decompression method corresponding to the checked mode information.

For example, when the header of the bitstream includes mode information indicating the aforementioned first mode, the decoder 230a may decompress the bitstream according to a decompression method corresponding to the first mode. Also, when the header of the bitstream includes mode information indicating the above-described second mode, the decoder 230a may decompress the bitstream according to a decompression method corresponding to the second mode.

11 is a diagram illustrating an encoder according to an exemplary embodiment of the present disclosure.

Specifically, FIG. 11 is a diagram illustrating the encoder 120a of FIG. 10 . 10 and 11 , the encoder 120a may include a bad pixel detector 121a, a compressor 123a, a first reference map generator 125a, and a mode selector 127a.

Referring to FIG. 11 , the bad pixel detector 121a , the compressor 123a , and the first reference map generator 125a compress image data IDT according to a compression method corresponding to each of a plurality of modes. Compressed data (CDT) can be generated. In addition, the compressor 123a may generate the error data ED based on the plurality of compressed data CDTs. In some embodiments, the bad pixel detector 121a, the compressor 123a, and/or the first reference map generator 125a may compress the image data IDT for each of a plurality of modes under the control of the mode selector 127a. have.

Specifically, the compressor 123a may generate restored data by restoring each compressed data CDT for each of the plurality of compressed data CDTs, and may calculate an error between the restored data and the image data IDT. In addition, the compressor 123a may generate error data ED including errors of a plurality of modes. In addition, the compressor 123a may transmit the error data ED to the mode selector 127a.

The mode selector 127a may identify a mode having the lowest error based on the error data ED, and transmit compressed data CDT corresponding to the checked mode to the image processing apparatus 200a through the interface 140a. .

Meanwhile, according to a deformable embodiment, the compressor 123a may generate the error data ED in units of pixel groups. Specifically, the compressor 123a may calculate an error between each of the n pixel groups (n is a positive integer) constituting the image data IDT and each of the n pixel groups constituting the restored data. In addition, the compressor 123a may generate error data ED including errors of a plurality of modes of each pixel group. The mode selector 127a may identify a mode having the lowest error based on the error data ED for each pixel group, and transmit a bitstream corresponding to the checked mode to the image processing apparatus 20aa.

Meanwhile, in the above-described example, the bad pixel detector 121a, the compressor 123a, and the first reference map generator 125a perform compression according to a compression method corresponding to each of the plurality of modes to obtain a plurality of compressed data (CDT). Although illustrated and described as generating a , the present disclosure is not limited thereto. For example, the encoder 120a may be implemented to include a bad pixel detector 121a, a compressor 123a, and/or a first reference map generator 125a corresponding to each of a plurality of modes. For example, the encoder 120a includes a bad pixel detector 121a corresponding to the first mode, a compressor 123a and/or a first reference map generator 125a, and a bad pixel detector corresponding to the second mode. It may be implemented to include a 121a, a compressor 123a, and/or a first reference map generator 125a.

In addition, although the mode selector 127a is illustrated and described as being included in the encoder 120a with reference to FIGS. 10 and 11 , the present disclosure is not limited thereto. For example, the mode selector 127a may be implemented in a configuration separate from the encoder 120a in the camera module 100a.

12A and 12B are tables for explaining compressed information according to an exemplary embodiment of the present disclosure. In detail, FIGS. 12A and 12B illustrate a compression mode (compression method) according to a standard proposed by the Mobile Industry Processor Interface (MIPI) coalition. 10 and 11 are referred to together.

Referring to FIG. 12A , image data (IDT of FIG. 3 ) of a Bayer pattern may be compressed according to various compression modes. As a compression mode, PD (Pixel-based Directional Differential) mode, DGD (DiaGonal Direction-based Differential) mode, eSHV (extended Slanted Horizontal or Vetical Direction-based Differential) mode, OUT (OUTlier compensation) mode, FNR (Fixed quantization and FNR) mode No-Reference) mode is used. Meanwhile, the names of the above-described compression modes are merely examples, and the present disclosure is not limited to the above-described examples.

In the PD mode, differential pulse code modulation (DPCM) may be performed on the image data (IDT) of the Bayer pattern. The PD mode can be divided into MODE0, MODE1, MODE2, MODE3, MODE12, and MODE13 according to the detailed implementation algorithm. Since 4 bits may be allocated to the header indicating the compression method, the 16 compression modes may express header information as different bits. For example, MODE0 may be expressed as bit 0000, MODE1 as bit 0001, MODE2 as bit 0010, MODE3 as bit 0011, MODE12 as bit 1100, and MODE 13 as bit 1101, respectively.

In the DGD mode, DPCM may be performed on diagonal image data (IDT). DGD mode can be divided into MODE4 (bit 0100), MODE5 (bit 0101), MODE8 (bit 1000), MODE9 (bit 1001), MODE10 (bit 1010), MODE11 (bit 1011) according to the detailed implementation algorithm. .

Similarly, the eSHV mode may include MODE14 (bit 1110) and MODE15 (1111), the OUT mode may include MODE7 (bit 0111), and the FNR mode may include MODE6 (bit 01110) . According to an exemplary embodiment of the present disclosure, MODE7 may refer to an OUT mode including a BP mode for processing a bad pixel, or a saturation mode, but one of two modes (saturation mode, or OUT mode) according to an operating environment. Either one can be selected.

In an exemplary embodiment, the mode selector 127a may sequentially evaluate the PD mode, DGD mode, HL mode, SAT mode, and FNR mode, and select the optimal mode according to compression evaluation indicators such as compression ratio and loss information. . However, the technical spirit of the present disclosure is not limited to the presented mode evaluation order.

Referring to FIG. 12B , image data of a tetra pattern ( FIG. 4 , IDT) may be compressed according to various compression modes. Meanwhile, the present disclosure is not limited thereto, and a red pixel group, a blue pixel group, a first green pixel group, and a second pixel group including pixels arranged in 2n X 2n or 3n X 3n (n is a positive integer) are Image data that is repeatedly arranged may be compressed according to various compression modes.

As a compression mode, AD (Average-based Directional Differential) mode, eHVD (extended Horizontal or Vertial Direction-based Differential) mode, OD (Oblique Direction-based Differential) mode, eMPD (extended Multi-Pixel-based Differential) mode, eHVA An (extended Horizontal or Vertical Average-based Differential) mode, an extended OUTlier compensation (eOUT) mode, and an FNR mode are used.

In the AD mode, DPCM may be performed on image data IDT in which one pixel group PG constituting the Bayer pattern includes a plurality of pixels. AD mode can be divided into MODE0, MODE1, MODE2, MODE3 according to the detailed implementation algorithm. Since 4 bits may be allocated to the header indicating the compression method, the 16 compression modes may express header information as different bits. For example, MODE0 may be expressed as bit 0000, MODE1 as bit 0001, MODE2 as bit 0010, and MODE3 as bit 0011, respectively.

The OD mode may compress diagonal image data (IDT). The OD mode may be divided into MODE4 (bit 0100) and MODE5 (bit 0101) according to a detailed implementation algorithm.

Similarly, the eMPD mode may include MODE8 (bit 1000), MODE9 (bit 1001), MODE10 (bit 1010) and MODE11 (bit 1011), and the eHVD mode may include MODE12 (bit 1100) and MODE 13 (bit 1101). ), the eHVA mode may include MODE14 (bit 1110), (e) the OUT mode (eOUT mode or OUT mode) may include MODE15 (1111) and MODE7 (0111), FNR The mode may include MODE6 (bit 01110). According to an exemplary embodiment of the present disclosure, MODE7 may refer to (e) OUT mode including a BP mode for processing bad pixels, or a saturation mode, but may refer to two modes (saturation mode, or ( e) OUT mode) may be selected.

In an exemplary embodiment, the mode selector 230 may sequentially evaluate the AD mode, the eHVD mode, the OD mode, the eMPD mode, the eHVA mode, the eOUT mode, and the FNR mode, and according to the compression evaluation index such as compression ratio, loss information, etc. You can select the optimal mode. However, the technical spirit of the present disclosure is not limited to the presented mode evaluation order.

13 is a diagram illustrating an image processing system according to an exemplary embodiment of the present disclosure. Specifically, FIG. 13 is a diagram illustrating a deformable embodiment of the image processing system 10b of FIG. 1 .

Referring to FIG. 13 , the image processing system 10b may include a camera module 100b and an image processing apparatus 200b. In some embodiments, the camera module 100b may include an image sensor 110b and an interface 140b. In some embodiments, the image processing apparatus 200b may include an interface 210b, a memory 220b, a decoder 230b, an image signal processor 240b, and an encoder 250b. The encoder 250b of FIG. 13 may correspond to the encoder 120 of FIG. 1 or the encoder 120a of FIG. 10 .

Comparing the image processing system 10b of FIG. 13 and the image processing system 10 of FIG. 1 , the difference is that the encoder 250b is included in the image processing device 200b instead of the camera module 100b, and the rest The configurations may be substantially the same. A description of components overlapping those of the image processing system 10 of FIG. 1 among components of the image processing system 10b will be omitted.

Referring to FIG. 13 , the image sensor 110b may generate image data IDT. The image data IDT may be transmitted to the image processing apparatus 200b through the interface 140b. Meanwhile, according to an embodiment, the camera module 100b may further include an image signal processor (not shown), the image data IDT may be image-processed by the image signal processor, and the processed image data IDT ) may be transmitted to the image processing apparatus 200b.

The image processing apparatus 200b may receive the image data IDT through the interface 210b. The encoder 250b may generate compressed data CDT by compressing the image data IDT using the first reference map RM1 stored in the memory 220b. Since the method for the encoder 250b to generate the compressed data CDT is substantially the same as that described above with reference to FIGS. 1 to 11 , a redundant description thereof will be omitted. The compressed data CDT may be stored in the memory 220b and may be read from the memory 220b by the decoder 230b. The decoder 230b may generate decompressed data DDT by decompressing the image data IDT using the second reference map RM2 stored in the memory 220b. Since the method for the decoder 230b to generate the decompressed data DDT is substantially the same as that described above with reference to FIGS. 1 to 11 , a redundant description thereof will be omitted.

14 is a diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure.

Referring to FIG. 14 , the electronic device 1000 includes a camera module 1100 , an application processor 1200 , a display 1300 , a memory 1400 , a storage 1500 , a user interface 1600 , and a wireless transceiver 1700 . ) may be included. The camera module 1100 of FIG. 14 may correspond to the camera module 100 of FIG. 1 , the camera module 100a of FIG. 10 , or the camera module 100b of FIG. 13 , and the application processor 1200 of FIG. The image processing apparatus 200 of 1 , the image processing apparatus 200a of FIG. 10 , or the image processing apparatus 200b of FIG. 13 may be included. A description overlapping with the above description in FIGS. 1, 10 and 13 will be omitted.

The application processor 1200 controls the overall operation of the electronic device 1000 and may be provided as a system-on-chip (SoC) that drives an application program, an operating system, and the like.

The memory 1400 may store programs and/or data processed or executed by the application processor 1200 . The storage 1500 may be implemented as a non-volatile memory device such as NAND flash or resistive memory. For example, the storage 1500 may be provided as a memory card (MMC, eMMC, SD, micro SD) or the like. The storage 1500 may store data and/or a program for an execution algorithm that controls the image processing operation of the application processor 1200 , and the data and/or program is loaded into the memory 1400 when the image processing operation is performed. can be

The user interface 1600 may be implemented with various devices capable of receiving a user input, such as a keyboard, a curtain key panel, a touch panel, a fingerprint sensor, and a microphone. The user interface 1600 may receive a user input and provide a signal corresponding to the received user input to the application processor 1200 . The wireless transceiver 1700 may include a modem 1710 , a transceiver 1720 , and an antenna 1730 .

15 is a diagram illustrating a part of an electronic device according to an exemplary embodiment of the present disclosure. 16 is a diagram illustrating a detailed configuration of a camera module according to an exemplary embodiment of the present disclosure. Specifically, FIG. 15 is a diagram illustrating the electronic device 2000 as a part of the electronic device 1000 of FIG. 16 , and FIG. 16 is a diagram illustrating a detailed configuration of the camera module 2100b of FIG. 15 .

Referring to FIG. 15 , the electronic device 2000 may include a multi-camera module 2100 , an application processor 2200 , and a memory 2300 . The memory 2300 may perform the same function as the memory 1400 shown in FIG. 14 , and thus a redundant description will be omitted.

The electronic device 2000 may capture and/or store an image of a subject using a CMOS image sensor, and may be implemented as a mobile phone, a tablet computer, or a portable electronic device. The portable electronic device may include a laptop computer, a mobile phone, a smart phone, a tablet PC, a wearable device, and the like.

The multi-camera module 2100 may include a first camera module 2100a, a second camera module 2100b, and a third camera module 2100c. The multi-camera module 2100 may perform the same function as the camera module 100 of FIG. 1 , the camera module 100a of FIG. 10 , or the camera module 100b of FIG. 13 . Meanwhile, although FIG. 15 illustrates that the multi-camera module 2100 includes three camera modules 1100a to 1100c, the present disclosure is not limited thereto, and various number of camera modules may be included in the multi-camera module 2100. can

Hereinafter, a detailed configuration of the camera module 2100b will be described in more detail with reference to FIG. 16 , but the following description may be equally applied to other camera modules 2100a and 2100c according to an embodiment.

Referring to FIG. 16 , the second camera module 2100b includes a prism 2105 , an optical path folding element (hereinafter, ½ OPFE½) 2110 , an actuator 2130 , an image sensing device 2140 and a memory. (2150).

The prism 2105 may include the reflective surface 2107 of the light reflective material to modify the path of the light L incident from the outside.

According to an exemplary embodiment, the prism 2105 may change the path of the light L incident in the first direction (X) to the second direction (Y) perpendicular to the first direction (X). In addition, the prism 2105 rotates the reflective surface 2107 of the light reflective material in the A direction about the central axis 2106 or rotates the central axis 2106 in the B direction in the first direction (X). The path of the incident light L may be changed in the second vertical direction Y. At this time, the OPFE 2110 may also move in a third direction (Z) perpendicular to the first direction (X) and the second direction (Y).

The OPFE 2110 may include, for example, an optical lens consisting of m (here, m is a natural number) number of groups. The m lenses may move in the second direction Y to change an optical zoom ratio of the camera module 2100b.

The actuator 2130 may move the OPFE 2110 or an optical lens (hereinafter, referred to as an optical lens) to a specific position. For example, the actuator 2130 may adjust the position of the optical lens so that the image sensor 2142 is located at a focal length of the optical lens for accurate sensing.

The image sensing device 2140 may include an image sensor 2142 , control logic 2144 , an encoder 2145 , and a memory 2146 . The image sensor 2142 may sense an image of a sensing target using light L provided through an optical lens. Since the image sensor 2142 of FIG. 16 may be functionally similar to the image sensor 110 of FIG. 1 , the image sensor 110a of FIG. 10 , or the image sensor 110b of FIG. 13 , a redundant description will be omitted. The control logic 2144 may control the overall operation of the second camera module 2100b.

The encoder 2145 may encode sensed image data. The encoder 2145 of FIG. 16 may perform a function similar to that of the encoder 120 of FIG. 1 , the encoder 120a of FIG. 10 , or the encoder 250b of FIG. 13 , and thus redundant description will be omitted. Meanwhile, for convenience of description, the encoder 2145 is illustrated as an individual functional unit different from other functional units, but is not limited thereto, and may be included in the control logic 2144 to compress and encode image data.

The memory 2146 may store information necessary for the operation of the second camera module 2100b, such as calibration data 2147 . The calibration data 2147 may include information necessary for the second camera module 2100b to generate image data using the light L provided from the outside. The calibration data 2147 may include, for example, information about a degree of rotation described above, information about a focal length, information about an optical axis, and the like. When the second camera module 2100b is implemented in the form of a multi-state camera in which the focal length is changed according to the position of the optical lens, the calibration data 2147 is the focal length for each position (or state) of the optical lens. It may include values and information related to auto focusing.

The memory 2150 may store image data sensed by the image sensor 2142 . In some embodiments, the memory 2150 may store compressed data generated by the encoder 2145 . The memory 2150 may be disposed outside the image sensing device 2140 , and may be implemented in a stacked form with a sensor chip constituting the image sensing device 1140 . In an exemplary embodiment, the memory 2150 may be implemented as an Electrically Erasable Programmable Read-Only Memory (EEPROM), but embodiments are not limited thereto.

15 and 16 together, in an exemplary embodiment, each of the plurality of camera modules 2100a , 2100b , and 2100c may include an actuator 2130 . Accordingly, each of the plurality of camera modules 2100a, 2100b, and 2100c may include the same or different calibration data 2147 according to the operation of the actuator 2130 included therein.

In an exemplary embodiment, one camera module (eg, the second camera module 2100b) of the plurality of camera modules 2100a, 2100b, 2100c includes the prism 2105 and the OPFE 2110 described above. It is a camera module in the form of a folded lens, and the remaining camera modules (eg, 2100a and 2100b) may be a camera module in a vertical form that does not include the prism 2105 and the OPFE 2110. However, embodiments are not limited thereto.

In an exemplary embodiment, one camera module (eg, the third camera module 2100c) of the plurality of camera modules 2100a, 2100b, 2100c uses, for example, Infrared Ray (IR) to provide depth ( It may be a vertical type depth camera that extracts depth) information. In this case, the application processor 2200 merges (eg, the first camera module 2100a or the second camera module 2100b) image data provided from the image data provided from the depth camera with the image data provided from the other camera module (eg, the first camera module 2100a or the second camera module 2100b). merge) to generate a 3D depth image.

In an exemplary embodiment, at least two camera modules (for example, the first camera module 2100a or the second camera module 2100b) among the plurality of camera modules 2100a, 2100b, 2100c have different observation fields ( field of view). In this case, for example, the optical lenses of at least two camera modules (eg, the first camera module 2100a or the second camera module 2100b) among the plurality of camera modules 2100a, 2100b, and 2100c are mutually It may be different, but is not limited thereto. For example, the first camera module 2100a among the plurality of camera modules 2100a, 2100b, and 2100c may have a smaller field of view (FOV) than the second and third camera modules 2100b and 2100c. However, the present invention is not limited thereto, and the multi-camera module 2100 may further include a camera module having a larger field of view (FOV) than the originally used camera modules 2100a, 2100b, and 2100c.

Also, in some embodiments, a viewing angle of each of the plurality of camera modules 2100a, 2100b, and 2100c may be different from each other. In this case, the optical lenses included in each of the plurality of camera modules 2100a, 2100b, and 2100c may also be different, but is not limited thereto.

In some embodiments, each of the plurality of camera modules 2100a, 2100b, and 2100c may be disposed to be physically separated from each other. That is, the plurality of camera modules 2100a, 2100b, and 2100c do not divide and use the sensing area of one image sensor 2142, but an independent image inside each of the plurality of camera modules 2100a, 2100b, 2100c. A sensor 2142 may be disposed.

The application processor 2200 may include a plurality of sub-processors 2210a , 2210b , and 2210c , an image generator 2220 , a camera module controller 2230 , a memory controller 2400 , and an internal memory 2250 . The application processor 2200 may be implemented separately from the plurality of camera modules 2100a, 2100b, and 2100c. For example, the application processor 2200 and the plurality of camera modules 2100a, 2100b, and 2100c may be implemented by being separated from each other as separate semiconductor chips.

Image data or compressed data generated from each of the camera modules 2100a, 2100b, and 2100c is provided to the corresponding sub-processors 2210a, 2210b, and 2210c through image signal lines ISLa, ISLb, and ISLc separated from each other. can Such image data transmission may be performed using, for example, a camera serial interface (CSI) based on MIPI, but embodiments are not limited thereto.

In an exemplary embodiment, one subprocessor may be arranged to correspond to a plurality of camera modules. For example, the first sub-processor 2210a and the third sub-processor 2210c are not implemented separately from each other as shown, but are integrated into one sub-processor, and the camera module 2100a and the camera module 2100c The image data or compressed data provided from ) may be selected through a selection element (eg, a multiplexer) and then provided to the integrated sub-image processor.

Each of the sub-processors 2210a , 2210b , and 2210c may include the decoder 230 of FIG. 1 , the decoder 230a of FIG. 10 , or the decoder 230b of FIG. 13 . The subprocessors 2210a, 2210b, and 2210c may decompress the received compressed data to generate decompressed data, and output the generated decompressed data to the image generator 2220 . The image generator 2220 may correspond to the image signal processor 240 of FIG. 1 , the image signal processor 240a of FIG. 10 , or the image signal processor 240b of FIG. 13 .

The camera module controller 2230 may provide a control signal to each of the camera modules 2100a, 2100b, and 2100c. The control signal generated from the camera module controller 2230 may be provided to the corresponding camera modules 2100a, 2100b, and 2100c through the control signal lines CSLa, CSLb, and CSLc separated from each other.

Exemplary embodiments have been disclosed in the drawings and specification as described above. Although embodiments have been described using specific terms in the present specification, these are used only for the purpose of explaining the technical spirit of the present disclosure and not used to limit the meaning or scope of the present disclosure described in the claims. . Therefore, it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical protection scope of the present disclosure should be defined by the technical spirit of the appended claims.

Claims (20)

An image compression method for compressing each of a plurality of pixel groups constituting image data, the image compression method comprising:
detecting bad pixels in the pixel group;
generating a flag indicating location information of the bad pixel;
calculating difference values between pixel values of pixels other than the bad pixel in the pixel group and a reference pixel value; and
and generating a bitstream including the flag and the difference values. According to claim 1,
The step of detecting the bad pixel comprises:
and detecting a bad pixel based on at least one of pixel values of pixels in the pixel group and pixels adjacent to the pixel group in the image data. 3. The method of claim 2,
The step of detecting the bad pixel comprises:
calculating an average pixel value of pixels adjacent to a target pixel in the pixel group; and
and detecting whether the target pixel is a bad pixel based on a difference value between the average pixel value and a pixel value of the target pixel. According to claim 1,
and determining the reference pixel value based on reference information including pixel values corresponding to pixels compressed before the pixel group. 5. The method of claim 4,
Determining the reference pixel value comprises:
and determining the reference pixel value based on a pixel value of at least one pixel adjacent to the pixel group in the reference information. 5. The method of claim 4,
and updating the reference information based on pixel values corresponding to pixels in the pixel group after generating the bitstream. 7. The method of claim 6,
Updating the reference information includes:
correcting a pixel value of the bad pixel in the pixel group; and
and adding the pixel values of the remaining pixels in the pixel group and the correction pixel value of the bad pixel to the reference information. 8. The method of claim 7,
Compensating the pixel value of the bad pixel in the pixel group comprises:
and correcting a pixel value of the bad pixel based on a pixel value of a pixel adjacent to the bad pixel. According to claim 1,
generating compression information indicating a compression method applied to the pixel group;
The step of generating the bitstream comprises:
Image compression method, characterized in that generating the bitstream additionally including the compression information. In the camera module,
an image sensor generating image data including a plurality of pixels;
an encoder that divides the plurality of pixels into a plurality of pixel groups and sequentially compresses the image data in units of pixel groups to generate compressed data including a plurality of bitstreams; and
a memory for storing reference information including pixel values corresponding to pixels compressed by the encoder;
The encoder is
A bad pixel included in the first pixel group is detected, and pixel values of the first pixels included in the first pixel group are compressed based on the detection result of the bad pixel and the reference information to correspond to the first pixel group. The camera module generates a first bitstream and updates the reference information based on a corrected pixel value obtained by correcting a pixel value of the bad pixel. 11. The method of claim 10,
The encoder is
Correcting the pixel value of the bad pixel based on the pixel value of at least one pixel adjacent to the bad pixel, pixel values of pixels other than the bad pixel among the first pixels and the corrected pixel value of the bad pixel and updating the reference information by adding it to the reference information. 12. The method of claim 11,
The encoder is
and detecting the bad pixel based on at least one of the first pixels and pixel values of pixels adjacent to the first pixels in the image data. 12. The method of claim 11,
The encoder is
In the reference information, a reference pixel value is determined based on at least one pixel adjacent to the first pixels, difference values between pixel values of the remaining pixels and the reference pixel value are calculated, and the calculated difference value Camera module, characterized in that for generating the first bitstream including 14. The method of claim 13,
The encoder is
and generating a flag indicating the position information of the bad pixel, and generating the first bitstream including the flag. 15. The method of claim 14,
The encoder is
A plurality of compressed data is generated by compressing the image data a plurality of times according to a plurality of modes corresponding to a plurality of compression methods, and a mode having the smallest error with the image data is selected based on the plurality of compressed data. A camera module further comprising a mode selector. 16. The method of claim 15,
The encoder is
and generating compressed information including information on the selected mode, and generating the first bitstream including the generated compressed information. An image processing system comprising:
an image sensor generating image data including a plurality of pixels;
an encoder for sequentially compressing a plurality of pixel groups constituting the image data to generate a plurality of bitstreams; and
a decoder that decompresses the plurality of bitstreams to restore the image data;
The encoder is
detecting, for each of the plurality of pixel groups, a bad pixel in the corresponding pixel group;
compressing pixel values of a corresponding pixel group based on reference information generated based on pixel values corresponding to pixels compressed before the corresponding pixel group; and
and updating the reference information based on the bad pixel detection result. 18. The method of claim 17,
The step of detecting the bad pixel comprises:
and detecting the bad pixel based on at least one of pixel values of pixels included in the corresponding pixel group and pixels adjacent to the corresponding pixel group. 19. The method of claim 18,
The compressing step is
generating a flag indicating location information of the bad pixel;
calculating difference values between pixel values of pixels other than the bad pixel in a corresponding pixel group and a reference pixel value determined based on the reference information;
and generating a bitstream including the flag and the difference values. 20. The method of claim 19,
Updating the reference information includes:
correcting a pixel value of the bad pixel based on at least one of pixel values of pixels adjacent to the bad pixel; and
and adding the corrected pixel value of the bad pixel and the pixel values of the remaining pixels to the reference information.

Images