KR102617241B1 - Camera module and image generating method performed therein - Google Patents

Abstract

A camera module according to an embodiment includes a lens unit that passes infrared signals incident from the outside; a sensor unit that generates a plurality of image frames using infrared signals passing through the lens unit; a flow plate unit disposed between the sensor unit and the lens unit and controlling the path of an infrared signal incident on the sensor unit; a control unit that controls the flow plate unit to adjust the path of the infrared signal incident on the sensor unit; and an image synthesis unit that generates a composite image by combining a plurality of image frames generated through the sensor unit, wherein the sensor unit generates a plurality of image frames with different parallaxes based on the position of the moving plate unit, The image synthesis unit synthesizes a plurality of image frames with different parallaxes to generate the composite image having a higher resolution than the plurality of image frames, and the flow plate unit transmits an infrared signal that has passed through the lens unit to the sensor unit. a filter unit that transmits; And an actuator that changes the arrangement angle of the filter unit under the control of the control unit to adjust the path of the infrared signal incident on the sensor unit.

Description

Camera module and image generation method thereof {CAMERA MODULE AND IMAGE GENERATING METHOD PERFORMED THEREIN}

The embodiment relates to a camera module, and in particular, to a camera module capable of generating a super resolution image and a method of generating an image thereof.

Users of portable devices have high resolution, small size, and various shooting functions [optical zoom function (zoom-in/zoom-out), auto-focusing (AF) function, image stabilization or image stabilization (Optical Image Stabilizer, They want an optical device that has OIS (OIS) function, etc. This shooting function can be implemented by combining multiple lenses and directly moving the lenses, but if the number of lenses is increased, the size of the optical device may increase.

The autofocus and image stabilization functions are performed by moving or tilting several lens modules with aligned optical axes fixed to a lens holder in the optical axis or the direction perpendicular to the optical axis, and a separate lens drive is used to drive the lens modules. The device is used. However, the lens driving device consumes high power and has the problem of increasing thickness due to the need to add a cover glass separate from the camera module to protect it.

Additionally, as consumer demand for high-quality images increases, image sensors capable of capturing high-resolution images are required. However, for this purpose, the number of pixels included in the image sensor must increase, which increases the size of the image sensor and may result in wasted power consumption.

In other words, existing camera modules use data from multiple arrays as is, so there is a limit to having a resolution equal to the physical resolution of the image sensor. Additionally, there is a limitation in that multiple cameras must be used to generate super-resolution images.

The embodiment provides a camera module that can generate a super-resolution image without increasing the number of pixels of the image sensor and a method of generating an image thereof.

Additionally, the embodiment provides a camera module that can further improve image resolution by applying a Gaussian filter and a Laplacian filter, and a method of generating its image.

The technical problems to be achieved in the embodiment are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

A camera module according to an embodiment includes a lens unit that passes infrared signals incident from the outside; a sensor unit that generates a plurality of image frames using infrared signals passing through the lens unit; a flow plate unit disposed between the sensor unit and the lens unit and controlling the path of an infrared signal incident on the sensor unit; a control unit that controls the flow plate unit to adjust the path of the infrared signal incident on the sensor unit; and an image synthesis unit that generates a composite image by combining a plurality of image frames generated through the sensor unit, wherein the sensor unit generates a plurality of image frames with different parallaxes based on the position of the moving plate unit, The image synthesis unit synthesizes a plurality of image frames with different parallaxes to generate the composite image having a higher resolution than the plurality of image frames, and the flow plate unit transmits an infrared signal that has passed through the lens unit to the sensor unit. a filter unit that transmits; And an actuator that changes the arrangement angle of the filter unit under the control of the control unit to adjust the path of the infrared signal incident on the sensor unit.

In addition, the plurality of image frames include first and second image frames, and the second image frame is an image frame in which the infrared signal is moved by a first pixel interval with respect to the first image frame, The first pixel spacing is 1/2 of the distance between the centers of neighboring pixels.

In addition, the plurality of image frames include first to fourth image frames sequentially generated through the sensor unit, and the first image frame is configured to transmit the infrared signal in a first direction based on the fourth image frame. is an image frame in which the infrared signal is moved by a first pixel interval in a second direction based on the first image frame, and the third image is an image frame in which the infrared signal is moved in a second direction by a first pixel interval, and the third image The frame is an image frame in which the infrared signal is moved by a first pixel interval in a third direction based on the second image frame, and the fourth image frame is an image frame in which the infrared signal is moved by a first pixel interval based on the third image frame. It is an image frame moved by a first pixel interval in four directions, and the image synthesis unit generates a first composite image using the first to fourth image frames.

Additionally, the plurality of image frames include a fifth image frame generated after the fourth image frame, and the image synthesis unit generates a second composite image using the second to fifth image frames.

In addition, the image synthesis unit may include a pre-processing filter unit that performs Gaussian filtering on a plurality of image frames generated through the sensor unit; an image generator that generates a composite image by combining a plurality of image frames filtered through the pre-processing filter unit; and a post-processing filter unit that performs Laplacian filtering on the composite image generated through the image generator.

Additionally, the pre-processing filter unit performs Gaussian filtering on the plurality of image frames by applying a sigma value within a range greater than 0.1 and less than 1.

Additionally, the post-processing filter unit applies Laplacian filtering to the composite image by applying a mask of size 3*3 or 5*5.

Meanwhile, the image generation method according to the embodiment includes moving the moving plate portion disposed between the lens portion and the sensor portion to a first position; With the moving plate unit positioned at the first position, generating a first image frame using an infrared signal incident on the sensor unit; moving the moving plate unit to a second position; generating a second image frame shifted from the first image frame by a first distance in a first direction; moving the moving plate unit to a third position; generating a third image frame shifted from the second image frame by the first distance in a second direction; moving the moving plate unit to a fourth position; generating a fourth image frame shifted from the third image frame by the first distance in a third direction; and generating a first composite image by synthesizing the first to fourth image frames, wherein the first composite image has a higher resolution than the resolution of the first to fourth image frames.

Additionally, after the fourth image frame is generated, re-moving the moving plate unit to the first position; generating a fifth image frame shifted from the fourth image frame by the first distance in a fourth direction; and generating a second composite image using the second to fifth image frames.

Additionally, the method includes Gaussian filtering the first to fourth image frames before generating the first composite image.

Additionally, the Gaussian filtering step includes Gaussian filtering the first to fourth image frames by applying a sigma value within a range greater than 0.1 and less than 1.

Additionally, when the first composite image is generated, the method includes Laplacian filtering the generated first composite image.

According to an embodiment, an ultra-high resolution image can be obtained without increasing the number of pixels of the image sensor. That is, in the embodiment, a plurality of array data shifted by half (0.5) of one pixel is used using a filter actuator that changes the optical path, so an image with a super-resolution higher than the physical resolution of the image sensor can be obtained.

Additionally, according to an embodiment, a decrease in frame rate can be prevented by continuously generating composite frames for sequentially input current frames.

The effects that can be obtained in the embodiment are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a block diagram showing the configuration of a camera module according to an embodiment.
FIG. 2 is a block diagram showing the detailed configuration of the image synthesis unit of FIG. 1.
Figure 3 is a diagram showing the structure of a camera module according to an embodiment.
Figure 4 is an exploded perspective view of the flow plate portion shown in Figure 3.
Figure 5 is a combined diagram of the flow plate portion of Figure 4.
Figure 6 is a diagram for explaining a method of operating a camera module according to an embodiment.
FIG. 7 is a diagram for explaining in more detail the operating method of the camera module described in FIG. 6.
Figure 8 is a timing diagram of a method of operating a camera module according to an embodiment.
Figure 9 is a diagram illustrating an example of a method for combining frames of a camera module according to an embodiment.
Figure 10 is a diagram showing an image frame generated through a sensor unit according to an embodiment.
Figure 11 is a diagram showing an image frame filtered in a pre-processing filter unit according to an embodiment.
Figure 12 is a diagram showing a super-resolution image frame generated by an image generator according to an embodiment.
Figure 13 is a diagram showing an image frame filtered in a post-processing filter unit according to an embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

However, the technical idea of the present invention is not limited to some of the described embodiments, but may be implemented in various different forms, and as long as it is within the scope of the technical idea of the present invention, one or more of the components may be optionally used between the embodiments. It can be used by combining and replacing.

In addition, terms (including technical and scientific terms) used in the embodiments of the present invention, unless specifically defined and described, are generally understood by those skilled in the art to which the present invention pertains. It can be interpreted as meaning, and the meaning of commonly used terms, such as terms defined in a dictionary, can be interpreted by considering the contextual meaning of the related technology. Additionally, the terms used in the embodiments of the present invention are for describing the embodiments and are not intended to limit the present invention.

In this specification, the singular may also include the plural unless specifically stated in the phrase, and when described as "at least one (or more than one) of A and B and C", it is combined with A, B, and C. It can contain one or more of all possible combinations. Additionally, when describing the components of an embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used.

These terms are only used to distinguish the component from other components, and are not limited to the essence, sequence, or order of the component. And, when a component is described as being 'connected', 'coupled' or 'connected' to another component, the component is not only directly connected, combined or connected to that other component, but also is connected to that component. It may also include cases where other components are 'connected', 'coupled', or 'connected' by another component between them.

In addition, when described as being formed or disposed "on top or bottom" of each component, top or bottom refers not only to cases where two components are in direct contact with each other, but also to one component. This also includes cases where another component described above is formed or placed between two components. In addition, when expressed as "top (above) or bottom (bottom)", it may include not only the upward direction but also the downward direction based on one component.

FIG. 1 is a block diagram showing the configuration of a camera module according to an embodiment, and FIG. 2 is a block diagram showing the detailed configuration of the image synthesis unit of FIG. 1.

Referring to Figures 1 and 2, the camera module 100 includes a lens unit 110, a sensor unit 120, an image synthesis unit 130, a moving plate unit 140, and a control unit 150.

Additionally, the image synthesis unit 130 includes a pre-processing filter unit 131, an image generation unit 132, and a post-processing filter unit 133.

The lens unit 110 can pass light incident from the outside and transmit an optical signal to the sensor unit 120. The lens unit 110 may include a plurality of lenses. At this time, the plurality of lenses included in the lens unit 110 form one optical system and may be aligned and arranged around the optical axis of the sensor unit 120.

At this time, the lens unit 110 may include lenses in the infrared band for detecting the infrared wavelength band generated from the subject.

That is, the camera module 100 according to the embodiment is a thermal imaging camera module. Accordingly, in order to detect infrared radiation generated from a subject, the lenses constituting the lens unit 110 do not pass through the infrared radiation energy wavelength band but do not pass through the visible light band, rather than the visible light band lenses used in general camera modules. Lenses made of germanium (Ge) or silicon (Si) can be used. Additionally, a reflective coating layer may be formed on the lens unit 110, thereby preventing reflection of infrared radiant energy and allowing all infrared radiant energy to pass through.

That is, the lens unit 110 may include a PGM (Precision Glass Molding) processed lens made of chalcogenide glass to transmit infrared rays emitted from the subject.

Additionally, the lens unit 110 may further include an infrared transmission window (not shown) on the front side to protect the lens. Infrared transmission windows can be made of materials such as CaF2, BaF2, or polyethylene.

The sensor unit 120 may include an element that is sensitive to infrared radiant energy. That is, the sensor unit 120 can detect energy corresponding to the infrared radiant energy using the device. Preferably, the sensor unit 120 may serve to convert the result of infrared radiant energy incident through the lens unit 110 into an electrical signal.

Preferably, the sensor unit 120 detects the temperature of the subject from infrared light transmitted through the lens unit 110 and outputs a corresponding change in physical characteristics (analog signal). As this sensor unit 120, a microbolometer array (MBA: MicroBolometer Array) may be used.

The image synthesis unit 130 may be an image processor that receives the output signal of the sensor unit 120 and performs image processing (eg, interpolation, frame synthesis, etc.) on it. In particular, the image synthesis unit 130 can generate one high-resolution image signal by combining image signals of a plurality of frames with parallax (difference in the position or direction of the subject depending on the observation position, parallax). At this time, the image signals of the plurality of frames may be individual image frames generated while the incident path of infrared rays is changed by the moving plate unit 140. At this time, the plurality of image frames may include a first image frame and a second image frame. At this time, the second image frame may be an image frame in which the incident path is moved by the first interval by the moving plate unit 140, based on the first image frame.

The flow plate unit 140 may change the path of infrared rays passing through the lens unit 110 and incident on the sensor unit 120. At this time, the flow plate unit 140 may be located at the reference position during initial operation. Additionally, the moving plate unit 140 may move (specifically, tilt) along the X-axis or Y-axis or Hereinafter, the movement of the moving plate unit 140 will be explained using the concept of tilting.

At this time, the moving plate unit 140 may move by a specific pixel interval excluding an integer multiple based on the reference position.

At this time, the moving plate unit 140 may move the incident path of the infrared rays by the first pixel interval in the X-axis direction based on the reference position according to the control signal from the controller 150. In addition, the moving plate unit 140 may move the mass beneficiation path by the first pixel interval in the Y-axis direction based on the reference position according to a control signal from the control unit 150. In addition, the moving plate unit 140 may move the optical path by a first pixel interval in the X-axis direction and a first pixel interval in the Y-axis direction based on the reference position according to a control signal from the controller 150. . Here, the first pixel spacing may be a value between 0 and 1. Preferably, the first pixel spacing may be 0.5. That is, the moving plate unit 140 may move the incident path of the infrared signal to the X-axis and/or Y-axis by 0.5 pixel intervals based on the reference position according to the control signal from the controller 150.

For this purpose, the flow plate unit 140 may include an actuator. The moving plate unit 140 can adjust the relative position of the sensor unit 120 with respect to the lens unit 110. The actuator may be a piezoelectric element, a voice coil motor (VCM), or a MEMS, but the embodiment is not limited to a specific type of actuator.

The structure of the flow plate unit 140 and its operating principle will be described in more detail below.

The control unit 150 may control the operation of the flow plate unit 140. Preferably, the control unit 150 sequentially moves the moving plate unit 140 in the order of the fourth pixel movement (D) to the first pixel movement (A).

Additionally, the control unit 150 allows image signals to be generated through the sensor unit 120 according to the sequential movement of the moving plate unit 140.

To this end, the control unit 150 may transmit and receive a first signal with the flow plate unit 140 and transmit and receive a second signal with the sensor unit 120. Based on this, the control unit 150 controls the flow plate unit 140 and the sensor unit 120 so that an image signal can be generated by the sensor unit 120 in synchronization with the moving state of the flow plate unit 140. Let it happen.

Here, the first signal is generated by the control unit 150 and may include a control signal for pixel movement of the moving plate unit 140. And, based on the control signal for pixel movement, the pixel movement of the moving plate unit 140 is performed, and thereby the angle of incidence of the infrared signal incident on the sensor unit 120 may be changed. Additionally, the first signal may include a response signal indicating that the movement of the flow plate unit 140 has been completed according to the control signal.

The second signal may be generated by the sensor unit 120 and may include a synchronization signal instructing to transmit a control signal of the flow plate unit 140. Depending on the embodiment, the second signal may be generated by the sensor unit 120 and may include control information that is the basis of a control signal for changing the angle of incidence of the infrared signal. Depending on the implementation, the second signal may be generated by the control unit 150 and may include a feedback signal notifying normal reception of a response signal indicating that the movement of the moving plate unit 140 has been completed. Additionally, the second signal may include a driving signal that drives the sensor unit 120.

Here, the signals included in the first signal and the second signal are illustrative, and may be omitted or other signals may be added as needed.

Referring to FIG. 2, the image synthesis unit 130 may include a pre-processing filter unit 131, an image generation unit 132, and a post-processing filter unit 133.

The pre-processing filter unit 131 may receive image frames sequentially acquired through the sensor unit 120. That is, the pre-processing filter unit 131 may receive the first image frame F1 in the fourth pixel movement of the moving plate unit 140. Additionally, the pre-processing filter unit 131 may receive the second image frame F2 from the first pixel movement of the flow plate unit 140. Additionally, the pre-processing filter unit 131 may receive the third image frame F3 from the second pixel movement of the flow plate unit 140. Additionally, the pre-processing filter unit 131 may receive the fourth image frame F4 from the third pixel movement of the flow plate unit 140. At this time, the first image frame (F1) is an image acquired through the sensor unit 120 with the flow plate unit 140 at the reference position, and the second image frame (F2) is an image obtained by the flow plate unit 140. ) is an image acquired through the sensor unit 120 while moving a distance of 0.5 pixels from the reference position in the It is an image acquired through the sensor unit 120 while moving a distance of 0.5 pixels in the direction and a distance of 0.5 pixels in the Y-axis direction, and the fourth image frame (F4) is based on the moving plate unit 140. This refers to an image acquired through the sensor unit 120 while moving a distance of 0.5 pixels from the location in the Y-axis direction.

In addition, the pre-processing filter unit 131 may perform Gaussian filtering on the first to fourth image frames obtained through the sensor unit 120. Here, Gaussian Filtering refers to a filtering technique that blurs the original image, and refers to an image blurring technique (Blur Filtering) used in image application tools such as Photoshop. When Gaussian filtering is performed, low and high frequencies among the frequencies of the image are not passed through, making the image appear blurry. That is, the pre-processing filter unit 131 applies a blurring effect by breaking down the high-frequency region of the first to fourth image frames obtained through the sensor unit 120, thereby applying a blurring effect to the first to fourth image frames included in the first to fourth image frames. Try to remove noise.

At this time, when Gaussian filtering is applied to an image frame acquired through a thermal imaging camera, as in the embodiment, the control factor setting becomes an important factor. Here, the control factor may include a σ (sigma) value, which means the area variable of the Gaussian distribution that forms Gaussian blur in the Gaussian filter. At this time, the image quality of the image finally output from the image synthesis unit 130 may change significantly based on the sigma value.

Accordingly, in the embodiment, the sigma value constituting the control factor of the Gaussian filter constituting the pre-processing filter unit 131 is set as shown in Equation 1 below.

0.1 < σ < 1 ----- Equation 1

At this time, the degree of blurring of the first to fourth image frames changes depending on the sigma value. Here, when the sigma value is greater than 1, the degree of blurring becomes severe, and the boundaries within the first to fourth image frames corresponding to the thermal image accordingly collapse. Additionally, if the sigma value is less than 0.1, the degree of blurring in the first to fourth image frames becomes too small and noise may not be removed properly. Therefore, in Gaussian filtering for preprocessing of the first to fourth image frames corresponding to the thermal image, the sigma value corresponding to the control factor is set to be greater than 0.1 and less than 1.

The image generator 132 generates a composite frame by synthesizing the first to fourth image frames that have been Gaussian filtered through the pre-processing filter unit 131.

The image generator 132 may generate an image obtained by a 2Nx2M pixel array rather than an NxM pixel array by combining the Gaussian filtered first to fourth frames. The method for the image generator 132 to synthesize the first to fourth frames is to simply merge the first to fourth frames according to the position of each pixel (for example, in the case of the first row, the pixel signal of A1 and B1 a method of generating one frame by arranging the pixel signal of A2, the pixel signal of B2, or in the case of adjacent pixels, the pixel signal of one pixel (e.g., C1) by using the pixel scene overlap. A method of correcting using pixel signals of adjacent pixels (e.g., A1, B1, A2, D1, D2, A3, B3, A4) may be used, but the scope of the embodiment is not limited to this and various super-resolution images A creation method may be used.

In addition, the image generator 132 synthesizes a first super-resolution image frame using the first to fourth image frames filtered by the pre-processing filter unit 131, and then synthesizes the first super-resolution image frame using the first to fourth image frames filtered through the pre-processing filter unit 131. A second super-resolution image frame can be synthesized using the 5 image frame and the second to fourth image frames.

The post-processing filter unit 133 performs post-processing on the super-resolution image frame generated through the image generation unit 132.

Preferably, the post-processing filter unit 133 performs Laplacian filtering on the super-resolution image frame generated through the image generation unit 132.

That is, when a super-resolution image frame is generated through the image generator 132, Laplacian filtering is performed in the post-processing filter unit 133, and image post-processing to sharpen the edges of the super-resolution image frame can be performed. there is.

Preferably, the post-processing filter unit 133 may include a Laplacian filter. At this time, the Laplacian filter can perform filtering within a certain mask size. Here, the mask size of the Laplacian filter may be 3*3 or 5*5.

FIG. 3 is a diagram showing the structure of a camera module according to an embodiment, FIG. 4 is an exploded perspective view of the flow plate portion shown in FIG. 3, and FIG. 5 is a combined view of the flow plate portion of FIG. 4.

3 to 5, the camera module 200 includes a holder 210, a lens barrel 220, a lens unit 230, a moving plate unit 240, a substrate 250, a sensor unit 260, and It may include a data processing element 270, and at least one of these components may be omitted or the vertical arrangement relationship may be changed. However, the moving plate unit 240 is disposed between the lens unit 230 and the sensor unit 260, and can change the incident path to the sensor unit 260 for the infrared signal passing through the lens unit 230. .

That is, the moving plate unit 240 can move to either the first position or the fourth position according to the control signal from the control unit 150. And, when the flow plate unit 240 is present at the first position, the infrared signal may be incident in a direction corresponding to the pixel center of the sensor unit 260. And, when the moving plate unit 240 is present in the second position, the infrared signal may be incident in a direction moved by 0.5 pixel distance in the +X axis direction from the pixel center of the sensor unit 260. And, when the flow plate unit 240 is present in the third position, the infrared signal is incident in a direction moved by 0.5 pixel distance in each of the +X-axis direction and +Y-axis direction from the pixel center of the sensor unit 260. It can be. And, when the moving plate unit 240 is present at the fourth position, the infrared signal may be incident in a direction moved by 0.5 pixel distance in the +Y-axis direction from the pixel center of the sensor unit 260.

The holder 210 is coupled to the lens barrel 220 to support the lens barrel 220, and may be coupled to the substrate 250 to which the sensor unit 260 is attached. Additionally, the holder 210 may have a space where the moving plate portion 240 can be attached to the lower part of the lens barrel 220. Holder 210 may include a spiral structure. Additionally, the lens barrel 220 may also include a spiral structure corresponding to the holder 210, and accordingly, the lens barrel 220 and the holder 210 may be rotationally coupled to each other. However, this is an example, and the holder 210 and the lens barrel 220 are coupled through an adhesive (e.g., an adhesive resin such as epoxy), or the holder 210 and the lens barrel 220 are formed as one piece. It could be.

The lens barrel 220 is coupled to the holder 210 and may have a space therein to accommodate the lens unit 230. The lens barrel 220 may be rotationally coupled to the lens unit 230, but this is an example and may be coupled in other ways, such as using an adhesive.

The lens unit 230 may be an infrared lens that passes infrared rays emitted from a subject. That is, the lens unit 230 may include a lens processed by Precision Glass Molding (PGM) made of chalcogenide glass to transmit infrared rays emitted from the subject.

Additionally, the lens unit 230 may further include an infrared transmission window (not shown) on the front side to protect the lens. Infrared transmission windows can be made of materials such as CaF2, BaF2, or polyethylene.

The sensor unit 260 may be mounted on the substrate 250 and may perform the function of converting an infrared signal (infrared radiant energy) passing through the lens unit 230 and the flow plate unit 240 into an image signal. there is. That is, the sensor unit 260 may include an element that is sensitive to infrared radiant energy. That is, the sensor unit 260 can detect energy corresponding to the infrared radiant energy using the device. Preferably, the sensor unit 260 may serve to convert the result of infrared radiant energy incident through the lens unit 230 into an electrical signal.

Preferably, the sensor unit 260 detects the temperature of the subject from infrared light transmitted through the lens unit 230 and outputs a corresponding change in physical characteristics (analog signal). As this sensor unit 230, a microbolometer array (MBA: MicroBolometer Array) may be used.

The substrate 250 may be placed below the holder 210 and may include the image synthesis unit 130 and the control unit 150 as well as wiring for transmitting electrical signals between each component. Additionally, a connector (not shown) may be connected to the board 250 to electrically connect an external power source of the camera module 100 or another device (eg, an application processor).

The substrate 250 is composed of a Rigid Flexible Printed Circuit Board (RFPCB) and can be bent as required by the space in which the camera module 200 is mounted, but the embodiment is not limited thereto.

A moving plate unit 240 may be disposed between the sensor unit 260 and the lens unit 230. The flow plate unit 240 changes the incident path of the infrared signal passing through the lens unit 230 to the sensor unit 260. For example, the flow plate portion 240 may include an actuator 246. Additionally, the actuator 246 may change the incident path of the infrared signal by adjusting the angle of at least a portion of the flow plate unit 240.

Referring to Figures 4 and 5, the moving plate part 240 includes a first bracket 241, a second bracket 242, a first spring part 243, a second spring part 244, and a filter part 245. , may include an actuator 246, an actuator board 247, and a connector 248.

The first bracket 241 is a board on which the actuator board 247 is mounted, and may include an open area in the center. Additionally, the first bracket 241 may have a closed loop shape with an open center. An actuator board 247 may be disposed on the outer surface of the first bracket 241. The actuator board 247 may be arranged to surround the outer surface of the first bracket 241. At this time, a coil (not shown) that functions as the actuator 246 may be formed on the actuator substrate 247. Additionally, an insertion groove (not shown) into which the coil is inserted may be formed in the first bracket 241.

A second bracket 242 may be disposed within the open area of the first bracket 241. The area of the second bracket 242 may be smaller than the area of the open area of the first bracket 241. Accordingly, the second bracket 242 can move in the up, down, left and right directions within the open area of the first bracket 241.

At this time, fixing parts (not shown) to which the first spring part 243 and the second spring part 244 are respectively formed are formed on the first bracket 241 and the second bracket 242. The first spring portion 243 may be disposed below the first bracket 241 and the second bracket 242. Additionally, the second spring unit 244 may be disposed on the first bracket 241 and the second bracket 242. The second bracket 242 may be arranged to float within the open area of the first bracket 241 by the first spring part 243 and the second spring part 244.

Meanwhile, a magnet groove (not shown) may be formed on the outer surface of the second bracket 242 to face the insertion groove formed in the first bracket 241. Additionally, magnets constituting the actuator 246 may be placed in the magnet groove. The magnet may be disposed facing the coil disposed on the actuator substrate 247. And, when a current in a specific direction is applied to the coil through the actuator board 247, a force in a specific direction is generated according to the direction of the applied current, which causes the second bracket 242 to It is possible to move within the open area.

Meanwhile, the second bracket 242 may include a filter mounting portion (not shown) in the central area. Additionally, a filter unit 245 may be disposed on the filter seating part. The filter unit 245 may pass the infrared signal in a specific direction according to the rotation of the second bracket 242.

The filter unit 245 may be formed of germanium (Ge) or silicon (Si) that passes the infrared band but does not pass the visible light band.

Figure 6 is a diagram for explaining a method of operating a camera module according to an embodiment. FIG. 7 is a diagram for explaining in more detail the operating method of the camera module described in FIG. 6.

Referring to FIG. 6, a schematic diagram of a method for obtaining a super resolution image is shown by changing the path of infrared rays incident on the sensor unit 120 by the flow plate unit 140.

The pixel array of the sensor unit 120 may include a plurality of pixels arranged in an NxM matrix. In the following description, for convenience of explanation, it will be assumed that the pixel array includes a plurality of pixels (A1 to A4) arranged in a 2*2 matrix as shown in FIG. 6.

Each pixel (A1 to A4) can generate image information (i.e., an analog pixel signal corresponding to an infrared signal) for each pixel scene (PS1 to PS4) through an infrared signal transmitted through the lens unit 110. .

When the distance between adjacent pixels in the x-axis direction (or y-axis direction) (for example, the distance between pixel centers) is 1 PD (pixel distance), half of it corresponds to 0.5 PD. Here, the first to fourth pixel movements (A to D) will be defined.

The first pixel movement (A) means moving each pixel (A1 to A4) to the right by 0.5 PD along the +x-axis direction, and the pixels after the first pixel movement (A) is completed are B1 to B4.

The second pixel movement (B) means moving each pixel (B1 to B4) downward by 0.5 PD along the +y-axis direction, and the pixels after the second pixel movement (B) is completed are C1 to C4.

The third pixel movement (C) means moving each pixel (C1 to C4) to the left along the -x axis direction by 0.5 PD, and the pixels after the third pixel movement (C) is completed are D1 to D4.

The fourth pixel movement (D) means moving each pixel (D1 to D4) upward by 0.5 PD along the -y-axis direction, and the pixels after the fourth pixel movement (D) is completed are A1 to A4.

Here, pixel movement does not move the physical position of a pixel in the pixel array, but rather moves a virtual pixel (e.g., B1) between two pixels (e.g., A1 and A2) to obtain a pixel scene. This refers to an operation of adjusting the incident path of an infrared signal using the plate unit 140.

In conclusion, pixels A1 to A4 corresponding to the fourth pixel movement D may mean pixels when the moving plate unit 140 is located at the first position corresponding to the pixel center.

And, pixels B1 to B4 corresponding to the first pixel movement (A) are moved by the moving plate unit 140 from the first position to the second position (by 0.5 PD in the +X-axis direction based on the pixel center). It can refer to a pixel in a moving state.

In addition, pixels C1 to C4 corresponding to the second pixel movement (B) are moved by the moving plate unit 140 from the second position to the third position (+X-axis direction and +Y-axis with respect to the pixel center). This may mean a pixel in a state of moving by 0.5 PD in each direction.

In addition, pixels D1 to D4 corresponding to the third pixel movement (C) are moved by the moving plate unit 140 from the third position to the fourth position (by 0.5 PD in the +Y axis direction based on the pixel center). It can refer to a pixel in a moving state.

Meanwhile, this is only an example, and the method for moving the first to fourth pixels may be changed.

Referring to FIG. 7, each pixel (A1 to A4) obtains a pixel scene (S1), and the sensor unit 120 can generate a first frame (F1) from the pixel signal of each pixel (A1 to A4). .

According to the control signal of the control unit 150 that changes the path of the infrared signal incident on the sensor unit 120 for the first pixel movement (A), the flow plate unit 140 changes the incident angle of the infrared signal. The first pixel movement (A) can be performed by changing the reference angle to the right by the first angle change amount (θx).

Thereafter, each pixel (B1 to B4) obtains a pixel scene (S2), and the sensor unit 120 may generate a second frame (F2) from the pixel signal of each pixel (B1 to B4).

According to the control signal of the control unit 150 that changes the path of the infrared signal incident on the sensor unit 120 for the second pixel movement (B), the flow plate unit 140 changes the incident angle of the infrared signal. The second pixel movement (B) can be performed by changing downward by the second angle change amount (θy). Thereafter, each pixel (C1 to C4) obtains a pixel scene (S3), and the sensor unit 120 can generate a third frame (F3) from the pixel signal of each pixel (C1 to C4).

According to the control signal of the control unit 150 to change the path of the infrared signal incident on the sensor unit 120 for the third pixel movement (C), the flow plate unit 140 changes the incident angle of the infrared signal. The third pixel movement (C) can be performed by changing to the left by the first angle change amount (θx). Thereafter, each pixel (D1 to D4) obtains a pixel scene (S4), and the sensor unit 120 can generate a fourth frame (F4) from the pixel signal of each pixel (D1 to D4).

According to the control signal of the control unit 150 to change the path of the infrared signal incident on the sensor unit 120 for the fourth pixel movement (D), the flow plate unit 140 changes the incident angle of the infrared signal. The fourth pixel movement (D) can be performed by changing the pixel upward by the second angle change amount (θy). Thereafter, each pixel (A1 to A4) obtains the pixel scene (S1), and the sensor unit 120 can generate the fifth frame (F5) from the pixel signal of each pixel (A1 to A4). Afterwards, pixel movement and frame creation operations using the moved pixels may be repeatedly performed.

Here, the first angle change amount (θx) and the second angle change amount (θy) each store information corresponding to the angle of incidence at which the infrared signal is incident on the sensor unit 120 so that the pixel can be moved by 0.5 PD. You can.

The image synthesis unit 130 may be an image processor that receives an image signal from the sensor unit 120 and processes the image signal (eg, interpolation, frame synthesis, etc.). In particular, the image synthesis unit 130 can synthesize an image signal (high resolution) of a single frame using multiple frames of image signals (low resolution). The plurality of image frames may be individual image frames generated as the incident path of the infrared signal is changed to different paths by the moving plate unit 140. The plurality of image frames may include a first image frame and a second image frame, and the second image frame may be an image frame moved by a first distance with respect to the first image frame.

The image synthesis unit 130 may generate an image obtained by a 2Nx2M pixel array rather than an NxM pixel array by combining the first to fourth frames. The method for the image synthesis unit 130 to synthesize the first to fourth frames is to simply merge the first to fourth frames according to the position of each pixel (for example, in the case of the first row, the pixel signal of A1 and B1 a method of generating one frame by arranging the pixel signal of A2, the pixel signal of B2, or in the case of adjacent pixels, the pixel signal of one pixel (e.g., C1) by using the pixel scene overlap. A method of correcting using pixel signals of adjacent pixels (e.g., A1, B1, A2, D1, D2, A3, B3, A4) may be used, but the scope of the embodiment is not limited to this and various super-resolution images A creation method may be used.

In addition, the image synthesis unit 130 synthesizes a first super-resolution image frame using the first to fourth image frames transmitted from the sensor unit 120, and then a fifth image frame output from the sensor unit 120. And a second super-resolution image frame can be synthesized using the second to fourth image frames.

According to the operating method of the camera module shown in FIGS. 6 and 7, an image with four times the resolution can be generated by combining a plurality of frames obtained through pixel movement.

Figure 8 is a timing diagram of a method of operating a camera module according to an embodiment.

Referring to FIG. 8, according to a control signal from the control unit 150 to change the path of the infrared signal incident on the sensor unit 120, the flow plate unit 140 changes the incident angle of the infrared signal to a second angle. The fourth pixel movement (D) can be performed by changing the pixel upward by the amount of change (θy). Depending on the embodiment, the control unit 150 may transmit a feedback signal indicating that the fourth pixel movement (D) of the moving plate unit 140 has been completed to the sensor unit 120. At this time, the control unit 150 may determine that the fourth pixel movement (D) has been completed through a response signal from the moving plate unit 140 or a separate timer. Each pixel (A1 to A4) of the sensor unit 120 that receives the feedback signal acquires a pixel scene (S1), and accordingly, the sensor unit 120 generates a first frame from the pixel signal of each pixel (A1 to A4). (F1) can be generated.

Thereafter, according to a control signal from the control unit 150 to change the path of the infrared signal incident on the sensor unit 120, the flow plate unit 140 changes the incident angle of the infrared signal to a first angle change amount (θx). The first pixel movement (A) can be performed by changing the pixel to the right. Depending on the embodiment, the control unit 150 may transmit a feedback signal indicating that the first pixel movement (A) of the moving plate unit 140 has been completed to the sensor unit 120. At this time, the control unit 150 may determine that the first pixel movement (A) is completed through a response signal from the moving plate unit 140 or a separate timer. Each pixel (B1 to B4) of the sensor unit 120 that receives the feedback signal acquires a pixel scene (S2), and accordingly, the sensor unit 120 generates a second frame from the pixel signal of each pixel (B1 to B4). (F2) can be generated.

Thereafter, according to a control signal from the control unit 150 to change the path of the infrared signal incident on the sensor unit 120, the flow plate unit 140 changes the incident angle of the infrared signal to a second angle change amount (θy). The second pixel movement (B) can be performed by changing the pixel downward. Depending on the embodiment, the control unit 150 may transmit a feedback signal indicating that the second pixel movement (B) of the moving plate unit 140 is completed to the sensor unit 120. At this time, the control unit 150 may determine that the second pixel movement (B) is completed through a response signal from the moving plate unit 140 or a separate timer. Each pixel (C1 to C4) of the sensor unit 120 that receives the feedback signal acquires a pixel scene (S3), and accordingly, the sensor unit 120 generates a third frame from the pixel signal of each pixel (C1 to C4). (F3) can be created.

Thereafter, according to a control signal from the control unit 150 to change the path of the infrared signal incident on the sensor unit 120, the flow plate unit 140 changes the incident angle of the infrared signal to a first angle change amount (θx). The third pixel movement (C) can be performed by changing the pixel to the left by that amount. Depending on the embodiment, the control unit 150 may transmit a feedback signal indicating that the third pixel movement (C) of the moving plate unit 140 has been completed to the sensor unit 120. At this time, the control unit 150 may determine that the third pixel movement (C) has been completed through a response signal from the moving plate unit 140 or a separate timer. Each pixel (D1 to D4) of the sensor unit 120 that receives the feedback signal acquires a pixel scene (S4), and accordingly, the sensor unit 120 generates a third frame from the pixel signal of each pixel (D1 to D4). (F4) can be created.

Thereafter, according to a control signal from the control unit 150 to change the path of the infrared signal incident on the sensor unit 120, the flow plate unit 140 changes the incident angle of the infrared signal to a second angle change amount (θy). The fourth pixel movement (D) can be performed by changing the pixel upward by the amount. Depending on the embodiment, the control unit 150 may transmit a feedback signal indicating that the fourth pixel movement (D) of the moving plate unit 140 has been completed to the sensor unit 120. At this time, the control unit 150 may determine that the fourth pixel movement (D) has been completed through a response signal from the moving plate unit 140 or a separate timer. Each pixel (A1 to A4) of the sensor unit 120 that receives the feedback signal acquires a pixel scene (S1), and accordingly, the sensor unit 120 generates a fifth frame from the pixel signal of each pixel (A1 to A4). (F5) can be created. Afterwards, pixel movement and frame creation operations using the moved pixels may be repeatedly performed.

Figure 9 is a diagram illustrating an example of a method for combining frames of a camera module according to an embodiment.

Referring to FIG. 9, it is assumed that the sensor unit 120 sequentially generates first to seventh frames (F1 to F7) according to sequential pixel movements (A to D).

The image synthesis unit 130 may sequentially receive frames and generate a composite frame that is a super-resolution image through super-resolution (SR) image synthesis.

At this time, as shown in FIG. 9, the image synthesis unit 130 may receive the first to fourth frames F1 to F4 and generate the first composite frame F1'. Thereafter, the image synthesis unit 130 may receive the second to fifth frames F2 to F5 and generate a second composite frame F2'. Thereafter, the image synthesis unit 30130 may receive the third to sixth frames F3 to F6 and generate a third composite frame F3'. Thereafter, the image synthesis unit 130 may receive the fourth to seventh frames F4 to F7 and generate the fourth composite frame F4' through a super-resolution image generation algorithm. At this time, the operation of generating the first to fourth composite frames F1' to F4' may be performed by the image generator 132 described above. At this time, before generating the above composite frame, preprocessing may be performed by applying a Gaussian filter to the first to fourth image frames. Additionally, after the above-mentioned composite frame is generated, post-processing by applying a Laplacian filter can be performed. Accordingly, the first to fourth composite frames (F1' to F4') that are finally output through the image synthesis unit 130 may be image frames that have undergone filtering by the post-processing filter unit 133.

Here, the image synthesis unit 130 sequentially receives the first to seventh frames (F1 to F7) from the sensor unit 120, and can store the three frames immediately preceding the currently input frame to generate the composite frame. . Depending on the embodiment, a buffer storing frames may have a storage capacity capable of storing at least three frames.

If a composite frame is created using the first to fourth frames and then a composite frame is created using the fifth to eighth frames, the original frame rate is reduced to 1/4. However, by continuously generating composite frames using the current frame and the three frames immediately preceding the current frame that are input sequentially, as in the method according to the embodiment, a decrease in frame rate can be prevented.

In this embodiment, a method of generating a super-resolution image with 4 times the resolution through four pixel movements has been described, but the scope of the present invention is not limited to this and a super-resolution image with higher resolution through another method of pixel movement has been described. High-resolution images can be created.

Hereinafter, the super-resolution image generation method performed in the above-described camera module 100 will be described as follows.

The method for generating a super-resolution image includes outputting a first image frame, generating a second image frame moved by a first distance in a first direction from the first image frame, and generating a second image frame in a second direction from the second image frame. Generating a third image frame shifted by 1 distance, generating a fourth image frame shifted by a first distance in a third direction in the third image frame, and synthesizing the first to fourth image frames. This may include generating a composite image. A composite image created through this method may have a higher resolution than multiple image frames.

According to the camera module 100 according to the embodiment, the high computational amount required to obtain a super-resolution image is achieved by using the flow plate unit 140 that changes the angle of incidence for the infrared signal incident on the sensor unit 120. This can be solved with hardware.

Additionally, by continuously generating composite frames for the current frames that are sequentially input, it is possible to prevent frame rate deterioration.

Hereinafter, each image frame generated according to the embodiment will be described.

Figure 10 is a diagram showing an image frame generated through a sensor unit according to an embodiment. Figure 11 is a diagram showing an image frame filtered in a pre-processing filter unit according to an embodiment. Figure 12 is a diagram showing a super-resolution image frame generated by an image generator according to an embodiment. Figure 13 is a diagram showing an image frame filtered in a post-processing filter unit according to an embodiment.

Referring to FIG. 10, the sensor unit 120 may generate an image frame for sequentially incident infrared signals according to pixel movement of the moving plate unit 140.

That is, the sensor unit 120 may generate the first image frame F1 at the first position of the flow plate unit 140. Thereafter, the sensor unit 120 may generate a second image frame F2 at the second position of the flow plate unit 140. The sensor unit 120 may generate a third image frame F3 at the third position of the flow plate unit 140. The sensor unit 120 may generate a fourth image frame F4 at the fourth position of the flow plate unit 140. At this time, each of the first to fourth image frames is an image with a different parallax.

Referring to FIG. 11, the pre-processing filter unit 131 may perform pre-processing on the first to fourth image frames obtained through the sensor unit 260, respectively. That is, the pre-processing filter unit 131 may perform Gaussian filtering on the first to fourth image frames obtained through the sensor unit 120. For this purpose, the preprocessing filter unit 131 may include a Gaussian filter.

Here, Gaussian Filtering refers to a filtering technique that blurs the original image, and refers to an image blurring technique (Blur Filtering) used in image application tools such as Photoshop. When Gaussian filtering is performed, low and high frequencies among the frequencies of the image are not passed through, making the image appear blurry. That is, the pre-processing filter unit 131 applies a blurring effect by breaking down the high-frequency region of the first to fourth image frames obtained through the sensor unit 120, thereby applying a blurring effect to the first to fourth image frames included in the first to fourth image frames. Try to remove noise.

At this time, when Gaussian filtering is applied to an image frame acquired through a thermal imaging camera, as in the embodiment, the control factor setting becomes an important factor. Here, the control factor may include a σ (sigma) value, which means the area variable of the Gaussian distribution that forms Gaussian blur in the Gaussian filter. At this time, the image quality of the image finally output from the image synthesis unit 130 may change significantly based on the sigma value.

Accordingly, in the embodiment, the sigma value constituting the control factor of the Gaussian filter constituting the pre-processing filter unit 131 is set to be greater than 0.1 and less than 1.

At this time, the degree of blurring of the first to fourth image frames changes depending on the sigma value. Here, when the sigma value is greater than 1, the degree of blurring becomes severe, and the boundaries within the first to fourth image frames corresponding to the thermal image accordingly collapse. Additionally, if the sigma value is less than 0.1, the degree of blurring in the first to fourth image frames becomes too small and noise may not be removed properly. Therefore, in Gaussian filtering for preprocessing of the first to fourth image frames corresponding to the thermal image, the sigma value corresponding to the control factor is set to be greater than 0.1 and less than 1.

Next, referring to FIG. 12, the image generator 132 generates a composite frame by synthesizing the first to fourth image frames that have been Gaussian filtered through the pre-processing filter unit 131.

The image generator 132 may generate an image obtained by a 2Nx2M pixel array rather than an NxM pixel array by combining the Gaussian filtered first to fourth frames. The method for the image generator 132 to synthesize the first to fourth frames is to simply merge the first to fourth frames according to the position of each pixel (for example, in the case of the first row, the pixel signal of A1 and B1 a method of generating one frame by arranging the pixel signal of A2, the pixel signal of B2, or in the case of adjacent pixels, the pixel signal of one pixel (e.g., C1) by using the pixel scene overlap. A method of correcting using pixel signals of adjacent pixels (e.g., A1, B1, A2, D1, D2, A3, B3, A4) may be used, but the scope of the embodiment is not limited to this and various super-resolution images A creation method may be used.

In addition, the image generator 132 synthesizes a first super-resolution image frame using the first to fourth image frames filtered by the pre-processing filter unit 131, and then synthesizes the first super-resolution image frame using the first to fourth image frames filtered through the pre-processing filter unit 131. A second super-resolution image frame can be synthesized using the 5 image frame and the second to fourth image frames.

Next, referring to FIG. 13, the post-processing filter unit 133 performs post-processing on the super-resolution image frame generated through the image generating unit 132.

Preferably, the post-processing filter unit 133 performs Laplacian filtering on the super-resolution image frame generated through the image generating unit 132.

That is, when a super-resolution image frame is generated through the image generator 132, Laplacian filtering is performed in the post-processing filter unit 133, and image post-processing to sharpen the edges of the super-resolution image frame can be performed. there is.

Preferably, the post-processing filter unit 133 may include a Laplacian filter. At this time, the Laplacian filter can perform filtering within a certain mask size. Here, the mask size of the Laplacian filter may be 3*3 or 5*5.

According to this, in the embodiment, an 80*60 level image can be provided with the resolution of the sensor unit 260 without the operation of the image synthesis unit 130.

On the other hand, the super-resolution image that has undergone image synthesis through the image synthesis unit 130 may be a 320*240 image, and thus an image with a higher super-resolution than the physical resolution of the sensor unit 260 can be provided. there is.

According to an embodiment, an ultra-high resolution image can be obtained without increasing the number of pixels of the image sensor. That is, in the embodiment, a plurality of array data shifted by half (0.5) of one pixel is used using a filter actuator that changes the optical path, so an image with a super-resolution higher than the physical resolution of the image sensor can be obtained.

Additionally, according to an embodiment, a decrease in frame rate can be prevented by continuously generating composite frames for sequentially input current frames.

Although only a few examples have been described as described above, various other forms of implementation are possible. The technical contents of the above-described embodiments can be combined in various forms unless they are incompatible technologies, and through this, can be implemented as a new embodiment.

It is obvious to those skilled in the art that the present invention can be embodied in other specific forms without departing from the spirit and essential features of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (12)

a lens unit that passes light incident from the outside;
a flow plate unit that blocks light in the visible light band from light passing through the lens unit, passes light in the infrared band, and adjusts the path of an infrared signal for the light in the infrared band;
a sensor unit that generates a plurality of image frames using infrared signals passing through the moving plate unit;
a control unit that controls the flow plate unit to adjust the path of the infrared signal incident on the sensor unit; and
An image synthesis unit that generates a composite image by combining a plurality of image frames generated through the sensor unit,
The sensor unit,
Generating a plurality of image frames with different parallaxes based on the position of the moving plate unit,
The image synthesis unit,
Combining a plurality of image frames with different parallaxes to generate the composite image having a higher resolution than the plurality of image frames,
The moving plate part,
a filter unit that blocks light in the visible light band and passes light in the infrared band from the light passing through the lens unit; and
An actuator that changes the arrangement angle of the filter unit under the control of the control unit to adjust the path of the infrared signal incident on the sensor unit.
Camera module. According to paragraph 1,
The plurality of image frames are,
comprising first and second image frames,
The second image frame is,
An image frame in which the infrared signal is moved by a first pixel interval based on the first image frame,
The first pixel spacing is,
1/2 the distance between the centers of neighboring pixels
Camera module. According to paragraph 1,
The plurality of image frames are,
Includes first to fourth image frames sequentially generated through the sensor unit,
The first image frame is,
An image frame in which the infrared signal is moved by a first pixel interval in a first direction based on the fourth image frame,
The second image frame is,
An image frame in which the infrared signal is moved by a first pixel interval in a second direction based on the first image frame,
The third image frame is,
An image frame in which the infrared signal is moved by a first pixel interval in a third direction based on the second image frame,
The fourth image frame is,
An image frame in which the infrared signal is moved by a first pixel interval in a fourth direction based on the third image frame,
The image synthesis unit,
Generating a first composite image using the first to fourth image frames
Camera module. According to paragraph 3,
The plurality of image frames are,
Includes a fifth image frame generated after the fourth image frame,
The image synthesis unit,
Generating a second composite image using the second to fifth image frames
Camera module. According to paragraph 1,
The image synthesis unit,
a pre-processing filter unit that performs Gaussian filtering on a plurality of image frames generated through the sensor unit;
an image generator that generates a composite image by combining a plurality of image frames filtered through the pre-processing filter unit; and
A post-processing filter unit that performs Laplacian filtering on the composite image generated through the image generator.
Camera module. According to clause 5,
The pre-processing filter unit,
Gaussian filtering of the plurality of image frames by applying a sigma value within a range greater than 0.1 and less than 1.
Camera module. According to clause 5,
The post-processing filter unit,
Laplacian filtering of the composite image by applying a mask of size 3*3 or 5*5
Camera module. A step of moving a moving plate unit having a filter unit disposed between a lens unit and a sensor unit to a first position, wherein the filter unit blocks light in the visible light band from light passing through the lens unit and transmits light in the infrared band. It is provided between the lens unit and the sensor unit to pass through the sensor unit,
With the moving plate unit positioned at the first position, generating a first image frame using an infrared signal incident on the sensor unit;
moving the moving plate unit to a second position;
generating a second image frame shifted from the first image frame by a first distance in a first direction;
moving the moving plate unit to a third position;
Generating a third image frame shifted from the second image frame by the first distance in a second direction.
moving the moving plate unit to a fourth position;
generating a fourth image frame shifted from the third image frame by the first distance in a third direction; and
Generating a first composite image by synthesizing the first to fourth image frames,
The first composite image is,
Having a higher resolution than the resolution of the first to fourth image frames
How to create an image. According to clause 8,
After the fourth image frame is generated, re-moving the moving plate unit to the first position;
generating a fifth image frame shifted from the fourth image frame by the first distance in a fourth direction; and
Comprising generating a second composite image using the second to fifth image frames.
How to create an image. According to clause 9,
Comprising Gaussian filtering of the first to fourth image frames before generating the first composite image.
How to create an image. According to clause 10,
The Gaussian filtering step is
Comprising Gaussian filtering of the first to fourth image frames by applying a sigma value within a range greater than 0.1 and less than 1.
How to create an image. According to clause 9,
When the first composite image is generated, comprising Laplacian filtering the generated first composite image.
How to create an image.

Images