KR20160037115A - Apparatus and method of processing images - Google Patents

Abstract

A described embodiment relates to an image processing device and a method thereof which improve quality of a displayed image when the image is enlarged and displayed. The image processing method comprises the following steps of: preparing a first image; designating at least one region of interest (ROI) in the first image; and generating restoration data. A second image restored by using the restoration data corresponds to the ROI. The resolution of the second image is higher than the resolution of a portion corresponding to the ROI in the first image.

Description

[0001] APPARATUS AND METHOD OF PROCESSING IMAGES [0002]

The embodiments described herein are related to an image processing apparatus and method capable of improving the quality of an image displayed when an image is enlarged and displayed.

With the development of communication technologies and the miniaturization of electronic devices, personal terminals have become widespread to general consumers. In recent years, personal portable terminals such as a smart phone or a smart tablet have become widespread. In addition, with the development of small camera technology and display technology, most smart phones include camera modules and displays. Most digital cameras also include camera modules and displays.

The user can take an image using an electronic device including a camera module and a display, such as a smart phone or a digital camera. The user can confirm the photographed image using a display included in the electronic device. The user can input a touch input corresponding to the enlargement of the image to the electronic device in order to observe the image more closely. In response to the touch input, the electronic device can enlarge and display a part of the image.

If the image is enlarged beyond the original resolution of the image, the quality of the displayed image may be low. For example, if the original resolution of the image is 16 million pixels and the resolution of the display is 2 million pixels, the maximum enlargement magnification that can be displayed on the display without degrading the displayed image quality is sqrt (16/2) = 2.828 (times). If the image is enlarged beyond 2.828 times, the quality of the image may be deteriorated due to up-scaling.

According to the embodiments described above, an image processing apparatus and method capable of improving the quality of an image displayed when an image is enlarged and displayed can be provided.

In addition, according to the embodiment, an image processing apparatus and method that can reduce the size of an additional storage space required for improving the quality of an image can be provided.

An image processing method according to an exemplary embodiment of the present invention includes preparing a first image, designating at least one region of interest (ROI) in the first image, and generating reconstruction data A second image reconstructed using the reconstructed data corresponds to the ROI and a resolution of the second image may be higher than a resolution of a portion corresponding to the ROI in the first image.

The generating of the reconstructed data may include generating reconstructed data of a portion corresponding to the region of interest in the first image, And storing the restoration data.

In addition, the first image may be a preview image generated using light reflected from a subject, and the step of preparing the first image may include displaying the preview image on a display, The step of generating reconstruction data may include the steps of: capturing a third image of the object; generating reconstruction data of a portion corresponding to the region of interest in the third image; And the like.

The generating of the reconstructed data may include capturing a plurality of images of the object, and generating a super resolution image of the ROI using at least a part of the plurality of images, . ≪ / RTI >

The generating of the restoration data may further include generating comparison data indicating a difference between a portion corresponding to the region of interest in the third image and a portion corresponding to the region of interest in the super resolution image .

The generating of the reconstruction data may include generating at least one of the resolutions in the third image that are larger than the resolution of the portion corresponding to the ROI and smaller than or equal to the resolution of the portion corresponding to the ROI in the super resolution image, Scaling the portion corresponding to the region of interest in the third image with each of the selected resolutions or downscaling a portion corresponding to the region of interest within the super resolution image, Making a resolution of a portion corresponding to the region of interest in the image and a resolution of a portion corresponding to the region of interest in the super resolution image equal to each other; A portion corresponding to the region of interest in the super resolution image And generating comparison data for each of the selected resolutions by comparing them with each other.

In addition, the step of designating the at least one region of interest may include: dividing the preview image into a plurality of regions, determining whether or not an object of interest (OOI) is included in each region And designating at least one of the plurality of areas as the ROI according to the determination result.

In addition, the step of designating the at least one region of interest may include the steps of: detecting a position of an object of interest in the preview image by analyzing the preview image, performing image segmentation using the detected position, Detecting an area corresponding to the object, and designating the detected area as the area of interest.

The detecting of the position of the object of interest may include detecting a position of a high frequency component in the preview image, performing a face recognition on the preview image, Detecting a position of the preview image, detecting the position of the text by performing optical character recognition (OCR) on the preview image, and detecting the position of the focus within the preview image .

The step of designating the at least one region of interest may further comprise receiving a drag input from a user, detecting a first region corresponding to the drag input within the preview image, receiving a tap input from a user Detecting a position corresponding to the tap input in the preview image, detecting a second region by performing image segmentation using the detected position, and detecting the first region and the second region, To the region of interest.

In addition, the step of designating the at least one region of interest may include the steps of: detecting a position of an object of interest in the preview image by analyzing the preview image, performing image segmentation using the detected position, Displaying the detected area on the preview image, receiving a tap input from a user, selecting an area corresponding to the tab input from the detected area, selecting or selecting an area corresponding to the tab input from the detected area, Receiving a drag input from a user, detecting a third region corresponding to the drag input within the preview image, and designating the selected region and the third region as the region of interest . ≪ / RTI >

The image processing method may further include displaying a portion corresponding to the region of interest in the stored third image and the third image, receiving an input from a user, receiving the input from the user using the received input, Generating an enlarged image of the third image using the enlargement magnification and the restored data, and displaying the enlarged image based on the enlargement magnification and the reconstruction data.

The step of generating the enlarged image may further include a step of displaying a portion to be displayed on the display in the third image when the enlargement magnification is larger than 1 and the resolution of the third image is smaller than a square root of a quotient of dividing the resolution of the third image by the resolution of the display Scaling down the portion to be displayed on the display in the third image if the enlargement magnification is equal to the square root of the quotient of the division of the resolution of the third image by the resolution of the display, And enlarging the enlarged image by cropping the enlarged image.

The step of generating the enlarged image may further include the step of dividing the resolution of the third image by the square of the quotient obtained by dividing the resolution of the third image by the resolution of the display, If the resolution of a portion corresponding to the RO in the super resolution image is smaller than or equal to a value obtained by multiplying a square root of a quotient obtained by dividing the resolution of a portion corresponding to the RO in the third image, Generating a partial image by upscaling a portion to be displayed on the display in the display unit and generating the enlarged image by combining a portion corresponding to the enlargement ratio and the partial image from the stored restored data with the partial image, .

The step of generating the enlarged image may further include the step of dividing the resolution of the third image by the square of the quotient obtained by dividing the resolution of the third image by the resolution of the display, In a case where the resolution of a portion corresponding to the region of interest in the super resolution image is smaller than a value obtained by multiplying a square root of a quotient obtained by dividing the resolution of a portion corresponding to the region of interest in the third image by the resolution of the portion corresponding to the region of interest, Scaling a portion corresponding to the region of interest from among the portions to be displayed on the display to generate a first partial image, upscaling another portion not corresponding to the region of interest among the portions to be displayed on the display in the third image Generating a second partial image from the restored data; Generating a super resolution partial image by combining a portion corresponding to the first partial image with the first partial image, and generating the enlarged image by combining the super resolution partial image and the second partial image can do.

The step of generating the enlarged image may further include the step of, when the enlargement magnification is equal to the square root of a quotient of the resolution of the third image divided by the resolution of the display, the resolution of the portion corresponding to the region of interest in the super resolution image, Generating a partial image by upscaling a portion to be displayed in the display in the third image when the square is greater than a value obtained by multiplying the square root of the quotient divided by the resolution of the portion corresponding to the region of interest in the 3 images, Synthesizing a portion corresponding to the enlargement ratio and the partial image from the reconstruction data with the partial image, and generating the enlarged image by upscaling the synthesized image.

According to another embodiment of the present invention, there is provided an image processing method comprising: displaying a stored image and a portion corresponding to a region of interest in the image; receiving an input from a user; determining an enlargement magnification using the received input; Generating an enlarged image corresponding to the enlargement magnification using reconstruction data of a portion corresponding to the region of interest in the image, and displaying the enlarged image; And the resolution of the enlarged image may be higher than the resolution of the portion corresponding to the region of interest in the image.

An image processing apparatus according to an embodiment of the present invention includes a light receiving unit that receives light reflected from an object, and a second image processing unit that prepares a first image using the received light, Wherein the second image restored using the restored data corresponds to the region of interest and the resolution of the second image corresponds to the region of interest in the first image, May be higher than the resolution of the portion corresponding to the region.

The image processing apparatus may further include a storage unit, wherein the first image is an image photographed for the object, and the control unit generates restoration data corresponding to the region of interest in the first image And the storage unit may store the first image and the restored data.

The image processing apparatus may further include a video signal processing unit for generating a preview image using the received light, a display unit for displaying the preview image, and a storage unit, Wherein the controller captures a third image of the object by controlling the light receiving unit and the image signal processing unit, and the control unit performs a restoration of a portion corresponding to the ROI in the third image, And the storage unit may store the third image and the restored data.

Also, the restoration data may be a super resolution image for the ROI.

The restoration data may be comparison data representing a difference between a portion corresponding to the region of interest in the third image and a portion corresponding to the region of interest in the super resolution image corresponding to the third image.

In addition, the controller may select at least one of resolutions that are larger than the resolution of the portion corresponding to the ROI in the third image and smaller than or equal to the resolution of the portion corresponding to the ROI in the super resolution image corresponding to the third image, Wherein the control unit scales up a portion corresponding to the region of interest in the third image with each of the selected resolutions or downscales a portion corresponding to the region of interest within the super resolution image, The resolution of the portion corresponding to the ROI in the third image and the resolution of the portion corresponding to the ROI in the super resolution image are made equal to each other, And a region of interest in the super resolution image By comparing the part with each other can generate the reconstruction data for each of the selected resolution.

The control unit may divide the preview image into a plurality of areas, and the control unit may determine whether or not an object of interest (OOI) is included in each of the areas, Accordingly, at least one of the plurality of regions can be designated as the region of interest.

In addition, the controller detects the position of the object of interest in the preview image by analyzing the preview image, and the controller performs image segmentation using the detected position to detect the region corresponding to the object of interest And the control unit may designate the detected area as the area of interest.

The image processing apparatus may further include an input unit for receiving a drag input, wherein the control unit detects a first area corresponding to the drag input in the preview image, and the input unit further receives a tap input , The controller detects a position corresponding to the tap input in the preview image and the controller detects a second region by performing image segmentation using the detected position, And the second region as the region of interest.

Further, the image processing apparatus may further include an input unit for receiving an input, and the control unit may determine an enlargement magnification based on the resolution of the display unit using the received input, An enlarged image of the third image is generated using restoration data, and the display unit displays the enlarged image.

According to another embodiment of the present invention, there is provided an image processing method comprising the steps of: preparing a first image; designating at least one region of interest (ROI) within the first image; Generating restored data corresponding to a region of interest, compressing the first image to generate a compressed image; And storing the compressed image and the restored data.

The image processing method may further include generating a restored image by restoring a portion corresponding to the region of interest in the compressed image using the restored data, and displaying the restored image, The reconstructed image corresponds to the region of interest and the resolution of the reconstructed image may be higher than the resolution of the portion corresponding to the region of interest in the compressed image.

According to the described embodiment, the quality of the displayed image can be improved when the image is enlarged and displayed.

Further, according to the embodiment, the size of the additional storage space required for improving the quality of the image can be reduced.

1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment.
2 is a flowchart illustrating a process of performing an image processing method according to an embodiment.
3 is a preview image displayed on the image processing apparatus according to the embodiment.
4 is a flowchart illustrating a process of designating a region of interest in a preview image according to an exemplary embodiment of the present invention.
5 is a preview image divided into a plurality of regions according to an embodiment.
FIG. 6 is a flowchart illustrating a process of specifying a region of interest in a preview image according to another embodiment.
FIG. 7 is a flowchart illustrating a process of detecting a position of an object of interest in a preview image according to an exemplary embodiment of the present invention.
8 is a preview image according to another embodiment.
9 is a view showing the result of detecting the position of the high frequency component in the preview image shown in FIG. 8 and performing image segmentation.
10 is a preview image according to another embodiment.
11 is a view showing a result of detecting the position of the face in the preview image shown in FIG. 10 and performing image segmentation.
12 is a preview image according to another embodiment.
FIG. 13 is a view showing a result of detecting the position of a text in the preview image shown in FIG. 12 and performing image segmentation.
14 is a preview image according to another embodiment.
FIG. 15 is a view showing the result of detecting the position of the focus in the preview image shown in FIG. 14 and performing image segmentation.
16 is a view showing a result of detecting a region corresponding to an object of interest in a preview image according to another embodiment.
FIG. 17 is a flowchart illustrating a process of specifying a region of interest in a preview image according to another embodiment.
18 is a preview image according to another embodiment.
FIG. 19 is a view showing a result of designating a region of interest in response to a user's input in the preview image shown in FIG.
FIG. 20 is a flowchart illustrating a process of specifying a region of interest in a preview image according to another embodiment.
21 is a screen showing a result of displaying an area corresponding to an object of interest on the preview image shown in Fig.
22 is a reference diagram for explaining a step of storing the first image and the restored data according to the embodiment.
23 is a flowchart illustrating a process of generating and displaying an enlarged image using restoration data according to an embodiment of the present invention.
24 is a screen showing a case where a touch input is received from a user for one thumbnail in the gallery screen displayed on the image processing apparatus.
25 is a screen showing a case where a pinch-out input is received from a user for an image displayed on the image processing apparatus.
26 is a screen showing a case where a drag input is received from a user for an image displayed on the image processing apparatus.
27 is a reference diagram for explaining a step of generating an enlarged image corresponding to an enlargement magnification using restoration data according to an embodiment.
28 is a screen showing a result of displaying an enlarged image according to the embodiment.
29 is a reference diagram for explaining a process of performing the image processing method according to the embodiment.
30 is another reference diagram for explaining the process of performing the image processing method according to the embodiment.
31 is a reference diagram for explaining a process of performing an image processing method according to another embodiment.
32 is another reference diagram for explaining a process of performing an image processing method according to another embodiment.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

Although "first" or "second" and the like are used to describe various components, such components are not limited by such terms. Such terms may be used to distinguish one element from another. Therefore, the first component mentioned below may be the second component within the technical spirit of the present invention.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. &Quot; comprises "or" comprising "as used herein mean that the stated element or step does not exclude the presence or addition of one or more other elements or steps.

Unless defined otherwise, all terms used herein are to be construed in a sense that is commonly understood by one of ordinary skill in the art to which this invention belongs. In addition, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

Hereinafter, the image processing apparatus 100 and the image processing method according to the embodiment will be described in detail with reference to FIG. 1 to FIG.

1 is a block diagram showing a configuration of an image processing apparatus 100 according to an embodiment. The image processing apparatus 100 may be, for example, a laptop computer, a smart phone, a smart tablet, a mobile terminal, a digital camera, or a portable electronic device. 1, the image processing apparatus 100 may include a light receiving unit 110, a video signal processing unit 120, a control unit 130, a display unit 140, an input unit 150, and a storage unit 160 have.

The light receiving unit 110 can receive light reflected from the object. The light receiving unit 110 can capture an object. The light receiving unit 110 can take a picture of the object as an image. The light receiving unit 110 may photograph a photograph, a moving image, a still image, a burst image, or a video. The light receiving unit 110 may include a camera sensor, a camera module, a lens, or an optical element. The light receiving portion 110 may include, for example, a CCD sensor or a CMOS sensor.

The light receiving unit 110 may generate a raw image using the received light. The source image may be composed of, for example, a Bayer format.

The image signal processing unit 120 may process the original image. The image signal processing unit 120 can generate an RGB image by processing a raw image composed of Bayer format. The video signal processing unit 120 may generate a preview image by processing the raw image. The image signal processing unit 120 may include an image signal processor.

The control unit 130 can control the operation of each component included in the image processing apparatus 100 and the image processing apparatus 100. The control unit 130 may perform an operation. The control unit 130 can process the image. The control unit 130 may perform post processing of the image. The control unit 130 may analyze the image. The control unit 130 can control the operation of the image processing apparatus 100 or control the image in response to the input from the user. The control unit 130 can generate data by processing the preview image.

The control unit 130 can execute the application. The control unit 130 may execute a camera application or a gallery application. The control unit 130 may control the light receiving unit 110, the video signal processing unit 120, and the display unit 140 through a camera application. The control unit 130 may cause the light receiving unit 110 to photograph the object through the camera application. The control unit 130 may cause the display unit 140 to display the photographed image through the camera application.

The control unit 130 may control the storage unit 160 and the display unit 140 through the gallery application. The control unit 130 may cause the display unit 140 to display the image stored in the storage unit 160 through the gallery application.

The control unit 130 may be connected to each component included in the image processing apparatus 100. The control unit 130 may include, for example, a central processing unit (CPU) or a microprocessor.

The display unit 140 may display the raw image processed by the image signal processing unit 120. [ The display unit 140 can display the preview image. The display unit 140 may display an image processed by the controller 130. FIG. The display unit 140 can display a screen. The display unit 140 may display a user interface. The display unit 140 may display another screen in response to the input from the user.

The display unit 140 may include a flat panel display device such as a liquid crystal display (LCD), an organic light emitting diode (OLED), or a plasma display panel (PDP). In addition, the display unit 140 may include a curved display or a flexible display. The display unit 140 and the input unit 150 may be integrally formed as a touch screen or separately.

The input unit 150 can receive an input from the outside. The input unit 150 can receive an input from a user of the image processing apparatus 100. [ The input unit 150 may be, for example, a button, a touch screen, a trackball, a camera, a keyboard, a mouse, or a fingerprint recognition device.

The input unit 150 may be a touch screen. The input unit 150 may receive a touch input from a user. A user may input a tap, a drag, a swipe, a slide, a flick, a pinch, or a pinch out as a touch input, Input, or a press-and-hold input.

The storage unit 160 may store the raw image generated by the light receiving unit 110, the image generated by the image signal processing unit 120, or the preview image. The storage unit 160 may store the image processed by the control unit 130 or the data generated by the control unit 130, for example.

 The storage unit 160 may be a volatile memory or a non-volatile memory. The storage unit 160 may be, for example, a flash memory, a ROM, a RAM, an EEROM, an EPROM, an EEPROM, a hard disk drive (HDD), or a register. The storage 160 may include, for example, a file system, a database, or an embedded database.

2 is a flowchart illustrating a process of performing an image processing method according to an embodiment.

Referring to FIG. 2, in the image processing method according to the embodiment, the step S100 of receiving reflected light from the object may be performed first. For example, the control unit 130 may execute a camera application. The control unit 130 may control the light receiving unit 110 through a camera application. The light receiving unit 110 can receive the light reflected from the object. The light receiving unit 110 may generate a raw image using the received light.

Next, a step S200 of displaying a preview image using the received light may be performed. The image signal processing unit 120 may generate a preview image by processing the original image generated using the received light. The preview image can display the object in real time. The display unit 140 can display the generated preview image.

3 is a preview image displayed on the image processing apparatus 100 according to the embodiment. Referring to FIG. 3, the preview image may include an image of one building and an image of two people.

Referring again to FIG. 2, step S300 of designating at least one region of interest (ROI) in the preview image may be performed. The control unit 130 can automatically designate at least one region of interest by analyzing the preview image. In addition, the control unit 130 may designate at least one region of interest in response to the input from the user. In addition, the control unit 130 may designate at least one region of interest in response to the touch input inputted from the user after the analysis result of the preview image is displayed on the display unit 140. [

In other words, the control unit 130 may display at least one region of interest automatically, manually, or in a hybrid manner. A method of automatically designating at least one region of interest will be described later with reference to FIG. A method of manually designating at least one region of interest will be described later with reference to FIG. A method of designating at least one region of interest in a hybrid manner will be described later with reference to FIG.

The region of interest may correspond to at least a portion of the preview image. When the image is captured and the image processing is performed on the photographed image, image processing is performed on only a portion corresponding to the region of interest in the region, instead of performing the image processing on the entire portion of the image . This can shorten the time required for image processing. Also, the size of the memory required for image processing can be reduced. In addition, the size of the storage space required for storing the result of the image processing can be reduced.

At least one region of interest may be included in one image. The region of interest may be a region specified by one closed curve.

4 is a flowchart illustrating a process of designating a region of interest (S300) in a preview image according to an exemplary embodiment of the present invention. Referring to FIG. 4, the step of dividing the preview image into a plurality of regions (S310) may be performed. The control unit 130 can divide the preview image into a plurality of areas. The control unit 130 may divide the preview image into two or more areas. The control unit 130 can divide the preview image into a plurality of areas of uniform size. The control unit 130 may divide the preview image into a plurality of regions of unequal size.

5 is a preview image divided into a plurality of regions according to an embodiment. As shown in FIG. 5, the control unit 130 can evenly divide the preview image shown in FIG. 3 into nine regions. The areas arranged in the second row and the second column among the areas shown in FIG. 5 may include images of one building. The regions arranged in the second row and the third column among the regions shown in FIG. 5 may include images of two persons.

Referring again to FIG. 4, a step S311 of determining whether or not an object of interest (OOI) is included in each of the areas may be performed. The controller 130 may analyze the divided regions. The control unit 130 can determine whether or not an object of interest is included in each area.

The object of interest may be a predefined kind of object. For example, the types of buildings, people, animals, plants, commodities, or natural phenomena can be predefined. The control unit 130 can determine whether or not an object belonging to the predefined type is included in each divided area.

Referring to FIG. 5, among the nine divided regions, the controller 130 can determine that the object of interest is included in the two regions. The control unit 130 can determine that the object of interest is included in the area arranged in the second row and the second column. In addition, the control unit 130 can determine that the object of interest is included in the area arranged in the second row and the third column. The control unit 130 can determine that the object of interest is not included in the remaining seven areas.

Referring again to FIG. 4, in operation S312, at least one of the plurality of regions may be designated as the region of interest according to the determination result. The control unit 130 may designate the region determined to contain the object of interest as the region of interest. Referring to FIG. 5, the controller 130 may designate the regions arranged in the second row and the second column as the ROIs. In addition, the controller 130 may designate the regions arranged in the second row and the third column as the ROIs. The control unit 130 may not designate the remaining seven areas as the ROIs.

FIG. 6 is a flowchart illustrating a process of designating a region of interest (S300) in a preview image according to another embodiment. Referring to FIG. 6, a step S320 of detecting a position of an object of interest in the preview image may be performed by analyzing the preview image. The control unit 130 may analyze the preview image. The control unit 130 can detect the position of the object of interest within the preview image.

The controller 130 may detect a position of a target of interest by detecting a position of a high frequency component in the preview image or performing facial recognition on the preview image, Can be detected. The controller 130 may detect a position of a text by performing optical character recognition (OCR) on the preview image, or may detect a position of a target in the preview image, Can be detected.

The control unit 130 may use all of the methods listed above to detect the position of the object of interest. Also, the control unit 130 may use only a part of the methods listed above. Depending on the user's preferences, the method of detecting the location of interest may vary. The user's preferences related to the method of detecting the location of the object of interest may be stored in the storage 160 in advance. In other words, the method by which the control unit 130 detects the position of the object of interest may be defined in advance in the storage unit 160.

FIG. 7 is a flowchart illustrating a process of performing a step S320 of detecting a position of an object of interest in a preview image according to an exemplary embodiment of the present invention. Referring to FIG. 7, a step S321 of detecting a position of a high frequency component in the preview image may be performed. The control unit 130 can determine whether or not a high frequency component is included in the preview image. If it is determined that the high-frequency component is included in the preview image, the controller 130 can detect the position of the high-frequency component in the preview image.

8 is a preview image according to another embodiment. Referring to FIG. 8, an image of a sculpture is included as a high-frequency component in a central portion of a preview image. In addition, the remaining portion of the preview image excluding the image of the sculpture may be a low frequency component. The control unit 130 can determine that the high-frequency component is included in the preview image shown in Fig. Also, the control unit 130 can detect the position of the molding within the preview image. 9 is a view showing the result of detecting the position of the high frequency component in the preview image shown in FIG. 8 and performing image segmentation.

Referring again to FIG. 7, a step S322 of detecting the position of the face by performing face recognition on the preview image may be performed. The control unit 130 can determine whether or not the face is included in the preview image by performing facial recognition on the preview image. If it is determined that the face image is included in the preview image, the controller 130 may detect the position of the face image in the preview image.

10 is a preview image according to another embodiment. Referring to FIG. 10, the preview image includes images of three human faces. Referring to FIG. 10, a part of the face disposed at the rightmost position is hidden by the face arranged at the center. The control unit 130 can determine that the face is included in the preview image shown in Fig. Also, the control unit 130 can detect the position of the face in the preview image. Referring to FIG. 10, the controller 130 may detect the positions of the leftmost face and the face located at the center. The control unit 130 may fail to recognize the face on the rightmost placed face partially hidden. Therefore, the control unit 130 may not detect the position with respect to the face disposed at the rightmost position. 11 is a view showing a result of detecting the position of the face in the preview image shown in FIG. 10 and performing image segmentation.

Referring again to FIG. 7, a step S323 of detecting the position of the text by performing optical character recognition (OCR) on the preview image may be performed. The control unit 130 can determine whether text is included in the preview image by performing optical character reading on the preview image. If it is determined that text is included in the preview image, the control unit 130 can detect the position of the text in the preview image.

12 is a preview image according to another embodiment. Referring to FIG. 12, a preview image includes a video of a car and a video of a car number. The control unit 130 can determine that text is included in the preview image shown in Fig. Also, the control unit 130 can detect the position of the text in the preview image. Referring to FIG. 12, the controller 130 can detect the position of the image of the vehicle number. FIG. 13 is a view showing a result of detecting the position of a text in the preview image shown in FIG. 12 and performing image segmentation.

Referring again to FIG. 7, the step of detecting the position of the focus within the preview image (S324) may be performed. The control unit 130 can detect the position of the focus in the preview image.

14 is a preview image according to another embodiment. Referring to FIG. 14, a preview image includes a bird image and an image of a leaf. Within the preview image, the focus is on the birds. Therefore, in the preview image, the bird image is clear, and the image of the leaf may be blurry. The control unit 130 can detect the position of the focus within the preview image shown in FIG. Referring to FIG. 14, the controller 130 can detect the position of a new image. FIG. 15 is a view showing the result of detecting the position of the focus in the preview image shown in FIG. 14 and performing image segmentation.

Referring again to FIG. 6, the step of detecting an area corresponding to the object of interest (S328) may be performed by performing image segmentation using the detected position. The control unit 130 may perform image segmentation using the detected position of the object of interest. As a result of the image segmentation being performed, an area corresponding to the object of interest can be detected. Figs. 9, 11, 13, and 15 are screens showing the result of detecting a region corresponding to an object of interest in the preview image.

According to another embodiment, a plurality of methods among the above listed methods can be simultaneously used as a method of detecting the position of the object of interest, with respect to one preview image. For example, it is assumed that an image of an ice cream as a high-frequency component, a video of a face of a person, and a video of a text written on a clothes worn by the person are included in one preview image. The control unit 130 can detect both the image of the ice cream, the image of the face of the person, and the image of the text as an object of interest. 16 is a view showing a result of detecting a region corresponding to an object of interest in a preview image according to another embodiment.

Referring again to FIG. 6, next, the step of designating the detected region as the region of interest (S329) may be performed. The control unit 130 may designate a region corresponding to the object of interest as a region of interest.

FIG. 17 is a flowchart illustrating a process of designating a region of interest (S300) in a preview image according to another embodiment. Referring to FIG. 17, first, a step S330 of receiving a drag input from a user may be performed.

The user can directly designate a desired region of interest. The user can designate the boundary line of the desired area of interest through the drag input. The input unit 150 may receive a drag input from the user.

18 is a preview image according to another embodiment. Referring to Fig. 18, two bird images, three flower images, and a leaf image are included in the preview image. The user can input the drag input along the outline of the image of two birds.

Referring again to FIG. 17, a step S331 of detecting a first region corresponding to the drag input in the preview image may be performed. In response to the drag input, the controller 130 may detect an area corresponding to the drag input. The control unit 130 may generate a closed curve based on the drag input. The control unit 130 may designate an area specified by the closed curve as an area corresponding to the drag input.

FIG. 19 is a view showing a result of designating a region of interest in response to a user's input in the preview image shown in FIG. Referring to FIG. 19, the controller 130 may designate an area specified by the outline of two bird images as an area corresponding to the drag input.

Referring again to FIG. 17, next, step (S332) of receiving a tap input from the user may be performed. The user can directly designate the position of the object of interest desired by the user. The user can designate the position of the target of interest through the tab input. The input unit 150 may receive a tap input from the user. For example, referring to FIG. 18, a user may wish to designate a flower placed in the upper right corner and a flower arranged in the lower left corner as objects of interest within the preview image. The user can input a tap input for the image of the flower arranged on the upper right side and the image of the flower arranged on the lower left side, respectively.

Referring again to FIG. 17, a step S333 of detecting a position corresponding to the tap input in the preview image may be performed. In response to the tap input, the controller 130 may detect a position corresponding to the tap input. Referring to FIG. 18, the controller 130 can detect the position of the image of the flower arranged on the upper right and the position of the image of the flower arranged on the lower left.

Referring again to FIG. 17, a step S334 of detecting a second region by performing image segmentation using the detected position may be performed. The control unit 130 may perform image segmentation using the detected position. As a result of performing the image segmentation, the extended region can be detected from the detected position. Referring to FIG. 19, the control unit 130 can detect an area corresponding to the flower by performing image segmentation using the position of the flower image disposed on the upper right. In addition, the control unit 130 can detect an area corresponding to the flower by performing image segmentation using the position of the flower image disposed at the lower left.

Referring again to FIG. 17, next, the step of designating the first area and the second area as the area of interest (S335) may be performed. The control unit 130 may designate the first region detected in response to the drag input of the user and the second region detected in response to the tab input of the user as the region of interest.

FIG. 20 is a flowchart showing a process of designating a region of interest (S300) in a preview image according to another embodiment. Referring to FIG. 20, a step of detecting a position of an object of interest in the preview image (S340) may be performed by analyzing the preview image.

The control unit 130 may analyze the preview image. The control unit 130 can detect the position of the object of interest within the preview image. The controller 130 may detect a position of a target of interest by detecting a position of a high frequency component in the preview image or performing facial recognition on the preview image, Can be detected. The controller 130 may detect a position of a text by performing optical character recognition (OCR) on the preview image, or may detect a position of a target in the preview image, Can be detected.

For example, referring to FIG. 18, the controller 130 detects a bird image disposed at the center of a preview image, a flower image disposed at an upper right portion, and a flower image disposed at an upper left portion as objects of interest can do. The control unit 130 may not detect an image of a flower arranged on the lower left and an image of a leaf placed on the lower right side as objects of interest.

Referring again to FIG. 20, the step of detecting the region corresponding to the object of interest (S341) may be performed by performing image segmentation using the detected position. The control unit 130 may perform image segmentation using the detected position of the object of interest. As a result of the image segmentation being performed, an area corresponding to the object of interest can be detected.

Next, the step of displaying the detected area on the preview image (S342) may be performed. The display unit 140 may display the detected area on the preview image. 21 is a screen showing a result of displaying an area corresponding to an object of interest on the preview image shown in Fig. Referring to FIG. 21, the display unit 140 can display an area corresponding to a new image on a preview image. Further, the display unit 140 can display, on the preview image, an area corresponding to the flower image on the upper right side. Further, the display unit 140 can display an area corresponding to the flower image on the upper left on the preview image.

Referring again to FIG. 20, next, step S343 of receiving a tap input from the user may be performed. After confirming the area displayed on the display unit 140, the user may desire to designate at least a part of the displayed area as the area of interest. According to another embodiment, the user may not want to designate any of the displayed areas as a region of interest. The user may input a tap input for an area that he / she desires to designate as a region of interest among the displayed areas.

For example, referring to FIG. 21, the user may desire to designate an area corresponding to an image of flowers on the upper right of the displayed area as an area of interest. The user can input a tap input for a point in the area corresponding to the image of the flower in the upper right corner.

Also, the user may not want to change his / her mind and designate the area in which the tab input has already been made as an area of interest. The user can again input the tab input for the area where the tab input is already inputted.

Further, the user may desire to designate, as an area of interest, an area corresponding to at least a part of the objects arranged in the remaining area except for the area displayed on the display part 140. [ The user may further input a tap input for an object corresponding to an area that he or she desires to designate as a region of interest.

For example, referring to FIG. 21, the user may desire to designate an area corresponding to an image of flowers on the lower left of the objects arranged in the remaining areas except the displayed area as an additional area of interest. The user can enter a tap input for the flower in the lower left corner.

Referring again to FIG. 20, a step of selecting or deselecting a region corresponding to the tap input from among the detected regions (S344) may be performed. In response to the tap input, the controller 130 may select or deselect an area corresponding to the tap input from among the detected areas. Referring to FIG. 21, the controller 130 may select a region corresponding to the upper right flower among the detected regions in response to a tap input to the upper right flower. In addition, the controller 130 may additionally select an area corresponding to the flowers on the lower left in response to the tap input for flowers on the lower left.

If a tap input for the upper left flower is additionally received, the controller 130 may additionally select an area corresponding to the upper left flower. If the tap input for the upper right flower is once again received, the control unit 130 can deselect the area corresponding to the upper right flower.

Referring again to FIG. 20, next, step (S345) of receiving a drag input from the user may be performed. The user can input a boundary line of a desired region of interest by inputting a drag input. The input unit 150 may receive a drag input from the user. For example, referring to FIG. 21, a user can input a drag input along the outline of two bird images.

Referring again to FIG. 20, a step S346 of detecting a third region corresponding to the drag input in the preview image may be performed. In response to the drag input, the controller 130 may detect an area corresponding to the drag input. The control unit 130 may generate a closed curve based on the drag input. The control unit 130 may designate an area specified by the closed curve as an area corresponding to the drag input. Referring to FIG. 21, the controller 130 may designate an area specified by the outlines of two bird images as an area corresponding to the drag input.

Referring again to FIG. 20, next, the step of designating the selected region and the third region as the region of interest (S347) may be performed. The control unit 130 may designate the selected region in response to the tap input and the third region detected in response to the drag input as the region of interest.

Fig. 19 is a screen showing the result of designating the region of interest. In comparison with the screen shown in Fig. 21, a region selected in response to a tap input for a flower placed on the upper right and a tap input for a flower arranged on the lower left can be designated as an ROI. Further, the third region detected in response to the drag input inputted along the outline of the images of the two birds can be designated as the region of interest.

Next, the control unit 130 may capture a first image of the object and generate restored data corresponding to the region of interest in the first image. According to another embodiment, the controller 130 may designate at least one region of interest in the first image after capturing the first image of the object, unlike the order in the above embodiment. The control unit 130 may designate a region of interest within the photographed first image in a manner similar to, for example, the method shown in Fig. The controller 130 may generate restored data corresponding to the region of interest in the first image.

The resolution of the restored data may be higher than the resolution of the portion corresponding to the region of interest in the first image. The second image reconstructed using the reconstructed data corresponds to the ROI, and the resolution of the second image may be higher than the resolution of the ROI corresponding to the ROI in the first image.

For example, the reconstructed data may be a super resolution image for the region of interest. The super resolution image may be a higher resolution image reconstructed using a plurality of images having a lower resolution. A relational expression between the low resolution images and the high resolution image can be calculated by analyzing the frequency using the relative difference between the plurality of images having low resolution. The high-resolution image can be reconstructed using the calculated relational expression. The resolution of the super resolution image may be four times the resolution of the low resolution image. For example, if the resolution of the original image is 16 million pixels, the resolution of the super resolution image reconstructed using the original image may be 64 million pixels.

According to another embodiment, the reconstructed data may include comparison data indicating a difference between a portion corresponding to the region of interest in the first image and a portion corresponding to the region of interest in the super resolution image corresponding to the first image Lt; / RTI > The control unit 130 may generate the restored data through the following process.

According to another embodiment, the control unit 130 may capture a first image or a first image stored in the storage unit 160. The control unit 130 may designate at least one region of interest in the first image. The control unit 130 may generate restoration data corresponding to the ROI using the first image. For example, the control unit 130 may crop a portion corresponding to the region of interest in the first image, and use the cut-out portion as reconstruction data. The control unit 130 can generate a compressed image by encoding the first image into JPEG. As the first image is compressed, loss of image information included in the first image may occur. Also, the resolution of the compressed image may be lower than the resolution of the first image.

The storage unit 160 may store the compressed image and the restored data. The controller 130 may generate a restored image by restoring a portion corresponding to the region of interest within the compressed image using the restored data. The display unit 140 may display the restored image. The reconstructed image corresponds to the ROI, and the resolution of the reconstructed image may be higher than the resolution of the portion corresponding to the ROI in the compressed image. By restoring the compressed image using the restored data, the lost image information in the compression process can be restored. In addition, it is possible to restore a compressed image having a low resolution to a high-resolution image having the same level as that of the first image using reconstruction data.

According to another embodiment, image processing may be performed as shown in FIGS. 29 to 32. FIG.

Referring again to FIG. 2, a step S400 of photographing a plurality of images of the object may be performed. The control unit 130 can take a plurality of images of the object by controlling the light receiving unit 110 and the image signal processing unit 120. [ The control unit 130 may capture burst images of the object. For example, the control unit 130 can shoot ten continuous images. The object included in the preview image and the object included in the captured continuous image may be the same. The photographed continuous image can be temporarily stored in the storage unit 160.

Next, a step S500 of generating a super resolution image for the ROI using at least a part of the plurality of images may be performed. The control unit 130 may use at least a part of the plurality of images. For example, the control unit 130 can use five or more images among 10 continuous images. The control unit 130 may generate a super resolution image for the ROI using the image.

The control unit 130 may generate a super resolution image only for a portion corresponding to the region of interest in the image, instead of generating a super resolution image for the entire region in the image. The control unit 130 may analyze frequencies using relative differences between portions corresponding to the ROIs in the captured images. The control unit 130 may generate a super resolution image for the ROI using the analysis result. The generated super resolution image may be temporarily stored in the storage unit 160.

(S600) generating restoration data indicating a difference between a portion corresponding to the region of interest in the first image included in the plurality of images and a portion corresponding to the region of interest in the super resolution image, Can be performed. The control unit 130 can select any one of the plurality of images as the first image. The control unit 130 may extract a portion corresponding to the ROI in the first image. The controller 130 may generate restoration data indicating a difference between the portion extracted from the first image and the portion corresponding to the region of interest in the super resolution image.

However, the resolution of the portion extracted from the first image may be different from the resolution of the portion corresponding to the region of interest in the super resolution image. Accordingly, in order to generate reconstructed data, the controller 130 may first make the resolution of the portion extracted from the first image and the resolution of the portion corresponding to the region of interest within the super resolution image equal to each other. The controller 130 may upscale a portion extracted from the first image or downscale a portion corresponding to the region of interest within the super resolution image. The controller 130 may upscale or downscale using bicubic interpolation.

The control unit 130 may select at least one of resolutions larger than the resolution of the portion extracted from the first image and smaller than or equal to the resolution of the portion corresponding to the RO in the super resolution image. The number of resolutions selected by the controller 130 and the selected resolutions may be stored in the storage unit 160 in advance.

For example, the controller 130 may select n different resolutions among resolutions larger than the resolution of the portion extracted from the first image and smaller than or equal to the resolution of the portion corresponding to the region of interest in the super resolution image have. n may be 5. The five selected resolutions may be resolution A, resolution B, resolution C, resolution D, and resolution E, respectively. The resolution E may be equal to the resolution of the portion corresponding to the region of interest in the super resolution image. Accordingly, resolution A, resolution B, resolution C, and resolution D excluding the resolution E are larger than the resolution of the portion extracted from the first image and smaller than the resolution of the portion corresponding to the RO in the super resolution image have.

The controller 130 may upscale a portion extracted from the first image to a resolution A. FIG. In addition, the controller 130 may downscale a portion corresponding to the region of interest within the super resolution image to a resolution A. [ The control unit 130 may calculate restoration data for the resolution A by comparing the upscaled portion and the downscaled portion.

The control unit 130 may upscale the extracted portion from the first image to the resolution B. [ Also, the controller 130 may downscale a portion corresponding to the region of interest in the super resolution image to resolution B. The control unit 130 may calculate restoration data for the resolution B by comparing the upscaled portion and the downscaled portion.

The control unit 130 may upscale the portion extracted from the first image to the resolution C. In addition, the controller 130 may downscale a portion corresponding to the region of interest in the super resolution image to a resolution C. The control unit 130 may calculate restoration data for the resolution C by comparing the upscaled portion and the downscaled portion.

The controller 130 may upscale a portion extracted from the first image to a resolution D. [ Also, the controller 130 may downscale a portion corresponding to the region of interest within the super resolution image to a resolution D. [ The control unit 130 may calculate restoration data for the resolution D by comparing the upscaled portion and the downscaled portion.

The controller 130 may upscale the portion extracted from the first image to a resolution E. [ The controller 130 may calculate restoration data for the resolution E by comparing the upscaled portion and the portion corresponding to the region of interest within the super resolution image. Thereby, n different restored data can be calculated.

In addition, the first image may include a plurality of ROIs, and the controller 130 may repeat the process for each ROI. If three regions of interest are included in the first image, 3n different reconstructed data can be calculated.

Next, the step S700 of storing the first image and the restored data may be performed. 22 is a reference diagram for explaining a step of storing the first image and the restored data according to the embodiment. Referring to FIG. 22, the storage unit 160 may store the first image as an image file. For example, the storage unit 160 may store the first image as an image file in a JPEG format or a PNG format.

In addition, the storage unit 160 may store the restored data. As shown in FIG. 22, the storage unit 160 may separately store 3n different restored data. The storage unit 160 may store the restored data as marker data of the application. According to another embodiment, the control unit 130 may compress the restored data using a standard JPEG encoding algorithm. The storage unit 160 may store the compressed restored data as marker data of the application. According to another embodiment, the storage unit 160 may store the compressed restored data as a separate image file. If the restored data is compressed, the amount of additional storage space required to store the restored data can be reduced.

23 is a flowchart illustrating a process of generating and displaying an enlarged image using restoration data according to an embodiment of the present invention. Referring to FIG. 23, a step S800 of displaying a stored image and a portion corresponding to a region of interest in the image may be performed. The display unit 140 may display the first image stored in the storage unit 160 and the portion corresponding to the region of interest within the first image.

The control unit 130 may execute the gallery application before the display unit 140 displays a portion corresponding to the first image and the region of interest. The display unit 140 may display a gallery screen including a plurality of thumbnails corresponding to a plurality of images stored in the storage unit 160. [ The input unit 150 may receive a touch input from a user for one thumbnail in the gallery screen. 24 is a screen showing a case where a touch input is received from a user for one thumbnail in the gallery screen displayed on the image processing apparatus 100. [

In response to the received touch input, the display unit 140 may display an image corresponding to the thumbnail and a portion corresponding to the region of interest within the image. For example, a user may input a touch input for a thumbnail corresponding to the first image within a gallery screen. In response to the touch input, the display unit 140 may display a first image and a portion corresponding to the region of interest within the first image, as shown in FIG.

Referring again to FIG. 23, next step S810 of receiving an input from the user may be performed. The user can input a pinch-out input for enlarging the image displayed on the display unit 140. [ 25 is a screen showing a case where a pinch-out input is received from a user for an image displayed on the image processing apparatus 100. [ Referring to FIG. 25, a user may input a pinch-out input for a face portion included in a first image. The input unit 150 may receive the pinch-out input from the user.

In addition, the user can input a drag input for panning or navigation of the image displayed on the display unit 140. 26 is a screen showing a case where a drag input is received from a user for an image displayed on the image processing apparatus 100. [ Referring to FIG. 26 (p), the display unit 140 already enlarges and displays the face portion included in the first image. The user can input a drag input to display another portion in the first image as in (q) or (r) in Fig. The input unit 150 may receive the drag input from the user.

Referring again to FIG. 23, the step of determining an enlargement magnification using the received input (S820) may be performed. The control unit 130 may determine an enlargement ratio in response to the received pinch-out input. The more the user spreads between the two fingers for the pinch-out input, the more the control unit 130 can determine the higher magnification. The controller 130 can determine a rational number greater than 1 as the enlargement magnification.

Generating an enlarged image corresponding to the enlargement magnification using reconstruction data representing a difference between the portion corresponding to the region of interest in the image and the portion corresponding to the region of interest in the super resolution image S830) may be performed. The control unit 130 may generate an enlarged image corresponding to the enlargement magnification using the reconstruction data stored in the storage unit 160. [

27 is a reference diagram for explaining a step of generating an enlarged image corresponding to an enlargement magnification using restoration data according to an embodiment. Referring to FIG. 27, the determined enlargement magnification may be any of the following five cases. The determined enlargement magnification is larger than 1 and smaller than the square root of the quotient of the resolution of the first image divided by the resolution of the display unit 140 as shown in FIG. 27 (a). For example, the resolution of the first image may be 16 million pixels. In other words, the resolution of the original image generated by the light receiving unit 110 may be 16 million pixels. In addition, the resolution of the display unit 140 may be 2 million pixels. Therefore, in the case of FIG. 27 (a), the enlargement magnification may be larger than 1 and smaller than sqrt (16/2) = 2.828.

27 (b), the determined enlargement magnification may be the same as the square root of the quotient obtained by dividing the resolution of the first image by the resolution of the display unit 140. [ Therefore, in the case of FIG. 27 (b), the enlargement magnification may be sqrt (16/2) = 2.828.

27 (c), the determined enlargement magnification is larger than the square root of the quotient obtained by dividing the resolution of the first image by the resolution of the display unit 140, and the resolution of the first image is set to the resolution of the display unit 140 And a square root of a quotient obtained by dividing the resolution of the portion corresponding to the RO in the super resolution image by the resolution of the portion corresponding to the RO in the first image. The square root of the quotient of the resolution of the first image divided by the resolution of the display unit 140 may be sqrt (16/2) = 2.828. Also, a square root of a quotient obtained by dividing the resolution of the portion corresponding to the RO in the super resolution image by the resolution of the portion corresponding to the RO in the first image may be sqrt (4/1) = 2 . Therefore, in the case of FIG. 27 (c), the enlargement magnification may be larger than 2.828 and smaller than 2.828 * 2 = 5.656.

27 (d), the determined enlargement magnification is obtained by multiplying the square root of a quotient obtained by dividing the resolution of the first image by the resolution of the display unit 140 in a portion corresponding to the region of interest in the super resolution image And a value obtained by multiplying the resolution by the square root of the quotient of the first image divided by the resolution of the portion corresponding to the region of interest. Therefore, in the case of FIG. 27 (d), the enlargement magnification may be 5.656.

27 (e), the determined enlargement magnification is obtained by multiplying the square root of a quotient obtained by dividing the resolution of the first image by the resolution of the display unit 140 in a region corresponding to the region of interest in the super resolution image May be greater than a value obtained by multiplying the resolution by the square root of the quotient of the first image divided by the resolution of the portion corresponding to the region of interest. Therefore, in the case of FIG. 27 (e), the enlargement magnification may be larger than 5.656.

The resolution of the enlarged image corresponding to the determined enlargement magnification can be calculated by multiplying the resolution of the display unit 140 by the square of the determined enlargement magnification. 27A, the resolution of the enlarged image corresponding to the enlargement magnification may be larger than the resolution of the display unit 140 and may be smaller than the resolution of the first image. The control unit 130 may generate the enlarged image by downscaling the first image to a resolution of the enlarged image corresponding to the enlargement magnification. According to another embodiment, the control unit 130 may generate an enlarged image by downscaling only the portion to be displayed by the display unit 140 in the first image. In the case of FIG. 27 (b), the resolution of the enlarged image corresponding to the enlargement magnification may be the same as the resolution of the first image. The control unit 130 may designate the first image as the enlarged image. According to another embodiment, the control unit 130 may generate an enlarged image by cropping only the portion to be displayed by the display unit 140 in the first image.

In the case of (c) of FIG. 27, the resolution of the enlarged image corresponding to the enlargement magnification is larger than the resolution of the first image, and the resolution of the portion corresponding to the region of interest in the super resolution image, May be less than a value obtained by multiplying a quotient divided by a resolution of a portion corresponding to the region of interest in the first image. The resolution of the portion corresponding to the region of interest in the super resolution image divided by the resolution of the portion corresponding to the region of interest in the first image is 4, And may be smaller than four times the resolution of the first image.

In the case of FIG. 27 (c), the portion to be displayed by the display unit 140 in the enlarged image may be any of the following three cases. As in the case of (p) in FIG. 26, all of the portions to be displayed by the display unit 140 can correspond to the region of interest. 26 (q), a part of the portion to be displayed by the display unit 140 may correspond to the region of interest, and another portion of the portion to be displayed by the display unit 140 may not correspond to the region of interest. Also, as in the case of (r) in FIG. 26, all of the portion to be displayed by the display unit 140 may not correspond to the region of interest.

27 (c) and 26 (p), the controller 130 may upscale a portion corresponding to the region of interest in the first image. According to another embodiment, the control unit 130 can upscale only the portion to be displayed by the display unit 140 in the first image. The control unit 130 may generate an enlarged image using the upscaled partial image and the restored data stored in the storage unit 160. [ The control unit 130 may combine the enlarged scale factor and the portion corresponding to the upscaled partial image with the upscaled partial image among the restored data stored in the storage unit 160. [

For example, the control unit 130 can determine that the restored data for the resolution B among the stored restored data corresponds to the enlargement magnification. The control unit 130 may combine the portion corresponding to the upscaled partial image with the upscaled partial image in the reconstruction data for the resolution B. [ Since the reconstruction data represents the difference between the portion extracted from the first image and the portion corresponding to the extracted portion in the super resolution image, a super resolution partial image corresponding to the upscaled partial image is generated .

27 (c) and 26 (q), the controller 130 can upscale only the portion to be displayed by the display unit 140 in the first image. The control unit 130 may upscale a portion corresponding to the region of interest among the portions to be displayed by the display unit 140 to generate the first partial image. The controller 130 may generate a second partial image by upscaling another part not corresponding to the region of interest among the portions to be displayed by the display unit 140. [ The control unit 130 may combine the enlargement ratio and the portion corresponding to the first partial image from the restored data stored in the storage unit 160 with the first partial image. The super resolution partial image corresponding to the first partial image may be generated due to the synthesis. The control unit 130 may generate an enlarged image by combining the generated super resolution partial image and the second partial image.

27 (c) and 26 (r), the control unit 130 can generate an enlarged image by upscaling only the portion to be displayed by the display unit 140 in the first image.

In the case of (d) of FIG. 27, the resolution of the enlarged image corresponding to the enlargement magnification is set such that the resolution of the portion corresponding to the region of interest in the super resolution image in the resolution of the first image, And a value obtained by multiplying the quotient divided by the resolution of the portion corresponding to the resolution of the image. The resolution of the portion corresponding to the region of interest in the super resolution image divided by the resolution of the portion corresponding to the region of interest in the first image is 4, 4 times.

27 (d), the portion to be displayed by the display unit 140 in the enlarged image may be any of the following three cases. As in the case of (p) in FIG. 26, all of the portions to be displayed by the display unit 140 can correspond to the region of interest. 26 (q), a part of the portion to be displayed by the display unit 140 may correspond to the region of interest, and another portion of the portion to be displayed by the display unit 140 may not correspond to the region of interest. Also, as in the case of (r) in FIG. 26, all of the portion to be displayed by the display unit 140 may not correspond to the region of interest.

27 (d) and 26 (p), the controller 130 may upscale a portion corresponding to the region of interest in the first image. According to another embodiment, the control unit 130 can upscale only the portion to be displayed by the display unit 140 in the first image. The control unit 130 may generate an enlarged image using the upscaled partial image and the restored data stored in the storage unit 160. [ The control unit 130 may combine the enlarged scale factor and the portion corresponding to the upscaled partial image with the upscaled partial image among the restored data stored in the storage unit 160. [

For example, the control unit 130 can determine that the restored data for the resolution E among the stored restored data corresponds to the enlargement magnification. The controller 130 may combine the portion corresponding to the upscaled partial image with the upscaled partial image in the reconstruction data for the resolution E. [ Since the reconstruction data represents the difference between the portion extracted from the first image and the portion corresponding to the extracted portion in the super resolution image, a super resolution partial image corresponding to the upscaled partial image is generated .

27 (d) and 26 (q), the controller 130 can upscale only the portion to be displayed by the display unit 140 in the first image. The control unit 130 may upscale a portion corresponding to the region of interest among the portions to be displayed by the display unit 140 to generate the first partial image. The controller 130 may generate a second partial image by upscaling another part not corresponding to the region of interest among the portions to be displayed by the display unit 140. [ The control unit 130 may combine the enlargement ratio and the portion corresponding to the first partial image from the restored data stored in the storage unit 160 with the first partial image. The super resolution partial image corresponding to the first partial image may be generated due to the synthesis. The control unit 130 may generate an enlarged image by combining the generated super resolution partial image and the second partial image.

In the case of FIGS. 27 (d) and 26 (r), the controller 130 may generate an enlarged image by up-scaling only the portion to be displayed by the display unit 140 in the first image.

In the case of (e) of FIG. 27, the resolution of the enlarged image corresponding to the enlargement magnification is set such that the resolution of the portion corresponding to the region of interest in the super resolution image to the resolution of the first image, Which is obtained by multiplying the quotient divided by the resolution of the portion corresponding to the number of pixels. The resolution of the portion corresponding to the region of interest in the super resolution image divided by the resolution of the portion corresponding to the region of interest in the first image is 4, May be greater than four times.

27 (e), the portion to be displayed by the display unit 140 in the enlarged image may be any of the following three cases. As in the case of (p) in FIG. 26, all of the portions to be displayed by the display unit 140 can correspond to the region of interest. 26 (q), a part of the portion to be displayed by the display unit 140 may correspond to the region of interest, and another portion of the portion to be displayed by the display unit 140 may not correspond to the region of interest. Also, as in the case of (r) in FIG. 26, all of the portion to be displayed by the display unit 140 may not correspond to the region of interest.

27 (e) and 26 (p), the controller 130 may upscale a portion corresponding to the region of interest in the first image. According to another embodiment, the control unit 130 can upscale only the portion to be displayed by the display unit 140 in the first image. The control unit 130 may generate an enlarged image using the upscaled partial image and the restored data stored in the storage unit 160. [ The control unit 130 may combine the enlarged scale factor and the portion corresponding to the upscaled partial image with the upscaled partial image among the restored data stored in the storage unit 160. [

For example, the controller 130 may combine the upscaled partial image with the portion corresponding to the upscaled partial image in the reconstruction data for the resolution E. Since the reconstruction data represents the difference between the portion extracted from the first image and the portion corresponding to the extracted portion in the super resolution image, a super resolution partial image corresponding to the upscaled partial image is generated . The control unit 130 can generate an enlarged image corresponding to the enlargement magnification by upscaling the generated super resolution partial image by the bi-cubic interpolation method.

27 (e) and 26 (q), the controller 130 can upscale only the portion to be displayed by the display unit 140 in the first image. The control unit 130 may upscale a portion corresponding to the region of interest among the portions to be displayed by the display unit 140 to generate the first partial image. The controller 130 may generate a second partial image by upscaling another part not corresponding to the region of interest among the portions to be displayed by the display unit 140. [ The control unit 130 may combine the enlargement ratio and the portion corresponding to the first partial image from the restored data stored in the storage unit 160 with the first partial image. The super resolution partial image corresponding to the first partial image may be generated due to the synthesis. The control unit 130 may combine the generated super resolution partial image and the second partial image. The control unit 130 can generate an enlarged image corresponding to the enlargement magnification by upscaling the combined image by the bi-cubic interpolation method.

27 (e) and 26 (r), the controller 130 can generate an enlarged image by up-scaling only the portion to be displayed by the display unit 140 in the first image.

Referring again to FIG. 23, the step of displaying the enlarged image (S840) may be performed. The display unit 140 may display an enlarged image generated by the control unit 130. FIG. 28 is a screen showing a result of displaying an enlarged image according to the embodiment.

When the resolution of the first image is 16 million pixels and the resolution of the display unit 140 is 2 million pixels, the maximum enlargement magnification that can be displayed on the display unit 140 without degrading the displayed image quality is sqrt (16 / 2) = 2.828 (times). According to the embodiment described above, if the first image is enlarged by more than 2.828 times, the upscaled image can be displayed from the super resolution image or the super resolution image. Therefore, when the image is enlarged and displayed, the quality of the displayed image can be improved.

In addition, according to the embodiment, only the region of interest in the image is processed, so that the time required for the image processing can be shortened. Also, the size of the memory required for image processing can be reduced. In addition, the size of the storage space required for storing the result of the image processing can be reduced.

Also, according to the above embodiment, instead of storing the super resolution image, restoration data indicating a difference between the portion corresponding to the region of interest in the first image and the portion corresponding to the region of interest in the super resolution image Lt; / RTI > The restored data may be compressed and stored. In addition, the super resolution image corresponding to the ROI can be restored using the stored restored data. Therefore, the size of the additional storage space required for improving the quality of the image can be reduced.

The embodiments described above may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer, and can include both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium can include both computer storage media and communication media. Computer storage media may include both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and may include any information delivery media.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

100: image processing device
110:
120: Video signal processor
130:
140:
150:
160:

Claims (30)

Preparing a first image;
Designating at least one region of interest (ROI) within the first image; And
Step of generating restoration data
Lt; / RTI >
Wherein the second image reconstructed using the reconstructed data corresponds to the region of interest and the resolution of the second image is higher than the resolution of the portion corresponding to the region of interest in the first image. The method according to claim 1,
Wherein the first image is an image taken with respect to an object,
Wherein the step of generating the restoration data comprises:
Generating restoration data of a portion corresponding to the region of interest in the first image; And
Storing the first image and the restored data
And an image processing method. The method according to claim 1,
The first image is a preview image generated using light reflected from the object,
Wherein the preparing the first image comprises:
Displaying the preview image on a display
Lt; / RTI >
Wherein the step of generating the restoration data comprises:
Capturing a third image of the object;
Generating restoration data of a portion corresponding to the region of interest in the third image; And
Storing the third image and the restoration data
And an image processing method. The method of claim 3,
Wherein the step of generating the restoration data comprises:
Capturing a plurality of images of the object; And
Generating a super resolution image for the ROI using at least a part of the plurality of images;
And an image processing method. 5. The method of claim 4,
Wherein the step of generating the restoration data comprises:
Generating comparison data representing a difference between a portion corresponding to the region of interest in the third image and a portion corresponding to the region of interest within the super resolution image
Further comprising the steps of: 5. The method of claim 4,
Wherein the step of generating the restoration data comprises:
Selecting at least one of resolutions in the third image that are larger than the resolution of the portion corresponding to the ROI and smaller than or equal to the resolution of the portion corresponding to the ROI in the super resolution image;
Scaling a portion corresponding to the region of interest in the third image with each of the selected resolutions or downscaling a portion corresponding to the region of interest within the super resolution image, Making a resolution of a corresponding portion and a resolution of a portion corresponding to the RO in the super resolution image equal to each other; And
Generating comparison data for each of the selected resolutions by comparing a portion corresponding to the region of interest in the third image and a portion corresponding to the region of interest in the super resolution image with each other
Further comprising the steps of: The method of claim 3,
Wherein the step of specifying the at least one region of interest comprises:
Dividing the preview image into a plurality of regions;
Determining whether an object of interest (OOI) is included in each of the regions; And
Designating at least one of the plurality of areas as the ROI according to the determination result
And an image processing method. The method of claim 3,
Wherein the step of specifying the at least one region of interest comprises:
Detecting a position of an object of interest in the preview image by analyzing the preview image;
Detecting an area corresponding to the object of interest by performing image segmentation using the detected position; And
Designating the detected region as the ROI
And an image processing method. 9. The method of claim 8,
Wherein detecting the location of the object of interest comprises:
Detecting a position of a high frequency component in the preview image;
Detecting a face position by performing face recognition on the preview image;
Detecting a position of a text by performing optical character recognition (OCR) on the preview image; And
Detecting a position of the focus within the preview image
And an image processing method. The method of claim 3,
Wherein the step of specifying the at least one region of interest comprises:
Receiving a drag input from a user;
Detecting a first region corresponding to the drag input in the preview image;
Receiving a tap input from a user;
Detecting a position corresponding to the tap input in the preview image;
Detecting a second region by performing image segmentation using the detected position; And
Designating the first region and the second region as the region of interest
And an image processing method. The method of claim 3,
Wherein the step of specifying the at least one region of interest comprises:
Detecting a position of an object of interest in the preview image by analyzing the preview image;
Detecting an area corresponding to the object of interest by performing image segmentation using the detected position;
Displaying the detected area on the preview image;
Receiving a tap input from a user;
Selecting or deselecting a region corresponding to the tap input from among the detected regions;
Receiving a drag input from a user;
Detecting a third region corresponding to the drag input in the preview image; And
Designating the selected region and the third region as the region of interest
And an image processing method. The method of claim 3,
Displaying a portion corresponding to the region of interest in the stored third image and the stored third image;
Receiving an input from a user;
Determining an enlargement factor based on the resolution of the display using the received input;
Generating an enlarged image of the third image using the enlargement magnification and the reconstruction data; And
Displaying the enlarged image
Further comprising the steps of: 13. The method of claim 12,
The step of generating the enlarged image may include:
Generating the enlarged image by downscaling a portion to be displayed on the display in the third image when the enlargement magnification is larger than 1 and the resolution of the third image is smaller than a square root of a quotient of dividing the resolution of the display; And
Generating the enlarged image by cropping a portion to be displayed on the display in the third image when the enlargement magnification is equal to a square root of a quotient of dividing the resolution of the third image by the resolution of the display,
And an image processing method. 13. The method of claim 12,
The step of generating the enlarged image may include:
Wherein the enlarged magnification is greater than a square root of a quotient of the third image divided by a resolution of the display and a square root of a quotient of the third image divided by a resolution of the display, Scaling the portion to be displayed on the display in the third image if the resolution of the corresponding portion is less than or equal to the value obtained by multiplying the square root of the quotient of the portion of the third image divided by the resolution of the portion corresponding to the region of interest Generating a partial image; And
Generating the enlarged image by synthesizing a portion corresponding to the enlargement ratio and the partial image from the stored restored data with the partial image,
And an image processing method. 13. The method of claim 12,
The step of generating the enlarged image may include:
Wherein the enlarged magnification is greater than a square root of a quotient of the third image divided by a resolution of the display and a square root of a quotient of the third image divided by a resolution of the display, Corresponding to the region of interest in the third image when the resolution of the corresponding portion is smaller than a value obtained by multiplying the square root of the quotient of the portion corresponding to the region of interest in the third image by the resolution of the portion corresponding to the region of interest Upscaling a portion to generate a first partial image;
Generating a second partial image by upscaling another portion of the portion of the third image not corresponding to the region of interest to be displayed on the display;
Generating a super resolution partial image by combining a portion corresponding to the enlargement ratio and the first partial image from the stored restored data with the first partial image; And
And generating the enlarged image by combining the super resolution partial image and the second partial image
And an image processing method. 13. The method of claim 12,
The step of generating the enlarged image may include:
Wherein the resolution of the portion corresponding to the ROI in the super resolution image is set to a square root of a quotient of the share of the resolution of the third image divided by the resolution of the display in the third image, Generating a partial image by upscaling a portion to be displayed on the display in the third image when the square is greater than a product of the square root of the quotient divided by the resolution of the partial image; And
Synthesizing a portion corresponding to the enlargement ratio and the partial image from the stored reconstruction data with the partial image; And
Generating the enlarged image by upscaling the synthesized image;
And an image processing method. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the image processing method of claim 1. Displaying a stored image and a portion corresponding to a region of interest within the image;
Receiving an input from a user;
Determining an enlargement magnification using the received input;
Generating an enlarged image corresponding to the enlargement magnification using restoration data of a portion corresponding to the region of interest in the image; And
Displaying the enlarged image
Lt; / RTI >
Wherein the enlarged image corresponds to the region of interest and the resolution of the enlarged image is higher than the resolution of the portion corresponding to the region of interest in the image. A light receiving unit for receiving light reflected from an object; And
A controller for preparing a first image using the received light, designating at least one region of interest (ROI) within the first image, and generating restored data,
Lt; / RTI >
Wherein the second image reconstructed using the reconstruction data corresponds to the region of interest and the resolution of the second image is higher than the resolution of the portion corresponding to the region of interest in the first image. 20. The method of claim 19,
Further comprising a storage unit,
Wherein the first image is an image photographed with respect to the object and the control unit generates restored data of a portion corresponding to the region of interest in the first image and the storing unit stores the first image and the restored data The image processing apparatus. 20. The method of claim 19,
A video signal processor for generating a preview image using the received light;
A display unit for displaying the preview image; And
The storage unit
Further comprising:
Wherein the first image is a preview image and the controller captures a third image of the object by controlling the light receiving unit and the image signal processing unit, And the storage unit stores the third image and the restored data. 22. The method of claim 21,
Wherein the reconstructed data is a super resolution image for the ROI. 22. The method of claim 21,
Wherein the reconstruction data is comparison data representing a difference between a portion corresponding to the region of interest in the third image and a portion corresponding to the region of interest within the super resolution image corresponding to the third image. 22. The method of claim 21,
Wherein the control unit controls the resolution of at least one of resolutions greater than a resolution of a portion corresponding to the ROI in the third image and a resolution smaller than or equal to a resolution of a portion corresponding to the ROI in the super resolution image corresponding to the third image And the control unit downscales a portion corresponding to the region of interest in the third image with each of the selected resolutions or downscales a portion corresponding to the region of interest within the super resolution image, The resolution of the portion corresponding to the ROI within the image and the resolution of the portion corresponding to the ROI within the super resolution image are made equal to each other, Corresponding to the region of interest in the super resolution image And generating the restoration data for each of the selected resolutions by comparing portions of the restored data. 22. The method of claim 21,
Wherein the control unit divides the preview image into a plurality of areas, and the control unit determines whether or not an object of interest (OOI) is included in each of the areas, Wherein at least one of the plurality of areas is designated as the area of interest. 22. The method of claim 21,
Wherein the control unit detects a position of an object of interest in the preview image by analyzing the preview image and the control unit detects an area corresponding to the object of interest by performing image segmentation using the detected position, Wherein the control unit designates the detected area as the area of interest. 22. The method of claim 21,
An input for receiving a drag input
Further comprising:
Wherein the control unit detects a first area corresponding to the drag input in the preview image,
Wherein the input further receives a tap input,
Wherein the control unit detects a position corresponding to the tap input in the preview image and the control unit detects a second area by performing image segmentation using the detected position, And designates the second region as the region of interest. 22. The method of claim 21,
An input for receiving an input
Further comprising:
Wherein the control unit determines an enlargement ratio based on the resolution of the display unit using the received input and the control unit generates an enlarged image of the third image using the enlargement ratio and the restoration data,
And the display unit displays the generated enlarged image. Preparing a first image;
Designating at least one region of interest (ROI) within the first image;
Generating restoration data corresponding to the region of interest using the first image;
Compressing the first image to generate a compressed image; And
Storing the compressed image and the restored data
And an image processing method. 30. The method of claim 29,
Generating a restored image by restoring a portion corresponding to the region of interest in the compressed image using the restored data; And
Displaying the restored image
Further comprising:
Wherein the reconstructed image corresponds to the region of interest and the resolution of the reconstructed image is higher than the resolution of the portion corresponding to the region of interest within the compressed image.

Images