KR20050120301A - Method and apparatus for compounding taken image in digital camera - Google Patents

Abstract

The present invention relates to an apparatus and method for capturing an image in a digital camera, and provides an apparatus and method for synthesizing images taken by the camera into a panoramic picture. In the present invention, a process of comparing the first image and the next image captured in the panorama mode for synthesizing the photographed image, and comparing the two images to be synthesized, searching for the overlapped portion, and recognizing the direction of the lens to synthesize It includes the process of doing.

Description

Image synthesizing apparatus and method in a digital camera {METHOD AND APPARATUS FOR COMPOUNDING TAKEN IMAGE IN DIGITAL CAMERA}

The present invention relates to an image capturing apparatus and method, and more particularly, to an apparatus and method for capturing an image using a digital camera.

In general, an image capturing device can be classified into an analog method and a digital method. The analog image photographing apparatus is a commonly used camera, and is an apparatus for recording an image transmitted by a camera lens to a film. On the other hand, a digital image capturing device (hereinafter referred to as a "digital camera") is a device for dimming an image transmitted through a camera lens using a predetermined device, and digitally converting the photosensitive information into a semiconductor memory or the like. Say. Such a digital camera device is to be equipped with a variety of functions as technology advances dramatically. For example, various functions such as enabling close-up photography, digital zoom and optical zoom, and panoramic functions are added. In addition, the digital camera is not limited to a digital camera, but is used in a mobile phone or a personal digital assistant (PDA) and an MP3 device.

As such, the panorama function will be described among various functions of the digital camera. In general, a panorama function in an analog camera means that a plurality of images can be taken in a short time. Therefore, it is a function to capture and image a series of momentary motions. This feature is applied to digital cameras as well as analog cameras.

In the digital camera, since a medium in which an image is stored is a storage device such as a semiconductor device, the digital camera may convert the image in various ways. For example, an image stored in a digital camera can be changed, replaced, or deleted, etc., through a graphic operation on a computer, or a combination of several other images can be combined into one image. As described above, the panorama method may be used to combine a single image into multiple images. An example in which the panorama method is used is as follows.

In the analog camera, panorama shooting is a method of taking a long side or upward direction than a normal picture to capture a lot of scenery in a picture (picture), and using a special lens such as a fisheye lens or an ultra wide-angle lens, and about 2 times the aspect ratio of a general picture. Most of these big pictures were taken. However, since the spread of digital cameras, it is possible to shoot 360-degree shots that contain all of the surrounding scenery in one image.

In addition to the methods described above, panorama pictures in analog cameras are made in several ways. One method that is still used is to take a series of shots with a camera mounted on a tripod. In some other cameras, film strips that are wider than ordinary film strips are exposed using optics with special mobility to produce panoramic images. Another traditional camera, such as 35mm, uses a mask to set the size of photos and images during the exposure. The effect is a panoramic effect rather than a total exposure limited by a subject input through the lens. Another way to create a panoramic image is to cut and paste photosensitive films by physically exposing them by carefully adjusting the boundaries between the ends of the film.

On the other hand, in order to provide a panoramic effect in a digital camera, a method of converting two or more images into a single image by combining the boundaries of images captured using a personal computer is used. In order to make panoramic images of images captured by the device having the digital camera, users have to use various programs on a computer. The panoramic image can be obtained from a digital camera equipped with a panorama function. However, when taking a panoramic image with the digital camera, the user has to directly determine and select a duplicate part after photographing a subject. The time delay will also be longer. In addition, there is a possibility that the combined images may not be connected smoothly, and if there is no fixed stand such as a tripod, there is a problem of obtaining an image for reproducing an accurate panorama image.

Accordingly, it is an object of the present invention to provide an image synthesizing apparatus and method in a device equipped with a digital camera.

Another object of the present invention is to provide a method and apparatus for synthesizing different images using overlapping portions by recognizing and comparing directionality by comparing a first image and an image taken afterwards in a mobile terminal equipped with a digital camera. have.

In order to achieve the above object, a method of photographing at least two or more images by a user, a process of recognizing direction by comparing arbitrary values of the images, and a process of recognizing the direction of the images and comparing them according to the direction And selecting a better one of the two images and combining the two images consecutively when the random value of the images is less than or equal to a threshold by comparing the directions of the images, and the image capturing time interval is less than the minimum time interval. The process of requesting a retake at a time.

An apparatus for achieving the above object is a mobile terminal having a camera and a display unit capable of displaying an image received from the camera, the second memory for temporarily storing the image, when two or more images are stored combined Comparing a random value of the first image with a random value of the second image to recognize a direction when the value is less than or equal to a threshold value, and recognizing the directionality according to the perceived directionality. And a controller configured to compare the predetermined blocks of the second image to synthesize two images when the threshold value is less than or equal to a threshold value, and store the synthesized image in the memory unit.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the same components in the figures represent the same numerals wherever possible. Specific details are set forth in the following description, which is provided to aid a more general understanding of the invention. In the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

The image capturing method according to an embodiment of the present invention described below may be applied to a digital camera and devices having such a function. In an embodiment of the present invention, a mobile terminal equipped with a camera will be described as an example. However, the present invention is not necessarily limited to a mobile terminal having a camera, and can be applied to any device capable of capturing an image by mounting a microprocessor having a relatively small processing capacity.

1 is an internal block diagram of a mobile terminal according to an embodiment of the present invention.

Referring to FIG. 1, the RF unit 100 includes an RF transmitter for up-converting and amplifying a transmitted signal, and an RF receiver for low noise amplifying and down-converting a received signal. The data processor 102 includes a receiver for demodulating and decoding a received signal, a transmitter for modulating and encoding a transmitted signal, and the like. That is, the data processor 102 may be configured of a modem and a codec. The codec plays a function of reproducing a received audio signal output from a data codec for processing packet data and an audio codec such as voice or transmitting a transmitted audio signal generated from a microphone to an audio codec of the data processor 102. Perform. If the data processor 102 does not find a duplicate image during the process of comparing and compositing the images, the data processor 102 may output a beep sound or another sound to request the user to retake the image. In addition, the controller 114 receives and processes a voice for displaying guidance and execution results while performing another program.

The key input unit 104 of FIG. 1 includes keys for inputting numeric and character information and function keys for setting various functions. In addition, the key input unit 104 may include function keys for controlling an image mode and photographing keys for driving a camera according to an embodiment of the present invention.

In FIG. 1, the memory unit 106 of the mobile terminal may include a read only memory (ROM) and a random access memory (RAM). The memory unit 106 stores various temporary data generated during the processing of the controller 114 and user data such as a phone number, ring tone, and image information. In addition, the memory unit 106 includes a program for controlling a general operation of the mobile terminal and a predetermined image for controlling a process of recognizing and combining directions by comparing images applied to the display unit 120 according to an embodiment of the present invention. You can save the programs.

In the present invention, the first memory 108 and the second memory 110 are illustrated in the memory unit 106 in order to help the understanding of the present invention. The first memory 108 and the second memory 110 are shown as shown in FIG. 108 and the second memory 110 are not actually divided.

The first memory 108 and the second memory 110 are separated in order to facilitate understanding in the embodiment of the present invention as a memory used to store and compare the first image and the next image. The first image is stored in the first memory 108, and the images input through the camera 116 are stored in the second memory 110 after the first image is captured. In addition to the memory unit 106, an expansion memory 112 may be provided externally to store more images.

The controller 114 performs a function of controlling the overall operation of the mobile terminal. In addition, the control unit 114 compares the images captured according to the mode command set from the key input unit 104 according to an embodiment of the present invention, synthesizes overlapping portions by recognizing the direction, and transmits the synthesized images and Control the function of receiving. In addition, the control unit 114 outputs a specific state, that is, the arrival of a text message, the dialing state, and the data received from the camera 116 to the display unit 120. In addition, the control unit 114 may display a general operation state such as a current time, a reception sensitivity, and a remaining battery capacity on the display unit 120.

The camera 116 of FIG. 1 includes a camera sensor for converting an optical signal detected at the time of image capturing into an electrical signal, and a signal processor for converting an analog image signal photographed from the camera sensor into digital data. The camera sensor may be implemented as a CCD (Charge Coupled Device) sensor, the signal processing unit may be implemented as a DSP (Digital Signal Processor), and may be implemented as an image processing device other than the DSP, the camera sensor and The signal processor may be integrally implemented or may be separately implemented.

The image signal output from the camera 116 is applied to the image processor 118, and the image processor 118 performs a function of generating screen data such that the input image signal is output to the display unit 120. The image processor 118 processes the image signal output from the camera 116 in units of frames, and outputs the frame image data according to the characteristics and size of the display unit 120.

In FIG. 1, the display unit 120 visually displays an image signal applied from the image processing unit 118, and the display unit 120 may use a liquid crystal display (LCD). In this case, the display unit 120 may be used. 120 may include an LCD controller, a memory capable of storing image data, and an LCD display device.

2A and 2B are flowcharts illustrating a method of recognizing and combining directionalities of images input from the camera 116 according to an embodiment of the present invention.

Before the processes of FIGS. 2A and 2B are performed, the user must proceed to the camera photographing mode through the key input unit 104. The controller 224 displays the image input from the camera 116 on the display unit 120 through the image processor 112, which is called a preview mode. If the controller 114 does not request to take a picture according to the operation state of the camera 116, the controller 114 continues to operate in the preview mode to show an image input from the camera 116 to the user. The image capturing method may be performed by first driving the camera 116 to perform a preview mode and then performing a capturing operation. The capturing operation may be performed simultaneously with driving the camera 116.

Then, when the user drives the camera 116 to photograph the subject, the image processor 118 detects this and performs a preview mode to display an image of the subject entered through the camera 116 on the display unit 120. Display.

A control flow according to an embodiment of the present invention will be described with reference to FIGS. 2A and 2B.

If the user selects the panorama photographing mode through the key input unit 104 in step 200 of FIG. 2A, the control unit 114 of the mobile terminal proceeds to step 202 to capture the subject. In step 202, if the image is taken, the process proceeds to step 204 and determines whether the captured image is the first captured image. If the captured image is the first image, the controller 114 proceeds to step 206 and stores it in the first memory 108. If the first image is not the first image, the controller 114 proceeds to step 208 and the minimum time after the first image is captured in step 202. Check if the image was taken after the interval. If the minimum time interval is not satisfied in step 208, the controller determines that the image is the same image and proceeds to step 210. The control unit 114 requests the data processor 102 to retake the image through the speaker or the display unit 120. Proceed to step 202 after delivering to the user. In step 208, the controller 114 proceeds to step 212 and stores the image captured after the minimum time interval in the second memory 110. After the process is completed, the controller 114 compares the image stored in the first memory 108 with the image stored in the second memory 110 in step 214 to recognize the moving direction of the mobile terminal. After comparing the two images in step 214, the controller 114 checks the moving direction of the mobile terminal in step 216. A process of recognizing the moving direction of the mobile terminal, that is, the moving direction of the camera lens will be described later with reference to FIGS. 3, 4, and 5. In addition, for convenience of understanding, an image stored in the first memory 108 will be referred to as a first image, and an image stored in the second memory 110 will be referred to as a second image.

The direction detection process in step 216 will be described in detail with reference to FIG. 2B. The control unit 114 compares the first image and the second image in step 20 of FIG. 2B and checks whether there is a duplicate part between the two images. If it is checked in step 20 that there is no overlapping part, the controller 114 proceeds to step 222 to request re-photographing, and if it is checked that there is a overlapping part between the two images, it proceeds to step 22. In step 22, the controller 114 detects the movement of the mobile terminal and determines which of the up / down / left / right directions it moves. The process of step 20 and step 22 will be described later with reference to FIGS. 3, 4, and 5. When the direction in which the mobile terminal moves in step 22 is determined, the controller 114 proceeds to step 24 and checks in which direction of up / down / left / right directions.

In the exemplary embodiment of the present invention, the moving direction of the mobile terminal is inspected in the order of right, left, and down, but it does not matter which direction is inspected first. If the mobile terminal moves to the right in step 24, the controller 114 proceeds to step 28 and compares the right value of the first image with the left value of the second image. In the above process, the process of comparing image values may use R (Red), G (Green), and B (Blue) values of each pixel, which are known in the art, or may use a discrete cosine transform (DCT).

If the moving direction is not the right in step 24, the controller 114 proceeds to step 26 to check whether the moving direction is left. If the movement direction is left in step 26, the controller 114 proceeds to step 30 and compares the left side of the first image with the right side of the second image. If the movement direction is not the left side in step 26, the controller 114 proceeds to step 32 and checks whether the movement direction is downward. If the moving direction is downward, the controller 114 proceeds to step 34 and compares the lower value of the first image with the upper value of the second image. If the movement direction of the mobile terminal is not downward in step 32, the movement direction of the mobile terminal is only upward. In step 36, the controller 114 compares the upper value of the first image with the lower value of the second image. do. After comparing the first image value and the second image value in steps 28, 30, 32, and 36, the controller 114 proceeds to step 220 of FIG. 2A and checks whether the compared values are less than or equal to a threshold value. do. If the comparison values exceed the threshold value, the process proceeds to step 222 and requires re-photographing. If it is less than or equal to the threshold value in step 220, the controller 114 proceeds to step 224 to combine and combine the overlapping portions of the first image and the second image. After synthesizing the photographed images in step 224, the controller 114 proceeds to step 202 to continue capturing the image if the photographing continues in step 226. When the photographing is finished through the key input unit 104, the panorama image is converted into a panoramic image. The combined image is stored in the memory unit 106.

A method for recognizing the orientation of the mobile terminal described above will be described with reference to FIGS. 3, 4, and 5.

FIG. 3 is an image of a full size used in a test according to an exemplary embodiment of the present invention. A diagram for describing a method for directional recognition is denoted by reference numeral 300, which corresponds to 400 of FIG. 4. Herein, the mobile terminal moves to the left and is an image photographed, and the process of the controller 114 recognizing the moving direction of the mobile terminal will be described through the images. 4, reference numeral 400 denotes nine equal parts 300, and reference numeral 402 denotes an arbitrary point of the middle block. Here, any one point is a value larger than 1 pixel for the controller 114 to compare with the next photographed image in order to recognize the moving direction of the mobile terminal. The controller 114 extracts a value of reference numeral 402 and stores it in the memory unit 106, and finds the extracted value at reference numeral 404, which is the next photographed image. Reference numeral 404 is a point where a difference from the value of the reference numeral 402 is equal to or less than a threshold value. In this case, the reference numeral 402 and the reference numeral 404 are recognized as the same area. It can be seen that the mobile terminal has moved to the left because the reference point 402 has moved the image to the right as shown by the reference numeral 404. The aforementioned directional search method will be described with reference to FIG. 5.

FIG. 5 is a diagram illustrating a search method for directional recognition according to an embodiment of the present invention. In FIG. 5, only some of the various methods are shown as an example. Reference numeral 500 illustrates a method of extracting an arbitrary region value of the center block of the first image and then searching while rotating from the center block of the second image. Reference numeral 502 illustrates a method of extracting an arbitrary region value of the center block of the first image and then searching for whether the arbitrary region value exists from the center block of the second image. In determining the size of the arbitrary area, the smaller the size, the higher the accuracy and the larger the calculation amount, so it is necessary to determine the optimal size through tradeoff. As shown in the reference numeral 500 and the reference numeral 502, the controller 114 determines the moving direction of the user according to which block in the second image is located an arbitrary region of the first image.

FIG. 6 is an image actually photographed from right to left according to an embodiment of the present invention, and is taken by determining a moving direction of an image by one of the methods described with reference to FIGS. 4 and 5.

7 is a diagram illustrating a process of combining images taken from right to left according to an embodiment of the present invention. In FIG. 7, since the movement direction of the camera 116 is detected from right to left, the leftmost block of the first image and the rightmost block of the next image are compared to examine whether there is an overlapping part. Combine the images. When searching for a duplicated portion, the pixel value of the left block of the first photographed image is stored in the first memory 108. The pixel value of the right block of the second image captured after the first image is stored in the second memory 110. The controller 114 compares the pixel values of the stored image. At this time, the pixel value has a margin of about 10 percent on both sides of the width in the direction of travel, and shifts by one pixel as shown by reference numeral 702 to check whether there is an overlapping portion between the first image and the second image. Will be examined. As a method of comparing the pixel values stored in the first memory 108 and the pixel values stored in the second memory 110, the red, green, and blue colors of each pixel are simply used. : B) There is a method of comparing and inspecting a value and a discrete cosine transform (DCT) method known in the art.

8 is a diagram illustrating comparison blocks of a captured image according to a moving direction of a mobile terminal according to an embodiment of the present invention. Reference numeral 800 is a diagram in which blocks of two images to be compared when the mobile terminal moves to the right are indicated by hatching. When the moving direction of the mobile terminal is the right side, the first rightmost block 801 of the first image and the leftmost block 802 of the second image are sequentially compared. Although the drawings are shown in block units for the sake of understanding, the comparison is actually performed by one pixel. Reference numeral 803 denotes blocks of two images to be compared when the mobile terminal moves to the left by hatching. When the moving direction of the mobile terminal is left, the leftmost block 804 of the first image and the rightmost block 805 of the second image are sequentially compared in units of 1 pixel. Reference numeral 806 is a diagram in which blocks of two images to be compared are indicated by hatching when the moving direction of the mobile terminal is upward. When the moving direction of the mobile terminal is upward, the bottom block 807 of the first image and the top block 808 of the second image are sequentially compared in units of 1 pixel. Reference numeral 809 is a diagram in which blocks of two images to be compared when the moving direction of the mobile terminal is downward are indicated by hatching. When the moving direction of the mobile terminal is downward, the first block 810 of the first image and the bottom block 811 of the second image are sequentially compared in units of 1 pixel.

If the user changes the direction downward while taking an image while moving to the right, the comparison value between the current image and the next image exceeds the threshold. In this case, the changed direction is recognized as a way of recognizing the first direction, and the captured images are compared and combined accordingly.

FIG. 9 is a diagram illustrating images in which sequential photographed images are determined and combined according to an exemplary embodiment of the present invention. FIG. 7 is a view illustrating images combined by comparing the images photographed in FIG. 7 by the method described above. As a user continuously photographs an image of a subject without using an auxiliary device such as a tripod, an unnecessary surplus region is generated above and below the synthesized photo as shown in FIG. 9. Therefore, it is necessary to remove these unnecessary areas to provide the user with an optimal composite image.

10 is a view showing a final image after removing unnecessary portions of the combined image according to an embodiment of the present invention. For the same reason as in FIG. 9, unnecessary parts are removed above and below the composite image to provide a final image to the user.

As described above, the present invention is a method of synthesizing images input from the camera like a panorama in all devices having a digital camera, the user can solve the problem of checking and compositing by eye directly, by recognizing the direction of movement By combining and increasing, the user's convenience can be increased. In addition, even when the length or width exceeds the range that can be photographed by the camera at once, there is an advantage of removing the limitation of the size of the image that can be actually collected by photographing and synthesizing in the form 'c', 'ㄹ'. In the embodiment of the present invention, but only an example of a mobile terminal having a camera, it will be apparent to those skilled in the art that the present invention can be applied to all devices having a camera as well as a mobile terminal.

1 is an internal block diagram of a mobile terminal according to an embodiment of the present invention;

2A is a diagram illustrating a process of combining images captured by a mobile terminal according to an embodiment of the present invention;

FIG. 2B is a control flowchart for explaining operation 216 of FIG. 2A in detail according to an embodiment of the present disclosure;

3 is a full-size image used in the test according to an embodiment of the present invention,

4 is a diagram illustrating an example of a search for directional recognition of a photographed image according to an embodiment of the present invention;

5 is a diagram illustrating a search method for directional recognition according to an embodiment of the present invention;

6 is an image actually photographed from right to left according to an embodiment of the present invention,

7 is a view showing a process of combining images taken from right to left according to an embodiment of the present invention;

8 is a diagram illustrating comparison blocks of a captured image according to a moving direction of a mobile terminal according to an embodiment of the present disclosure;

9 is a view showing an image of combining images taken in succession according to an embodiment of the present invention by inspecting the directionality,

10 illustrates a final image after removing unnecessary portions of the combined image according to an embodiment of the present invention.

Claims (6)

What is claimed is: 1. A method for capturing and combining images in a digital camera device. In the panorama shooting mode, the process of taking images in preset time units, A direction of the camera lens may be selected by selecting a predetermined portion of the captured image in time from among the photographed images, and inspecting a portion corresponding to the selected predetermined portion in an image photographed immediately after the selected image. Detecting the process, Inspecting an error of an adjacent region between two images according to a moving direction of the lens; And combining the two images when the error between the adjacent regions is smaller than a preset threshold value. The method of claim 1, And re-shooting is required if the error between the adjacent areas is not smaller than the threshold value. The method of claim 1, wherein the detecting of the moving direction of the camera lens comprises: After dividing the first image and the second image into a predetermined number of predetermined blocks, the moving direction of the camera lens is detected according to which part of the second image the specific region of the first image is moved. Image composition in a digital camera characterized in that. The method of claim 1, wherein the checking of the error of the adjacent region between the two images according to the moving direction of the lens comprises: An area adjacent to the first image and the second image is selected according to a direction in which the second image moves from the first image, and the area is moved inward from an edge of the first image and the second image. And dividing the block into several blocks, comparing the blocks of the two images, and checking a difference from a preset threshold. The method of claim 4, wherein In comparing the blocks of the two images, if the difference between the blocks is less than a threshold value, the image is synthesized in the digital camera, wherein the overlapped portions of the first image and the second image are combined and stored in the first memory. Way. A digital camera having a camera and a display unit for displaying an image received from the camera, A second memory for temporarily storing an image, A first memory for finally storing the combined image information when two or more images are combined; In the panorama photographing mode, images are taken at preset time units, a predetermined portion of an image captured earlier in time is selected among the photographed images, and the selected predetermined portion is selected from an image immediately after the selected image. Inspecting a part coinciding with a part to detect a moving direction of the lens of the camera, and inspecting an error of an adjacent area between two images according to the moving direction of the lens, wherein the error between the adjacent areas is smaller than a preset threshold And a controller for combining the two images.