KR20100035381A - Image processing method and apparatus, and digital photographing apparatus - Google Patents

Abstract

PURPOSE: An image processing method, an image processing device, and a digital photographing device using the same are provided to minimize power consumption and processing time delay by only displaying a screen nail image and face screen nail image. CONSTITUTION: An image processing device temporarily stores RAW data in a memory(302). The image processing device generates a screen nail image through the extraction of the RAW data(304). If a face is detected from the screen nail image, the image processing device generates a face screen nail image with screen nail image size through the coordinate of detected face area(306,310). If the face is not detected, the image processing device displays the screen nail image(308).

Description

Image processing method and apparatus, and digital photographing apparatus

The present invention relates to image processing, and more particularly, to a method and apparatus for processing an image captured by a digital photographing apparatus capable of high speed still image capturing.

As a sensor capable of high speed still image capturing, for example, a high speed CMOS sensor, is commercially available, expectations for high speed still image capturing have increased. At present, some products are equipped with such sensors and are already commercialized and placed on the market.

In order to implement such a system, the amount of input data is several times higher than that of the camera signal processor, so that input processing is not immediately performed, and the data is first stored in a memory as a RAW file. Therefore, the original file stored in the memory can be developed to check the picture taken by the user.

This high speed shooting provides the user with the advantage of obtaining multiple images, but there are some inconveniences. Among them, the most important point is to select and store the acquired images to reduce the storage space. Moreover, when processing and storing all the obtained full resolution images through an image processor, a delay of 5 to 10 times is generally caused by a delay of processing speed and a delay of a storage medium. For example, in order to JPEG-compress an image taken for 1 second and store it in a flash storage medium, a general camera system may need about 10 times as much as 10 times. This is the part that makes the user feel very uncomfortable.

In order to solve this problem, the existing screen nail form is not processed without processing all the images stored in the full resolution RAW form so that the user can conveniently select and save the acquired images without developing all the acquired images. A method has been proposed in which a picture is reduced so that only the reduced image is developed to be shown to the user in advance so that the image can be selected.

However, this method has a problem in that the developed image is very small, and it is very difficult to check whether the focus is correctly performed in the important person photographing in the real picture, and whether there is no noise or vibration.

The present invention has been made to solve the problems of the conventional method, and provides an image processing method and apparatus which can show a face screen nail image by separating a face part together with a reduced screen nail image instead of showing the entire image. The purpose is to.

In addition, it is possible to conveniently check the quality of the image while minimizing the power consumption and processing time delay caused by processing all the full-resolution images. There is another purpose to provide.

In order to solve the above technical problem, an image processing method according to an embodiment of the present invention comprises the steps of generating a screen nail image using the RAW data output from the image sensor; Detecting a face region from the screen nail image; And generating a face screen nail image using the detected coordinates of the face region from the screen nail image.

Preferably, the image sensor is a high speed CMOS image sensor.

Preferably, the method further comprises displaying the face screen nail image.

The method may further include displaying a screen nail image corresponding to the face screen nail image when the user selects the displayed face screen nail image.

The method may further include displaying the screen nail image and the face screen nail image together.

Advantageously, the method further comprises storing RAW data output from said high speed CMOS image sensor in a memory.

Preferably, the method may further include resizing the size of the face area to fit the face screen nail image when the size of the detected face area is variable.

Preferably, the method further comprises the step of recording an original image corresponding to the selected face screen nail image or screen nail image on a predetermined recording medium when the displayed face screen nail image or screen nail image is selected by the user. It is characterized by.

In accordance with another aspect of the present invention, there is provided an image processing method including setting an ROI of a predetermined subject and photographing the subject; Generating a screen nail image using the RAW data output from the high speed CMOS image sensor; Generating a region of interest screen nail image using the set coordinates of the region of interest from the screen nail image; And displaying the region of interest screen nail image.

The displaying step may further include displaying a screen nail image corresponding to the ROI screen nail image when the user selects the displayed ROI screen nail image.

Preferably, the region of interest is selected by a user's selection on a preview screen.

In order to achieve the another technical problem, a recording medium including a program for executing the method on a computer according to another embodiment of the present invention.

According to another aspect of the present invention, an image processing apparatus includes a screen nail generator configured to generate a screen nail image using RAW data output from an image sensor; A face detector for detecting a predetermined face area from the screen nail image; And a face screen nail generator configured to generate a face screen nail image using coordinates of the detected face region from the screen nail image.

Preferably, the image sensor is characterized in that a high speed CMOS image sensor.

Preferably, the apparatus further comprises a display unit for selectively displaying the screen nail image and the face screen nail image.

Preferably, the apparatus includes a region of interest detector for detecting a region of interest selected by a user on a preview screen; And a region of interest screen nail generator configured to generate a region of interest screen nail image using the detected coordinates of the region of interest from the screen nail image.

The display device may further include a display configured to selectively display the screen nail image and the ROI nail image.

In accordance with still another aspect of the present invention, there is provided a digital photographing apparatus including a memory configured to temporarily store RAW data output from a high speed CMOS image sensor; A screen nail generator configured to generate a screen nail image using the stored RAW data; A face detector for detecting a predetermined face area from the screen nail image; A face screen nail generator configured to generate a face screen nail image by using the coordinates of the detected face region from the screen nail image; An image signal processor configured to process the generated screen nail image and the face screen nail image; And a display unit for selectively displaying the developed screen nail image and the face screen nail image.

Preferably, the apparatus includes a region of interest detector for detecting a region of interest selected by the user on a preview screen; And a region of interest screen nail generator configured to generate a region of interest screen nail image using the detected coordinates of the region of interest from the screen nail image.

The display unit may be configured to selectively display the screen nail image and the region of interest screen nail image.

According to an embodiment of the present invention, by developing and displaying only the screen nail image and the face screen nail image, the quality of the corresponding image can be conveniently checked while minimizing power consumption and processing time delay caused by processing both full resolution images. In the high speed still picture shooting, the user can conveniently select and store the image.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, only parts necessary for understanding the operation according to the present invention will be described, and descriptions of other parts may be omitted so as not to distract from the gist of the present invention.

In addition, terms and words used in the following description and claims should not be construed to be limited to ordinary or dictionary meanings, but are to be construed in a manner consistent with the technical idea of the present invention As well as the concept.

A digital photographing apparatus and a photographing method according to the first embodiment of the present invention will be described. 1 is a diagram for explaining the configuration of a digital photographing apparatus 100 according to a first embodiment of the present invention. Hereinafter, the structure of the imaging device which concerns on 1st Example of this invention is demonstrated using FIG.

As shown in FIG. 1, the digital photographing apparatus according to the first embodiment of the present invention includes a zoom lens 102, an aperture 104, a focus lens 106, driving devices 102a, 104a, 106a, and CMOS (Complementary). Metal Oxide Semiconductor (CIM) device 108, CDS (Correlated Double Sampling) circuit 110 integrated with amplifier, A / D converter 112, image input controller 114, image signal processor 116, compression processor 120 On Screen Display (OSD) 121, Liquid Crystal Display (LCD) Driver 122, LCD 124, Timing Generator 126, Central Processing Unit 128, Control Panel 132, Shutter Button 133, memory 134, video random access memory (VRAM) 136, media controller 138, recording media 140, motor drivers 142a, 142b, 142c, and flash 144 do.

The zoom lens 102 is a lens in which the focal length is continuously changed by moving back and forth in the optical axis direction by the drive device 102a. The zoom lens 102 photographs by changing the size of the subject. The diaphragm 104 adjusts the amount of light entering the CMOS element 108 by the driving device 104a when the image is captured. The focus lens 106 adjusts the focus of the subject by moving back and forth in the optical axis direction by the drive device 106a.

In the present embodiment, only one zoom lens 102 and one focus lens 106 are shown. The number of the zoom lenses 102 may be two or more, and the number of the focus lenses 106 may be two or more.

The CMOS element 108 is an element for converting light incident from the zoom lens 102, the aperture 104, and the focus lens 106 into an electrical signal. In this embodiment, the time for extracting the electric signal by controlling the incident light by the electronic shutter is adjusted. The time for extracting the electric signal by controlling the incident light by using the mechanical shutter may be adjusted. In the present embodiment, the image capturing unit is constituted by the zoom lens 102, the aperture 104, the focus lens 106, and the CCD element 110. Moreover, the set of imaging sections is not limited to this, The structure which does not include the zoom lens 102 and the diaphragm 104 may be sufficient.

In this embodiment, the CMOS element 108 is used, and preferably, it is a high speed CMOS image sensor. Here, the digital photographing apparatus 100 may shoot 60 images per second continuously through the high speed CMOS image sensor. The CDS circuit 110 is a circuit in which a CDS circuit which is a kind of sampling circuit that removes noise of an electrical signal output from the CMOS element 108 and an amplifier that amplifies the electrical signal after removing the noise are integrated. In the present embodiment, the digital photographing apparatus 100 is configured by using a circuit in which the CDS circuit and the amplifier are integrated, but the CDS circuit and the amplifier may be configured as separate circuits.

The A / D converter 112 converts an electric signal generated by the CMOS element 108 into a digital signal to generate RAW data of an image.

The image input controller 114 controls the input of the RAW data generated by the A / D converter to the memory 134.

The image signal processing unit 116 adjusts the gain correction and the white balance of the amount of light with respect to the electric signal output from the CMOS element 108. The image signal processing unit 116 acquires exposure data of the captured image. The exposure data includes an in-focus evaluation value (AF evaluation value) and an AE (Auto Exposure) evaluation value, and the image signal processing unit 116 calculates the in-focus evaluation value and AE evaluation value.

The compression processing unit 120 performs compression processing for compressing the image developed by the image signal processing unit 116 into image data of an appropriate format. The compressed form of the image may be a reversible type or a non-reversible type. As an example of a suitable format, you may convert into the JPEG (Joint Photographic Experts Group) format or the JPEG 2000 format.

The OSD 121 is a function of displaying the setting screen of the digital photographing apparatus 100 on the LCD 124. The LCD 124 performs live view display before the shooting operation, various setting screens of the imaging device 100, display of the captured image, and the like. The display of the image data and various information of the imaging device 100 on the LCD 124 is performed via the LCD driver 122.

The timing generator 126 inputs a timing signal to the CMOS element 108. The shutter speed is determined by the timing signal from the timing generator 126. That is, the driving of the CMOS element 108 is controlled by the timing signal from the timing generator 126, and the image light from the subject is incident within the time period in which the CMOS element 108 is driven, thereby causing the electric power to be the basis of the image data. The signal is generated.

The CPU 128 issues a signal system command to the CMOS element 108, the CDS circuit 110, or the like, or issues an operation system command for the operation of the operation unit 132. In the present embodiment, only one CPU is included. Alternatively, the signal system command and the operation system command may be executed by separate CPUs.

The operation unit 132 is arranged with a member for operating the imaging device 100 or for performing various settings during shooting. On the member disposed on the operation unit 132, a power button, a cross key for selecting the photographing mode or the photographing drive mode, and setting the soft focus effect, the selection button, and the like are arranged. The shutter button 133 is for performing a photographing operation. The shutter button 133 combines the subject in a half-pressed state, and captures the subject in a fully-pressed state.

The memory 134 is an example of an image storage unit, which temporarily stores a photographed image or an image synthesized by the image combining unit 118. The memory 134 has a storage capacity enough to store a plurality of images. Reading and writing of an image into the memory 134 is controlled by the image input controller 114.

The VRAM 136 retains the contents displayed on the LCD 124, and the resolution and the maximum number of colors of the LCD 124 depend on the capacity of the VRAM 136.

The recording medium 140 is an example of an image recording unit that records a photographed image. Input and output to the recording medium 140 are controlled by the media controller 138. As the recording medium 140, a memory card which is a card-type storage device for recording data in a flash memory can be used.

The motor drivers 142a, 142b, 142c control the driving devices 102a, 104a, 106a for operating the zoom lens 102, the aperture 104, and the focus lens 106. By operating the zoom lens 102, the aperture 104, and the focus lens 106 via the motor drivers 142a, 142b, and 142c, the size of the subject, the amount of light, and the focus are adjusted.

The flash 144 is for illuminating a subject brightly when shooting outdoors at night or in a dark place. When performing flash photography, a light emission command is issued from the CPU 128 to the flash device 144, and the flash 144 is caused to emit light in accordance with the light emission command from the CPU 128, in accordance with the light emitted by the flash 144. The subject is shining brightly.

In the above, the digital photographing apparatus 100 according to the first embodiment of the present invention has been described with reference to FIG. 1. Next, the configuration of the CPU 128 of the imaging device according to the first embodiment of the present invention will be described.

FIG. 2 is a diagram for explaining the configuration of the CPU 128 according to the first embodiment of the present invention. Hereinafter, the configuration of the CPU according to the first embodiment of the present invention will be described with reference to FIG. 2.

The CPU 128 controls the overall operation of the digital photographing apparatus 100. In a preferred embodiment of the present invention, the CPU 128 may include a screen nail generator 201, a face detector 202, a region of interest (ROI) detector 203, a face screen nail generator 204, and a region of interest screen nail. The generation unit 205 is included. Here, while the CPU 128 is described as including both the configuration 202 and 204 related to the face region extraction and the configuration 203 and 205 related to the region of interest extraction, the CPU 128 is the face region extraction and the face screen nail. It may consist of only the configuration 202 and 204 relating to the generation or only the configuration 203 and 205 relating to the region of interest extraction and the generation of the region of interest screen nail.

The screen nail generator 201 generates a screen nail image by using the RAW data output from the CMOS device 108. Here, the screen nail image is an image compressed to fit the size of the visible area of the LCD screen separately from the thumbnail image. The screen nail image is used to accelerate the playback of the digital photographing apparatus, and may have a resolution of 640 × 480, for example. In general, a digital photographing apparatus capable of high-speed still photography captures 60 frames per second, 60 images per second, and makes 60 screen nail images for the user. RAW data output from the CMOS 108 is stored in the memory 134 through the CDS AMP 110, the A / D converter 112, and the image input controller 114. Processing of RAW data required for screen nail generation is performed by storing RAW data necessary for screen nail generation separately in the memory 134 together with storing the full resolution RAW data. In addition, the screen nail generator 201 may generate the screen nail image by extracting the RAW data necessary for the screen nail image by importing the full resolution RAW data output from the CMOS device 108 temporarily stored in the memory 134. . Here, the screen nail image is developed through the image signal processor 116 and the compression processor 120, and is compressed and displayed on the LCD 120.

The face detector 202 detects a predetermined face area from the screen nail image. The face nail detector 202 may input a screen nail image generated from the screen nail generator 201 or a screen nail image developed through the image signal processor 116. In general, face detection detects a face by receiving an image having a size of about a screen nail image, for example, about 640 × 480. Face detection detects where a face is in the input image. The face detection unit 202 compares the feature data of the face stored in advance with the input image data, recognizes whether there is data of the face in the input image, and if it is determined that the face data exists, the image data of the face is located at a certain position. Recognize it. There are many techniques for face region detection or algorithm, and it can be used for face detection according to a preferred embodiment of the present invention. For example, a face region may be detected using a motion vector technique, a feature point detection technique, an Adaboost learning technique, or the like.

The face detector 202 detects the coordinates of the face area from the screen nail image, and stores the detected face area coordinates every frame.

The face screen nail generator 204 generates a face screen nail image by using coordinates of a face region detected from the screen nail image. Here, the face screen tomorrow image is an image in which the detected face region is enlarged to the size of the screen nail image. In the preferred embodiment of the present invention, the description is limited to the face screen nail image, but not only the face but also various objects may be detected, and the object screen nail image may be generated in the same manner. In addition, when the face size is variable, the resizing process may be performed so that the face detected by the face detection unit 202 may be included in the face screen nail image. The scaler for performing such a resizing may be included in the CPU 128 or the image signal processor 116.

The face screen nail image generated by the face screen nail generator 204 is displayed every frame through the LCD 124. The user may selectively store only the desired picture in the recording media 140 by displaying a predetermined check box so that only the picture well displayed through the displayed face screen nail image may be selected.

The region of interest detector 203 receives the screen nail image and detects a region of interest preset by the user. Here, the ROI is an area preset by the user on the preview screen through the operation unit 132. The region of interest is set as a rectangular window or various windows in a desired area among the overall composition of the subject to be photographed by the user.

The region of interest screen nail generator 205 generates the region of interest screen nail image from the screen nail image by using the coordinates of the region of interest detected by the region of interest detector 203.

In the preferred embodiment of the present invention, the CPU 128, the image signal processing unit 116 and the compression processing unit 120 has been described separately, of course, all of them can be implemented together in the DSP chip.

The face screen nail generator 203 and the region of interest screen nail generator 205 do not generate face screen nails and region of interest screen nails from full-resolution RAW data, but face screen nails from a reduced sized screen nail image. Generating the screen nail image with the image and the region of interest can be processed at high speed. In addition, since the user may select and store only the well-known picture by checking the face portion or the region of interest together with the screen nail image, which is a reduced image of the entire image, the user may reduce power consumption and storage time.

3 is a flowchart illustrating an image processing method according to another exemplary embodiment.

Referring to FIG. 3, in step 300, high speed still image shooting is performed. In step 302, the RAW data is temporarily stored in the memory. In step 304, RAW data stored in the memory is extracted to generate a screen nail image. In step 306, the face is retrieved from the screen nail image. If a face is detected, the process proceeds to step 310 and a face screen nail image having a screen nail image size is generated using the detected coordinates of the face region. If no face is detected, a screen nail image is displayed.

In operation 312, a face screen nail image and a screen nail image are displayed, and only a face screen nail image is displayed, and if selected by the user, a screen nail image corresponding to the face screen nail image is displayed. In addition, the face screen nail image and the screen nail image may be displayed together. 5 is a diagram for describing a screen nail image and a face screen nail image according to another exemplary embodiment. Referring to FIG. 5, 60 pictures are included in one screen. Here, the pictures were taken at high speed at 60 frames per second, and screen nail images and face screen nail images are displayed together frame by frame. The user can check the composition and contrast of the entire screen through the screen nail image. After checking the face shot nail image to see if the focus is correct or if noise or blur occurs, selectively save the desired frame. Can be. In step 314, when the user selects a face screen nail image or a screen nail image of a specific frame, in step 316, the selected image is stored. Here, the original screen image of the face screen nail image or the screen nail image is stored in the recording medium.

4 is a flowchart illustrating an image processing method according to another exemplary embodiment.

Referring to FIG. 4, in step 400, a region of interest is set. The user can select only a specific part of the composition to be photographed on the preview screen. FIG. 6 is a diagram for describing a method of setting an ROI according to another exemplary embodiment. Referring to FIG. In the screen of FIG. 6, an area 600 including a house and a tree is set as a rectangular window.

In step 402, high speed still image shooting is performed. In step 404, the RAW data is temporarily stored in the memory. In step 406, a screen nail image is generated from RAW data. In step 408, the ROI preset by the user is detected. Here, the region of interest is the region 600 desired by the user described with reference to FIG. 6. In operation 410, an ROI screen nail image is generated from the screen nail image. In operation 412, the ROI screen nail image and the screen nail image are displayed for each frame. FIG. 7 is a diagram illustrating a screen nail image and a region of interest screen nail image according to another exemplary embodiment. Referring to FIG. 7, 60 pictures are included in one screen. Here, the pictures are taken at high speed at 60 frames per second, and the screen nail image and the region of interest screen nail image are displayed together frame by frame. The user can check the composition and contrast of the entire screen through the screen nail image, and after checking whether the focus is correctly set in the preset area 600 or the noise or vibration occurred through the screen nail image of the region of interest, You can optionally save the desired frame.

In operation 414, when the user selects the ROI screen nail image or the screen nail image of the specific frame, in operation 416, the selected image is stored. Here, the region of interest screen nail image or the original image of the screen nail image is stored in the recording media.

The above embodiments have been described with reference to a digital camera as an example of a digital photographing apparatus to which the present invention can be applied, but is not limited thereto. Those skilled in the art will appreciate that the present invention can be applied to camera phones, personal digital assistants (PDAs), and portable multimedia players (PMPs) with added camera functionality and strobe.

Meanwhile, the present invention can be embodied as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored.

Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like, which may also be implemented in the form of carrier waves (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will understand that the present invention can be embodied in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown not in the above description but in the claims, and all differences within the scope should be construed as being included in the present invention.

1 is a schematic block diagram of a digital photographing apparatus 100 according to an exemplary embodiment.

FIG. 2 is a schematic block diagram of the CPU 128 shown in FIG.

3 is a flowchart illustrating an image processing method according to another exemplary embodiment.

4 is a flowchart illustrating an image processing method according to another exemplary embodiment.

5 is a diagram for describing a screen nail image and a face screen nail image according to another exemplary embodiment.

FIG. 6 is a diagram for describing a method of setting an ROI according to another exemplary embodiment. Referring to FIG.

FIG. 7 is a diagram illustrating a screen nail image and a region of interest screen nail image according to another exemplary embodiment.

<Explanation of symbols for the main parts of the drawings>

100: digital photographing device 108: CMOS image sensor

128: central processing unit (CPU) 201: screen nail generating unit

202: Face detector 203: Region of interest (ROI) detector

204: the face screen nail generator

205: region of interest screen nail generator

Claims (20)

Generating a screen nail image using the raw data output from the image sensor; Detecting a face region from the screen nail image; And And generating a face screen nail image using coordinates of the detected face region from the screen nail image. The method of claim 1, The image sensor, An image processing method comprising a high speed CMOS image sensor. The method of claim 2, And displaying the face screen nail image. The method of claim 3, wherein And when the user selects the displayed face screen nail image, displaying a screen nail image corresponding to the face screen nail image. The method of claim 2, And displaying the screen nail image and the face screen nail image together. The method of claim 2, And storing the RAW data output from the high speed CMOS image sensor in a memory. The method of claim 2, And resizing the size of the face area to match the face screen nail image when the size of the detected face area is variable. The method of claim 5, And when the displayed face screen nail image or screen nail image is selected by the user, recording an original image corresponding to the selected face screen nail image or screen nail image on a predetermined recording medium. . Setting a region of interest of a predetermined subject and photographing the subject; Generating a screen nail image using the RAW data output from the high speed CMOS image sensor; Generating a region of interest screen nail image using the set coordinates of the region of interest from the screen nail image; And And displaying the region of interest screen nail image. The method of claim 9, The display step, And displaying a screen nail image corresponding to the ROI screen nail image when the displayed ROI screen nail image is selected by the user. The method of claim 9, The region of interest is, And a user's selection on the preview screen. A recording medium having recorded thereon a program for executing a method according to any one of claims 1 to 11 on a computer. A screen nail generator configured to generate a screen nail image using RAW data output from the image sensor; A face detector for detecting a predetermined face area from the screen nail image; And And a face screen nail generator configured to generate a face screen nail image using coordinates of the detected face region from the screen nail image. The method of claim 13, The image sensor, An image processing apparatus, characterized by a high speed CMOS image sensor. The method of claim 14, And a display unit for selectively displaying the screen nail image and the face screen nail image. The method of claim 14, A region of interest detector configured to detect a region of interest selected by the user on the preview screen; And And a region of interest screen nail generator configured to generate a region of interest screen nail image using the detected coordinates of the region of interest from the screen nail image. The method of claim 16, And a display unit for selectively displaying the screen nail image and the region of interest screen nail image. A memory for temporarily storing RAW data output from the high speed CMOS image sensor; A screen nail generator configured to generate a screen nail image using the stored RAW data; A face detector for detecting a predetermined face area from the screen nail image; A face screen nail generator configured to generate a face screen nail image by using the coordinates of the detected face region from the screen nail image; An image signal processor configured to process the generated screen nail image and the face screen nail image; And And a display unit for selectively displaying the developed screen nail image and the face screen nail image. The method of claim 18, A region of interest detector configured to detect a region of interest selected by the user on the preview screen; And And a region of interest screen nail generator configured to generate a region of interest screen nail image using the detected coordinates of the region of interest from the screen nail image. The method of claim 19, The display unit, And selectively displaying the screen nail image and the region of interest screen nail image.

Images