KR101508977B1 - Apparatus, method and computer readable recording medium for editting the image automatically by analyzing an image - Google Patents

Abstract

The present invention relates to an image automatic editing apparatus for generating an image suitable for various screen configurations such as a thumbnail image by automatically editing a target image to be edited, ; An image analysis module that receives the at least one image to be edited from the content information database and detects at least one main area of the entire area of the provided image to be edited; An image configuration information database storing configuration information of a layout in which an edited image is to be placed; And an image editing module for automatically generating an edited image including a main area detected by the image analysis module according to the configuration information stored in the image configuration information database.

Description

[0001] APPARATUS, METHOD AND COMPUTER READABLE RECORDING MEDIUM FOR EDITING THE IMAGE AUTOMATICALLY BY ANALYZING AN IMAGE [0002]

[0001] The present invention relates to an apparatus for automatically editing an image, and more particularly, to an apparatus for automatically editing an image to be edited, An automatic editing apparatus, a method, and a computer-readable recording medium.

Today, the widespread use of the Internet has led to the rapid development of wireless mobile communication technology beyond the fixed line, and in the real world, information retrieval on the Internet through portable terminals such as mobile phones, PDAs, and hand- It became possible regardless of time and place.

There is a demand for consuming contents in various layouts such as a magazine or an e-book in a digital medium of a Web or an application. For example, for tablet applications such as flip boards, images can be laid out to support existing web content.

At this time, the photographic images included in the web content play an important role in the layout. In order to expose the original image in the form of a thumbnail image, a part of the original photographic image is cropped or overlayed with text or another photograph Various layouts are generated.

Conventionally, when producing such a thumbnail image, if the aspect ratio of the original image is not equal to the aspect ratio of the thumbnail, a thumbnail image may be generated manually by hand to include the main region of the photo in the thumbnail image, Directly edited the image to create a layout. In this way, the original image image is simply re-sized and applied to the content layout even if it is automated without manual production.

This manual editing of thumbnail images is not only resource-intensive, but also works when thumbnails of different shapes, sizes, and aspect ratios (ie, aspect ratio) are required on various display screens There is a problem that the image must be re-edited again.

On the other hand, as a technique for automatically generating a thumbnail or the like from such an original image, Korean Patent Laid-Open Publication No. 2004-0044373 entitled " Video thumbnail and its generating method (Hewlett Packard) A method for generating an animated video thumbnail is disclosed.

However, such an automatic resizing of an image may result in the image not being proportional to the specific content layout to be positioned, or the major area of the image being resized, resulting in a lower readability on a smaller screen such as a smartphone .

Particularly, with the emergence of smart phones, tablets, and smart TVs, various sizes of screens are emerging, so that there is a need for a technology capable of automatically generating dynamic layouts suitable for various screen sizes.

[Patent Document 1] Korean Patent Laid-Open Publication No. 2004-0044373 Video thumbnail and method of generating the same (Hewlett Packard) 2004.05.28

An object of the present invention is to provide an automatic image editing apparatus and method capable of automatically generating an image suitable for various layout configurations by analyzing an image to be edited and dividing the image into main areas and blank areas, An apparatus, a method, and a computer-readable recording medium.

It is another object of the present invention to provide an image processing method and apparatus capable of automatically generating an image suitable for various layout configurations by analyzing an image to be edited and dividing the image into a main area and a margin area, An image automatic editing apparatus, a method, and a computer-readable recording medium.

It is still another object of the present invention to provide an image processing apparatus and method capable of automatically generating an image suitable for various layout configurations by analyzing an image to be edited and dividing the image into a main area and a margin area and overlaying An apparatus and method for automatic image editing by image analysis, and a computer-readable recording medium.

It is still another object of the present invention to provide a method and apparatus for analyzing a plurality of images to be edited and dividing the image into a main area and a margin area, overlaying a plurality of cropped images after cropping a main area of each image to be edited, An image automatic editing apparatus, a method, and a computer-readable recording medium by image analysis capable of automatically generating an image.

It is still another object of the present invention to provide an image editing method and apparatus capable of automatically creating an image suitable for various layout configurations by adding a separate margin area to an image to be edited, An apparatus, a method, and a computer-readable recording medium.

In order to achieve the above-described object of the present invention and to achieve the specific effects of the present invention described below, the characteristic structure of the present invention is as follows.

According to an aspect of the present invention, an apparatus for automatically editing an image by image analysis includes: a content information database storing at least one image to be edited; An image analysis module that receives the at least one image to be edited from the content information database and detects at least one main area of the entire area of the provided image to be edited; An image configuration information database storing configuration information of a layout in which an edited image is to be placed; And an image editing module for automatically generating an edited image including a main area detected by the image analysis module according to the configuration information stored in the image configuration information database.

Preferably, the image analysis module includes an object detection unit that detects at least one object among the entire area of the image to be edited, and determines an area including the detected object as a main area.

Preferably, the image analysis module calculates a focus value and a contrast value for each pixel in the image to be edited, calculates a saliency value based on the calculated focus value and the contrast value, And a pattern analyzer for detecting at least one main area according to the distribution of the values.

Preferably, the image analysis module includes a margin detection unit that determines an area excluding the detected main area as a margin area among the entire area of the image to be edited.

Preferably, the margin detecting unit sets a region excluding the detected main region as a first-step margin region, divides the region into a plurality of regions having similar pixel values with respect to the first-step margin region, An area having a change rate of a pixel value lower than a first threshold value set in advance and having a size or shape higher than a preset second threshold value is set as a two-step margin area.

Preferably, the first-stage blank area and the second-stage blank area are applied differently in the overlay editing process.

Preferably, the image editing module includes an image cropping unit for cropping the main area detected by the image analysis module from the image to be edited.

Advantageously, the image editing module includes a text overlay portion for overlaying the text in the margin region.

Preferably, the color of the text overlaid on the blank area is set in consideration of the color of the blank area.

Preferably, the content of the text overlaid on the margin area is any one of a caption, a title, and an article content of a content related to the editing target image.

Preferably, the image editing module includes an image overlay portion for overlaying a main area of the second image in a margin area of the first image with respect to a plurality of editing images where the main area is detected.

The image editing module may further include a margin adding unit for adding a margin area to the image to be edited if the area other than the main area is not sufficient to overlay the text.

Preferably, the image editing module includes an image resizing unit for resizing the size or aspect ratio of the edited image including the main area according to the configuration information.

According to another aspect of the present invention, there is provided a method of automatically editing an image by image analysis, comprising: receiving an image to be edited in an image analysis module; Detecting at least one main area of the entire area of the provided editing target image in the image analysis module; And generating an edited image by automatically editing the image in accordance with the configuration information of the preset layout so that the detected main area is included in the image editing module.

Preferably, the step of detecting the main area may include detecting at least one object among the entire area of the image to be edited, and determining the area including the detected object as a main area.

Advantageously, the step of detecting the main region comprises the steps of: calculating a focus value and a contrast value for each pixel in the image to be edited; Calculating a saliency value based on the calculated focus value and the contrast value; And detecting at least one main region according to the distribution of the calculated persistence value.

Preferably, after the step of detecting the main area, the step of determining an area excluding the detected main area as the blank area among the entire area of the image to be edited is further included.

Preferably, the step of judging the blank area includes: setting an area excluding the detected main area as a blank area of a first stage; Dividing the image into a plurality of regions having similar pixel values for the first-stage blank region; And setting a region having a change rate of a pixel value lower than a preset first threshold value among the divided regions and having a size or shape higher than a preset second threshold value as a two-step margin region.

Preferably, the first-stage blank area and the second-stage blank area are applied differently in the overlay editing process.

Advantageously, the step of generating the edited image comprises the step of cropping the detected key area from the edited image.

Advantageously, the step of creating the edited image comprises overlaying the text in the margin area.

Preferably, the color of the text overlaid on the blank area is set in consideration of the color of the blank area.

Preferably, the content of the text overlaid on the margin area is any one of a caption, a title, and an article content of a content related to the editing target image.

Advantageously, the step of generating the edited image comprises overlaying the main area of the second image in the margin area of the first image for a plurality of edited images for which the main area is detected.

Advantageously, the step of generating the edited image further includes adding a margin area to the image to be edited if there is not enough space to overlay the text in the area other than the main area of the entire area .

Advantageously, the step of generating the edited image includes the step of resizing the size or aspect ratio of the edited image including the main area according to the configuration information.

Meanwhile, the information for performing the image auto-editing procedure by the image analysis may be stored in a recording medium readable by the server computer. Such a recording medium includes all kinds of recording media in which programs and data are stored so that they can be read by a computer system. Examples include ROMs (Read Only Memory), Random Access Memory, CD (Compact Disk), DVD (Digital Video Disk) -ROM, magnetic tape, floppy disk, optical data storage device, (For example, transmission over the Internet). Such a recording medium may also be distributed over a networked computer system so that computer readable code in a distributed manner can be stored and executed.

As described above, according to the present invention, the target image to be edited is automatically edited according to a specific content layout rather than manually, thereby eliminating the disadvantages of manual image editing.

In addition, when a thumbnail image having various shapes, sizes, and aspect ratios is required, a dynamic layout corresponding to various screen sizes can be automatically generated by automatically editing a thumbnail image corresponding to a corresponding condition according to the present invention .

In addition, according to the present invention, a main area of a photographic image is automatically judged and edited, thereby making it possible to automatically generate an image having excellent readability even after re-sizing without missing a main part.

In addition, according to the present invention, an effective thumbnail image can be automatically generated by automatically analyzing a non-important area in a photograph and overlaying caption, title, and article content of a photograph or related content on an unimportant area There is an advantage.

In addition, according to the present invention, it is possible to automatically generate various layouts such as a dynamic magazine by automatically determining a main area of a plurality of editing object images, and cropping and overlaying a plurality of main areas .

1 is a view showing a concept of automatic image editing by image analysis according to the present invention.
2 is a block diagram illustrating a detailed configuration of an automatic image editing apparatus by image analysis according to an embodiment of the present invention.
3 is a flowchart illustrating an image auto-editing process by image analysis according to an embodiment of the present invention.
4 is a flowchart showing the detailed procedure of the main area detection according to the embodiment of the present invention.
FIG. 5 is a flowchart illustrating a main area detection procedure by image pattern analysis according to an embodiment of the present invention.
6A and 6B are views showing an image in which a detected main area is displayed according to an embodiment of the present invention.
7 is a flowchart illustrating a procedure for detecting a blank area according to an embodiment of the present invention.
FIG. 8A is a diagram illustrating an image in which a blank region of a first stage according to an embodiment of the present invention is displayed. FIG.
8B is a view showing an image in which a two-step margin region according to an embodiment of the present invention is displayed.
9 is a flowchart illustrating a procedure for cropping an original image according to the first embodiment of the present invention.
10 is a view showing an image cropped from an original image according to the first embodiment of the present invention.
11 is a flowchart illustrating a procedure for overlaying text in a margin area according to a second embodiment of the present invention.
12 is a diagram illustrating an image in which text is overlaid in a margin area according to a second embodiment of the present invention.
13 is a diagram showing an image in which a margin area is added to an original image according to the third embodiment of the present invention.
14 is a view showing an image in which a plurality of images are overlaid according to a fourth embodiment of the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which the claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

The present invention discloses an apparatus and method for automatic image editing by image analysis capable of automatically detecting a position of a main part in an image such as a photograph and automatically editing an image in accordance with a desired layout around a detected main part .

For example, according to an embodiment of the present invention, after a position of a main part of a photograph is automatically detected, a detected main part is cropped, and a thumbnail image is generated by adjusting the aspect ratio according to a layout to be produced, It is possible to automatically generate an image of a desired layout without missing a part.

In addition, according to the embodiment of the present invention, it is possible to automatically analyze an unimportant area (e.g., a blank area) in an image such as a photograph and automatically search for a caption, a title, By overlaying and overlaying on the non-critical area, or overlaying with another image, various types of layouts such as dynamic magazines can be automatically generated.

In the present invention, an editing target image (i.e., a pre-editing image) for generating a desired image (e.g., a thumbnail image) by automatic analysis and editing is referred to as an 'original image' It is the best concept including paintings, cartoons and so on.

In addition, 'crop' used in the present invention to be described later means to cut out a selected area (for example, a main area) selected from all the areas of a specific image, and 'overlay' Which means editing.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

First, the structure of a concept and apparatus according to an embodiment of the present invention will be described with reference to FIG. 1 and FIG. 2, and then a procedure according to an embodiment of the present invention will be described in detail with reference to FIG.

Concept of image auto-editing

1 is a view showing a concept of automatic image editing by image analysis according to the present invention. Referring to FIG. 1, an original image (i.e., an image to be edited) for editing an image to be composed on a web page is stored in content information (i.e., an article database). In addition, the content information database may store not only the original image but also the caption, title, and article content of the content related to the original image linked together.

First, when an image of content to be analyzed stored in the content information database is fed, image analysis is performed in various ways. At this time, according to the embodiment of the present invention, the original image is analyzed to detect a main area including a face or an object, and a blank area which is not a main area.

For example, an object included in the original image is detected by various methods, and an area including the detected object is set as a main area. At this time, the remaining area excluding the area set as the main area among the entire images may be set as a blank area. If the entire image area is divided into the main area and the blank area, the information such as the position, color, brightness, and optimal horizontal / vertical ratio of the main area and the position, color, brightness, Information such as a ratio can be generated. As such, the generated information can be referred to later in image editing.

In addition, the pattern analysis method may be applied as another embodiment for distinguishing the main area and the blank area. For example, the saliency value may be calculated by the focus value and the contrast value of each pixel in the image, and the main area may be detected according to the distribution of the calculated saliency value.

As a method of detecting a blank area, a remaining area may be set as a blank area after the main area is detected as described above. According to an embodiment of the present invention, a blank area is divided into a blank area in step 1 and a blank area in step 2 You can also implement different editing methods by separating them with margins.

As described above, if the main area and the blank area are separated, the image can be edited according to the layout to be applied to a web page or the like according to various embodiments. Accordingly, the analyzed image is edited with reference to layout information (for example, image size, aspect ratio, resolution, shape, shape, orientation, etc.) to be configured in the web page for image editing.

For example, according to the first embodiment of the present invention, an image (for example, a thumbnail image) to be applied to a web page can be produced by cropping the main area. At this time, the cropped image may be further resized to generate an image having a desired size and ratio.

In addition, according to the second embodiment of the present invention, the text can be overlaid on the margin area of the analyzed image. At this time, the text may be a caption, a title, an article content, etc. related to an original image linked to the original image in the contents information database, and the present invention is not limited thereto.

In addition, according to the third embodiment of the present invention, when the margin area of the analyzed image is not sufficient to overlay the text, a margin area may be automatically generated and added. Therefore, an edited image can be generated by overlaying the text or other text or the like as in the second embodiment in the added margin area.

In the second and third embodiments, similarly to the first embodiment, an image in which a text is overlaid is resized to generate an image of a desired size and ratio suitable for the layout.

Further, according to the fourth embodiment of the present invention, an edit image can be generated by overlaying a plurality of original images with each other. For example, when two original images are to be overlaid on each other, the main regions of each original image are detected according to the embodiment of the present invention, and the detected main regions of each original image are overlapped with each other, Lt; / RTI >

In this way, the edited image produced according to each embodiment is transmitted to the web server, and the web server arranges the edited image according to the layout in the web server. That is, a web page corresponding to a screen size of a user terminal (e.g., a phone, a tablet, a notebook, a PO, etc.) is configured and provided to the user terminal.

Hereinafter, an apparatus according to an embodiment of the present invention will be described with reference to FIG. 2, and a method according to an embodiment of the present invention will be described with reference to FIG.

<Automatic image editing device by image analysis>

2 is a block diagram illustrating a detailed configuration of an automatic image editing apparatus by image analysis according to an embodiment of the present invention. Referring to FIG. 2, the automatic image editing apparatus according to an embodiment of the present invention may include an image analysis module 220 and an image editing module 230. The image analysis module 220 may include an object detection unit 220, a pattern analysis unit 220 and a margin detection unit 223. The image editing module 230 may include an image cropping unit 231, A text overlay unit 232, an image overlay unit 233, a margin adding unit 234, and an image resizing unit 235. [

In addition, the automatic image editing apparatus may include at least one database such as a content information database 210, an image configuration information database 240, and an edit image database 250.

At least one original image for editing (i.e., an image to be edited) is stored in the contents information database 210. In accordance with an embodiment of the present invention, caption, title, May be mapped together and stored.

The image analysis module 220 receives an original image to be edited from the content information database 210 and detects at least one main area of the entire image area through image analysis. In this case, the object detecting unit 221 may selectively detect the main area according to the embodiment of the present invention, and the pattern analyzing unit 222 may detect the main area. In addition, according to an embodiment of the present invention, at least one margin area of the entire image area can be detected by the margin detection unit 223.

For example, when the image analysis module 220 receives an image of content to be analyzed stored in the content information database 210, image analysis is performed by various methods described below. That is, according to the embodiment of the present invention, the original image is analyzed to detect a main area including a face or an object, and a blank area that is not a main area.

More specifically, the object detecting unit 221 detects at least one object (e.g., face, text, object, etc.) included in the original image by various methods, and detects an area containing the detected object Set as main area. At this time, the remaining area excluding the area set as the main area among the entire images may be automatically set as the blank area.

If the entire image area is divided into the main area and the blank area, the information such as the position, color, brightness, and optimal horizontal / vertical ratio of the main area and the position, color, brightness, Information such as a ratio can be generated. As such, the generated information can be referred to later in image editing.

Also, the main area detection by the pattern analyzer 222 may be performed as another method for detecting the main area. For example, the saliency value may be calculated by the focus value and the contrast value of each pixel in the image, and the main area may be detected according to the distribution of the calculated saliency value. A detailed description thereof will be given later in the description of FIG.

As described above, after the main area is detected by the object detecting unit 221 or the pattern analyzing unit 222, the remaining area may be set as the blank area. In accordance with another embodiment of the present invention, the blank detecting unit 223 The margins can be further segmented and detected. For example, the margin detecting unit 223 may divide the margin area into one-step margins and two-step margins according to an embodiment of the present invention, and the first-step margin and the second-step margin may be edited by different editing methods .

If the main image area and the blank area are separated in the image analysis module 220, the image editing module 230 may divide the original image into a plurality of regions according to various embodiments (for example, according to four embodiments) Edit the image to fit the layout to be applied to the web page or the like. In other words, the image editing module 230 analyzes the layout information (for example, the size, the aspect ratio, the resolution, the shape, the shape, the direction, etc.) of the layout to be configured in the web page stored in the image configuration information database 240 And the image analyzed by the image analysis module 220 is edited.

For example, in the image cropping unit 231, an image (for example, a thumbnail image) to be applied to a web page is produced by cropping the main region classified by the image analysis module 220 according to the first embodiment of the present invention. At this time, the cropped image may be resized through the image resizing unit 235 to generate an image having a desired size and ratio.

In addition, the text overlay unit 232 edits the original image by overlaying the text in the margin area of the image separated by the image analysis module 220 according to the second embodiment of the present invention. At this time, the text may be a caption, a title, an article content, and the like related to an original image mapped with the original image in the contents information database 210, as described above, but the present invention is not limited thereto .

Also, according to the third exemplary embodiment of the present invention, when the margin area of the analyzed image is not sufficient to overlay the text, the margin adding unit 234 may automatically generate and add a margin area. Therefore, an edit image can be generated by overlaying the text as in the second embodiment in the margin area added by the margin adding unit 234.

On the other hand, the image edited by the text overlay unit 232 and the margin adding unit 234 is resized by the image resizing unit 235 as in the first embodiment, so that an image of a desired size and ratio Can be generated.

In addition, the image overlay unit 233 can generate an edited image by overlaying a plurality of original images with each other according to the fourth embodiment of the present invention. For example, in order to overlay two original images, the image analysis module 220 receives a plurality of images according to an embodiment of the present invention and detects a main area of each original image, 233 may overlay the detected main regions of the original images so that they are not overlapped with each other, thereby generating a desired edited image.

In this way, the edited image produced by the image editing module 230 according to each embodiment is stored in the edited image database 250, and the stored edited image is applied to the web server to form various types of web pages do. That is, as described above, a web page corresponding to the screen size of the user terminal (e.g., a phone, a tablet, a notebook, a PO, etc.) can be configured and provided to the user terminal.

<Automatic image editing by image analysis>

3 is a flowchart illustrating an image auto-editing process by image analysis according to an embodiment of the present invention. Referring to FIG. 3, the main image is detected (S301) by analyzing the original image fed according to the embodiment of the present invention as described above. At this time, if the main area is to be cropped and edited (S302), the main area is cropped to generate an image according to the layout of the corresponding page to constitute the web page. At this time, the main region cropped by the image generation method may be generated by resizing, and the main region may be cropped considering the layout to which the image is applied. Thus, the generated edited image (e.g., thumbnail image) is applied to the corresponding layout of the web page.

On the other hand, the original image may not be cropped and the text or image may be overlaid on a margin area that is not a main area according to another embodiment as described above. Accordingly, it is possible to overlay text or other images after detecting a blank area which is not a main area (S304) by various methods described later.

If the text is to be overlaid on the margin area (S305), it is first determined whether the detected margin area is sufficient to overlay the text (S306). The determination method may calculate the size of the entire text area in consideration of the minimum font size of the text to be overlaid, and compare the size of the entire text area with the size of the detected blank area to determine whether the overlay is possible.

If it is determined that the size of the margin area is not sufficient to overlay the text, the margin area is added to the image in which the original image or the main area is cropped according to the new embodiment of the present invention (S307). Then, the text is overlaid on the added margin area (S308).

As a result of the determination, if the size of the margin area is sufficient to overlay the text, the text is overlaid on the detected margin area (S308).

Meanwhile, according to another embodiment of the present invention, when overlaying at least one original image different from the original image (S309), the plurality of images are overlaid on each other. At this time, it is preferable to detect a main area from each original image, and overlay the main areas of the original images so that they do not overlap each other.

For example, when it is desired to overlay the first image and the second image in accordance with the layout, the main area and the blank area of the first image and the second image are respectively detected, and the main area of the second image is cropped, It is possible to overlay the margin area. By doing so, the main area of the first image and the main area of the second image do not overlap with each other, so that the main object on each image can be displayed without missing. As another method, an edited image can be generated by cropping the main area of the first image and the main area of the second image, respectively, and rearranging each cropped image according to the layout.

Hereinafter, a method for detecting a main area and a blank area from an original image will be described in detail with reference to FIGS.

&Lt; Detection of main area >

4 is a flowchart showing the detailed procedure of the main area detection according to the embodiment of the present invention. Referring to FIG. 4, an area including the identified object can be detected as a main area by identifying an object included in the original image to be edited. At this time, the object may be a face, a text, and other main objects.

For example, it is possible to detect whether there is a face included in the image first (S401). If the detected face exists as a result of the detection (S402), the region including the detected face may be determined as a main region (S403). At this time, a method of detecting a face from an image can be implemented by various known image processing methods, and a detailed description thereof will be omitted. On the other hand, when a plurality of faces are detected as a result of the face detection, the importance degree of each face (for example, importance determined based on size, position, sharpness, etc.) is discriminated and regions including faces with high importance It may be judged as a main area, or one area including all of a plurality of detected faces may be judged as a main area.

In addition, it is possible to determine the main area by analyzing whether text is included in the image (S404). At this time, it is also possible to detect the text using the OCR (Optical Character Reader) technique. As described above, when the text is detected in the image (S405), the detected text area can be determined as the main area (S406).

In addition, it is possible to detect whether there is at least one predetermined specific object according to the embodiment of the present invention (S407). If the predetermined object exists in the image as a result of the detection (S408), the region containing the detected object may be determined as the main region (S409).

On the other hand, in the above-described object detection method, each method of detecting face, text, and other main objects can be selectively applied, and the present invention is not limited to the detection order shown in FIG.

FIG. 5 is a flowchart illustrating a main area detection procedure by image pattern analysis according to an embodiment of the present invention. Referring to FIG. 5, unlike the method described in FIG. 4, a main region can be detected by analyzing a pattern of an entire image without detecting a specific object.

In other words, if the focused portion and the non-focused portion coexist in the image, the region having a relatively high average value of the sharpness can be determined as the main region through the sharpness analysis. At this time, in the sharpness analysis method, it can be judged that the shorter the period of change of the surrounding pixel value with respect to the specific pixel in the image, the higher the sharpness.

Alternatively, for each pixel in the image, a difference value of color or contrast relative to surrounding pixels may be obtained, and an area having a relatively high average value of the differences may be determined as a main area.

As another method, it is also possible to determine the main area by combining the focus value and the contrast value. That is, the focus value and the contrast value of each pixel may be calculated (S501, S502), and the focus value and the contrast value may be calculated to determine a saliency value (S503). For example, the apparentness value can be calculated by multiplying the focus value and the contrast value.

When the conspicuousness value for each pixel is calculated, the distribution of the conspicuousness values in the image is modeled as a two-dimensional Gaussian distribution (S504) It is judged as a main area (S505). At this time, the main region may be detected as one or more than two according to the distribution of the saliency values in the image.

6B, the extracted main region may have a shape 620 of a closed curve (e.g., an ellipse) that basically represents the size, shape, direction, etc. of the main region And may be finally determined as a figure 610, such as a rectangle, including the closed curve (s) as shown in FIG. 6A, as needed.

<Margin Area Detection Method>

7 is a flowchart illustrating a procedure for detecting a non-salient area according to an embodiment of the present invention. In the meantime, the blank area in the present invention means an area having relatively little influence in the image, and has a meaning opposite to that of the main area. Therefore, the remaining area excluding the main area determined as described above may be determined as the blank area, or the blank area may be determined by another method. Meanwhile, depending on the type of the main area detected in the above, some of the main area and the blank area may overlap with each other.

Referring to FIG. 7 as an embodiment for extracting a blank area, first, a main area of an image is detected (S701) as described above. Then, as shown in FIG. 8A, the remaining area excluding the main area 810 is designated as a first-step blank area 820, which is a candidate area of the blank area (S702).

Then, image segmentation is performed on the designated one-step margin area (S703). In this case, the screen division is a task of grouping adjacent pixels having similar pixel values (e.g., focus value, contrast value, and conspicuous value), and finally dividing one screen into several areas having similar pixel values. For example, the color or brightness of each pixel in the first-stage blank region is compared with neighboring pixels, and if the difference is less than a preset reference, it is regarded as similar pixels and grouped into one region. Then, an area having a change rate of a pixel value lower than a preset first threshold value and having a size or shape higher than a preset second threshold value among the divided areas is referred to as a 2-step margin Area 830 (S704).

In the meantime, the first-stage blank region means a region that is cut or overlapped with other images and may be masked. The second-stage blank region means a region having a flat pixel value and a small screen change rate even in the first-step blank region. Each of the thus determined blank areas may be applied differently in the subsequent overlay editing process (S705). For example, in the embodiments of the present invention described later, the first-stage blank area may be used when overlaying a plurality of images, and the second-stage blank area may be used when overlaying text on an image.

In the foregoing, a method of detecting a main area and a blank area according to an embodiment of the present invention has been described in detail with reference to FIGS. Next, various image editing methods according to the embodiments of the present invention will be described in detail with reference to FIG. 9 to FIG.

&Lt; First Embodiment-Main Region Crop >

9 is a flowchart illustrating a procedure for cropping an original image according to the first embodiment of the present invention. Referring to FIG. 9, an image can be edited by cropping a main area of an original image according to the first embodiment of the present invention.

That is, when the main area of the original image is detected (S901) as described above, in consideration of the layout of the image on the web page, the characteristic value of the area in which the edited image (for example, thumbnail image) Image size, aspect ratio, etc.) (S902). Then, the image is edited by cropping and resizing the detected main area according to the input characteristic value (S903). In another embodiment, a web page may be configured by calculating an optimum crop ratio and size for a main region of an image for a plurality of contents, and suggesting a content layout most suitable for a combination of the images.

Meanwhile, according to an exemplary embodiment of the present invention, the determined main region may be cropped and then resized according to a corresponding size or aspect ratio. Alternatively, when the main region is cropped, it may be cropped considering the size or aspect ratio .

10 is a view showing an image cropped from an original image according to the first embodiment of the present invention. Referring to FIG. 10, a crop image can be placed on a web page or the like by cropping the main area 1000 as described above in the original image. For example, images can be automatically and automatically edited by cropping them around the main area for use on relatively small-sized screens, such as smart phones, or by cropping them for a specific layout.

&Lt; Second Embodiment - Image + Text Overlay >

11 is a flowchart illustrating a procedure for overlaying text in a margin area according to a second embodiment of the present invention. Referring to FIG. 11, a main area is detected (S1101) as described above in the original image, and a remaining area excluding the detected main area is set as a blank area (S1102).

Then, the size of the blank area is calculated to determine the length of the overlayable text string (S1103), and the text (e.g., an article title, a part of the content, etc.) corresponding to the area is arranged and automatically overlaid (S1106). At this time, according to the embodiment of the present invention, the color of the blank area overlaid with the text is analyzed (S1104), and the color of the text is determined in consideration of the color of the analyzed blank area (S1105). For example, it is preferable that the color of the marginal area and the color of the complementary color are determined as the color of the text, so that the text can be seen clearly according to the color of the margin. Also, when the color of the marginal area is a faded color, the color of the text can be set to the font of the opposite color, or the text background can be boxed in a separate color.

12, the main area 1200 and the blank area 1210 are detected in the original image shown on the left side, and the size and color of the blank area 1210 are analyzed. That is, the size of the text area to be arranged may be determined by analyzing the size of the blank area 1210, and the color of the text to be arranged may be determined by analyzing the color of the blank area 1210.

Thus, the headline 1220 of the text string &quot; fresh spring interior know-how &quot; at the top of the margin area 1210 can be arranged in a color opposite to the color of the margin area 1210, as in the edited image shown on the right And a text string in the form of a text box 1230 at the bottom. The text box 1230 can increase the readability of the text by setting the color of the text in the box and the background color of the text box 1230 to be opposite to each other.

&Lt; Third Embodiment - Adding a margin area to an image >

13 is a diagram showing an image in which a margin area is added to an original image according to the third embodiment of the present invention. Referring to FIG. 13, when a separate margin area is not detected in the original image, the background image complexity outside the main area detected in the original image is discriminated. When the image complexity is low, And extracts a blank area 1310 of a color (e.g., similar color, complementary color, the same color, a color of a main area, etc.) matching the extracted basic color with the original image 1300 according to an embodiment of the present invention, . &Lt; / RTI &gt;

Thus, an additional title / caption area that can overlay the text in a simple background area outside the main area can be created as a blank area. Accordingly, a new image may be created in which the margin area 1310 is added to the original image 1300, or a new caption area may be created using HTML / CSS or the like outside the image.

&Lt; Fourth Embodiment - Image + Image Overlay >

14 is a diagram showing an image in which a plurality of images are overlaid according to a fourth embodiment of the present invention. A plurality of images can be overlaid and automatically edited by comparing the size and position of the main area and the blank area of each image according to the fourth embodiment of the present invention when a plurality of images exist in a content to be placed on a web page or the like .

For example, referring to FIG. 14, the main region 1411 is discriminated from the first image 1410 at the upper end and the main region 1421 is discriminated from the second image 1420 at the lower end by using the above-described method .

The main image 1420 of the second image 1420 may then be overlaid on the margin region of the first image 1410 or the first image 1410 may be overlaid on the margin region of the second image 1420. [ Can overlay the main area 1411 of FIG. This makes it possible for the main regions of a plurality of images to automatically overlay images without overlapping each other. 14 illustrates a method of overlaying two images, it is also possible to overlay three or more images by the same method.

The various operations and functions as described herein with respect to the various methods may be performed by any of a number of types of functions, such as a particular processing function and / or a processing function implemented therein, and / &Lt; / RTI &gt; For example, such functions may be described herein, as well as performing various operations and processes as described herein, or any other operations and functions as described herein, etc., or the like, And may perform such operations, processes, and the like.

In some embodiments, such functions (which may be implemented on the same device or on distinct devices) may be implemented in such a manner and in accordance with various aspects of the present invention, etc., and / Operations and functions, and their equivalents. In some embodiments, such processing is performed together by the first functional portion in the first device and the second functional portion in the second device. In other embodiments, such processing, operations, etc. are performed entirely by the processing units within one particular apparatus. Even in other embodiments, such processing and operations are performed using at least a first functional portion and a second functional portion within a single device.

Also, in the foregoing description, the terms &quot; system, &quot; &quot; device, &quot; and &quot; section &quot; may be a single processing device or a plurality of processing devices. Such a processing device may be a microprocessor, a microcontroller, a digital signal processor, a microcomputer, a central processing unit, a field programmable gate array, a programmable logic device, a state machine, a logic circuit, an analog circuit, a digital circuit, and / (Analog and / or digital) based on hard coding of operational instructions. The processing module, module, processing circuit, and / or processing unit may be implemented as a single memory device, a plurality of memory devices, and / or a combination of the processing modules, the processing circuitry, and / Lt; / RTI &gt; memory and / or an integrated memory element. Such a memory device may be any device that stores read only memory (ROM), random access memory (RAM), volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and / . If the processing module, the module, the processing circuit, and / or the processing unit include one or more processing devices, the processing devices may be centrally located (i.e., directly or indirectly via a wired and / Connected), distributed (e.g., cloud computing via an indirect connection over a local and / or wide area network). If the processing module, module, processing circuit, and / or processing unit implements one or more of those functions through a static machine, analog circuitry, digital circuitry, and / or circuitry, memory and / It should further be noted that the corresponding operating instructions may be internal or external to the circuit including the static machine, analog circuitry, digital circuitry, and / or logic circuitry. The memory element may be stored and the processing module, module, processing circuitry, and / or processing unit may be configured to store a hard-coded and / or hard-coded data item corresponding to at least a portion of the steps and / or functions depicted in one or more of the figures It should be further noted that it carries out operational instructions. Such a memory device or memory element may be included within the components of the manufacture.

The invention has been described above with the aim of method steps illustrating the performance of certain functions and their relationships. The boundaries and order of these functional components and method steps have been arbitrarily defined herein for convenience of description. Alternative boundaries and sequences may be defined as long as the specific functions and relationships are properly performed. Any such alternative boundaries and sequences are therefore within the scope and spirit of the claimed invention. In addition, the boundaries of these functional components have been arbitrarily defined for ease of illustration. Alternative boundaries can be defined as long as certain important functions are properly performed. Likewise, the flow diagram blocks may also be arbitrarily defined herein to represent any significant functionality. For extended use, the flowchart block boundaries and order may have been defined and still perform some important function. Alternative definitions of both functional components and flowchart blocks and sequences are therefore within the scope and spirit of the claimed invention. Those skilled in the art will also appreciate that the functional components and other illustrated blocks, modules, and components herein may be implemented as illustrated or as separate components, such as semiconductor integrated circuits (ASICs) And the like, or any combination thereof.

The invention may also be described, at least in part, in the language of one or more embodiments. Embodiments of the invention are used herein to describe the invention, aspects thereof, features thereof, concepts thereof, and / or examples thereof. The physical embodiment of an apparatus, article of manufacture, machine, and / or process for implementing the invention may include one or more aspects, features, concepts, examples, etc., described with reference to one or more embodiments described herein . Moreover, in the entire drawings, embodiments may incorporate the same or similarly named functions, steps, modules, etc. that may use the same or different reference numerals, and so forth, Steps, modules, etc., may be the same or similar functions, steps, modules, etc., or the like.

In the meantime, the term " processing unit "is used in the description of various embodiments of the present invention. A module includes a functional block implemented via hardware to perform one or more functions, such as processing one or more input signals to produce one or more output signals. The hardware implementing the functional block may operate directly by combining software and / or firmware. As used herein, a module may include one or more sub-modules that are themselves modules.

While various combinations of features and specific combinations of features of the present invention are explicitly described herein, other combinations of these features and functions are likewise possible. The present invention is not limited to the specific examples disclosed herein, and explicitly incorporates these different combinations.

As described above, the present invention has been described with reference to particular embodiments, such as specific elements, and specific embodiments and drawings. However, it should be understood that the present invention is not limited to the above- And various modifications and changes may be made thereto by those skilled in the art to which the present invention pertains.

Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

210: content information database 220: image analysis module
221: Object Detection Module 222: Pattern Analysis Unit
223: margin detection unit 230: image editing module
231: image cropping section 232: text overlay section
233: image overlay unit 234: margin adding unit
235: image resizing part
240: image configuration information database
250: Edit image database

Claims (27)

A content information database for storing at least one edit target image;
An image analysis module that receives the at least one image to be edited from the content information database and detects at least one main area of the entire area of the provided image to be edited;
An image configuration information database storing configuration information of a layout in which an edited image is to be placed; And
And an image editing module for automatically generating an edit image including a main area detected by the image analysis module according to the configuration information stored in the image configuration information database,
Wherein the image analysis module includes a margin detection unit that determines an area excluding the detected main area as a margin area among the entire area of the editing object image,
Wherein the blank space detecting unit sets a region excluding the detected main region as a first-step blank region, and divides the region into a plurality of regions having similar pixel values with respect to the first-step blank region, An area having a rate of change of a pixel value lower than the first threshold value and having a size or shape higher than a preset second threshold value is set as a two-step margin area.
The image analysis module of claim 1,
And an object detection unit that detects at least one object among all the regions of the image to be edited and determines an area including the detected object as a main area.
The image analysis module of claim 1,
Calculating a focus value and a contrast value for each pixel in the image to be edited, calculating a saliency value based on the calculated focus value and the contrast value, and calculating at least one main region And a pattern analyzing unit that detects a pattern of the image.
delete delete The apparatus of claim 1, wherein the first-stage blank region and the second-stage blank region are applied differently in an overlay editing process.
The image editing apparatus according to claim 1,
And an image cropping unit for cropping the main region detected by the image analysis module from the image to be edited.
The image editing apparatus according to claim 1,
And a text overlay section for overlaying the text in the margin area.
The automatic image editing apparatus according to claim 8, wherein the color of the text overlaid on the blank area is set in consideration of the color of the blank area.
9. The method of claim 8, wherein the content of the text overlaid on the margin area
Wherein the editing unit is any one of a caption, a title, and an article content of a content related to the editing target image.
The image editing apparatus according to claim 1,
And an image overlay unit for overlaying a main area of the second image in a blank area of the first image for a plurality of editing target images for which a main area is detected.
The image editing apparatus according to claim 1,
And a margin adding unit for adding a margin area to the editing target image when the area other than the main area is not sufficient to overlay text in the entire area.
The image editing apparatus according to claim 1,
And an image resizing unit for resizing the size or aspect ratio of the edited image including the main area according to the configuration information.
Receiving an image to be edited in an image analysis module;
Detecting at least one main area of the entire area of the provided editing target image in the image analysis module;
Determining an area excluding the detected main area as the blank area among the entire area of the image to be edited; And
Generating an edited image by automatically editing an image according to a configuration information of a predetermined layout so that the detected main area is included in the image editing module,
Setting an area excluding the detected main area as a first-step margin area;
Dividing the image into a plurality of regions having similar pixel values for the first-stage blank region; And
Setting an area having a change rate of a pixel value lower than a preset first threshold value among the divided areas and having a size or shape higher than a preset second threshold value as a two-step margin area, Automatic image editing by analysis.
15. The method of claim 14,
Detecting at least one object among all the regions of the image to be edited and determining the region including the detected object as a main region.
15. The method of claim 14,
Calculating a focus value and a contrast value for each pixel in the image to be edited;
Calculating a saliency value based on the calculated focus value and the contrast value; And
And detecting at least one main region according to the calculated distribution of the saliency value.
delete delete 15. The method of claim 14, wherein the one-step margin region and the two-step margin region are applied differently in the overlay editing process.
15. The method of claim 14, wherein generating the edit image comprises:
And cropping the detected main region from the image to be edited.
15. The method of claim 14, wherein generating the edit image comprises:
And overlaying the text in the margin area.
The method according to claim 21, wherein the color of the text overlaid on the blank area is set in consideration of the color of the blank area.
22. The method of claim 21 wherein the content of the text overlaid on the margin area is
Wherein the image editing method is any one of caption, title, and content of a content related to the image to be edited.
15. The method of claim 14, wherein generating the edit image comprises:
Overlaying a main region of a second image in a margin region of a first image for a plurality of editing target images for which a main region is detected.
15. The method of claim 14, wherein generating the edit image comprises:
And adding a margin area to the editing target image when the area other than the main area is not sufficient to overlay the text in the entire area.
15. The method of claim 14, wherein generating the edit image comprises:
And resizing the size or aspect ratio of the edited image including the main area according to the configuration information.
A computer-readable recording medium having recorded thereon a program for executing the method according to any one of claims 14 to 16 and claims 19 to 26.

Images