KR20140023596A - Apparatus, method and computer readable recording medium for editting the image automatically by analyzing an image - Google Patents

Abstract

The present invention relates to an image automatic editing apparatus by image analysis for generating an image appropriate for various screen configuration like a thumbnail image by editing a target image to edit automatically, especially including: a content information database for storing at least one editing target image; an image analysis module which is provided with the at least one editing target image from the content information database, and detects at least one major area of the whole area of the provided editing target image; an image configuration information database for storing configuration information of a layout where the edited image is to be arranged; and an image editing module for generating the edited image including the major area detected by the image analysis module automatically according to the configuration information stored in the image configuration information database. [Reference numerals] (210) Content information D/B; (220) Image analysis module; (221) Object detection part; (222) Pattern analysis part; (223) Blank detection part; (230) Image editing module; (231) Image crop part; (232) Text overlay part; (233) Image overlay part; (234) Blank adding part; (235) Image resizing part; (240) Image configuration information D/B; (250) Edited image D/B

Description

FIELD, METHOD AND COMPUTER READABLE RECORDING MEDIUM FOR EDITTING THE IMAGE AUTOMATICALLY BY ANALYZING AN IMAGE}

The present invention relates to an apparatus for automatically editing an image, and more particularly, an image by analyzing an image for generating an image suitable for various screen configurations such as a thumbnail image by automatically editing a target image to be edited. An automatic editing apparatus, method, and computer readable recording medium.

Today, the widespread use of the Internet has led to the rapid development of wireless mobile communication technology beyond the fixed line, and in the real world, information retrieval on the Internet through portable terminals such as mobile phones, PDAs, and hand- It became possible regardless of time and place.

There is a demand to consume content in various layouts, such as magazines and e-books, in the digital medium of the web or application. For example, tablet applications, such as flipboards, support the consumption of existing web content by laying out images.

The photo images included in the web content play an important role in the layout, and to crop the part of the original photo image or to overlay text or other pictures on the photo to expose the original image in the form of thumbnail images. You will create various layouts.

On the other hand, when producing such a thumbnail image, if the aspect ratio of the original image and the aspect ratio of the thumbnail are not the same, the thumbnail image is manually generated to include the main area of the photo in the thumbnail image, or the editing designer Created the layout by directly processing and editing the image. Even in the case of automation without manual production, the original photo image was simply resized and applied to the content layout.

This manual editing process for creating thumbnail images is not only resource intensive, but can also be applied immediately when thumbnail images of different shapes, sizes and aspect ratios (ie aspect ratios) are required on various display screens. There is a problem that the image must be re-edited again.

On the other hand, as a technique for automatically generating thumbnails and the like from such an original image, Korean Patent Publication No. 2004-0044373 "Video thumbnail and its generation method (Hewlett Packard)" (Document 1) is a relatively short, low resolution from the video A method of generating an animated video thumbnail is disclosed.

However, if the image is automatically resized as described above, the image may not fit the ratio of the specific content layout in which the image is to be placed, or the main area of the image may be resized to reduce readability on a small screen such as a smartphone. Can be.

In particular, with the advent of smart phones, tablets, and smart TVs, various sizes of screens have emerged, and thus there is a need for a technology for automatically generating a dynamic layout suitable for various screen sizes.

[Document 1] Korean Patent Laid-Open Publication No. 2004-0044373 Video thumbnail and its creation method (Hewlett Packard) 2004.05.28

An object of the present invention is to automatically edit the image by analyzing the image to be edited to divide the main area and the margin area, and to automatically create an image suitable for various layout configuration by automatically editing the image around the main area An apparatus, method, and computer readable recording medium are provided.

In addition, another object of the present invention is to analyze the image to be edited by dividing the main area and the margin area, by cropping the main area to automatically edit the image by image analysis that can automatically generate an image suitable for various layout configurations An automatic image editing apparatus, method, and computer-readable recording medium are provided.

In addition, another object of the present invention is to analyze the image to be edited to divide the main area and the margin area, and overlay (overlapping) the text in the margin area to automatically generate an image suitable for various layout configurations An automatic image editing apparatus, method, and computer readable recording medium by image analysis are provided.

In addition, another object of the present invention is to analyze a plurality of editing target image to be divided into a main area and a margin area, cropping the main area of each editing target image and then overlaying a plurality of cropped images suitable for various layout configurations Disclosed are an apparatus and method for automatically editing an image by image analysis capable of automatically generating an image, and a computer-readable recording medium.

In addition, another object of the present invention is to automatically edit the image by image analysis that can automatically create an image suitable for various layout configurations by adding a separate margin area to the editing target image, and overlaying text on the added margin area An apparatus, method, and computer readable recording medium are provided.

In order to achieve the above-described object of the present invention and to achieve the specific effects of the present invention described below, the characteristic structure of the present invention is as follows.

According to an aspect of the present invention, the automatic image editing apparatus by image analysis, the content information database for storing at least one image to be edited; An image analysis module configured to receive the at least one image to be edited from the content information database, and detect at least one main region among all areas of the provided image to be edited; An image configuration information database that stores configuration information of a layout in which the edited image is to be placed; And an image editing module for automatically generating an edited image including a main region detected by the image analyzing module according to the configuration information stored in the image configuration information database.

Preferably, the image analysis module includes an object detector that detects at least one object of the entire region of the editing target image, and determines the region including the detected object as the main region.

Preferably, the image analysis module calculates a focus value and a contrast value for each pixel in the editing target image, calculates a salient value based on the calculated focus value and the contrast value, and calculates the calculated salience It includes a pattern analysis unit for detecting at least one main region according to the distribution of the value.

Preferably, the image analysis module includes a margin detection unit that determines a region except for the detected main region among all regions of the editing target image as a margin region.

Preferably, the margin detection unit sets an area excluding the detected main area as a first step margin area, and divides the screen into a plurality of areas having similar pixel values with respect to the first step margin area. A region having a change rate of a pixel value lower than a preset first threshold value and having a size or shape higher than the preset second threshold value is set as a second step margin area.

Preferably, the first step margin area and the second step margin area are differently applied to each other in the overlay editing process.

Preferably, the image editing module includes an image cropping unit for cropping the main region detected by the image analyzing module from the editing target image.

Preferably, the image editing module includes a text overlay for overlaying text on the margin area.

Preferably, the color of the text overlaid on the margin area is set in consideration of the color of the margin area.

Preferably, the content of the text to be overlaid on the margin area is any one selected from caption, title, and article content of the content related to the editing target image.

Preferably, the image editing module includes an image overlay unit for overlaying the main area of the second image on the margin area of the first image with respect to the plurality of editing target images from which the main area is detected.

Preferably, the image editing module includes a margin adding unit for further adding a margin area to the editing target image when there is not enough space to overlay text in an area other than the main area of the entire area.

Preferably, the image editing module includes an image resizing unit for resizing the size or aspect ratio of the edited image including the main area according to the configuration information.

According to another aspect of the present invention, the image automatic editing method by image analysis, the step of receiving an image to be edited in the image analysis module; Detecting at least one major area of the entire area of the provided editing object image in the image analysis module; And generating an edited image by automatically editing the image according to configuration information of a preset layout by including the detected main region in the image editing module.

Preferably, the detecting of the main area includes detecting at least one object from all areas of the editing target image, and determining an area including the detected object as the main area.

Preferably, the detecting of the main area comprises: calculating a focus value and a contrast value for each pixel in the editing target image; Calculating a salient value based on the calculated focus value and contrast value; And detecting at least one main region according to the calculated salient value distribution.

Preferably, after the detecting of the main region, the method may further include determining a region excluding the detected main region of the entire region of the editing target image as a margin region.

Preferably, the determining of the margin area comprises: setting an area excluding the detected main area as a first step margin area; Splitting the screen into a plurality of areas having similar pixel values with respect to the first blank area; And setting a region having a size or shape higher than a preset second threshold while having a rate of change of a pixel value lower than a preset first threshold value among the divided regions as a second step margin area.

Preferably, the first step margin area and the second step margin area are differently applied to each other in the overlay editing process.

Preferably, generating the edited image includes cropping the detected main region from the edited target image.

Advantageously, generating said edited image comprises overlaying text in said margin area.

Preferably, the color of the text overlaid on the margin area is set in consideration of the color of the margin area.

Preferably, the content of the text to be overlaid on the margin area is any one selected from caption, title, and article content of the content related to the editing target image.

Preferably, the generating of the edited image includes overlaying the main region of the second image on the margin region of the first image with respect to the plurality of edit target images in which the main region is detected.

Preferably, the generating of the edited image may further include adding a margin area to the edited image when there is not enough space to overlay text in the non-main area of the entire area. .

Preferably, generating the edited image includes resizing the size or aspect ratio of the edited image including the main area according to the configuration information.

Meanwhile, the information for performing the automatic image editing procedure by the image analysis may be stored in a recording medium readable by a server computer. Such a recording medium includes all kinds of recording media in which programs and data are stored so that they can be read by a computer system. Examples include ROMs (Read Only Memory), Random Access Memory, CD (Compact Disk), DVD (Digital Video Disk) -ROM, magnetic tape, floppy disk, optical data storage device, (For example, transmission over the Internet). Such a recording medium may also be distributed over a networked computer system so that computer readable code in a distributed manner can be stored and executed.

As described above, according to the present invention, there is an advantage that the disadvantages of manually editing the image can be eliminated by automatically editing the target image to be edited according to a specific content layout instead of manually.

In addition, when thumbnail images having various shapes, sizes, and aspect ratios are required, the present invention has the advantage of automatically generating dynamic layouts for various screen sizes by automatically editing thumbnail images according to the corresponding conditions. .

Further, according to the present invention, by automatically determining and editing the main region of the photographic image, there is an advantage in that an image having excellent readability can be automatically generated even after resizing without missing a main portion.

Furthermore, according to the present invention, an effective thumbnail image can be automatically generated by automatically analyzing a non-essential area in the picture and overlaying captions, titles, and article contents of the picture or related content on non-essential areas. There is an advantage.

In addition, according to the present invention, it is possible to automatically determine a main area of a plurality of editing target images, and crop a plurality of main areas to overlay each other, thereby automatically generating various layouts such as dynamic magazines. .

1 is a view showing the concept of automatic image editing by image analysis according to the present invention.
2 is a block diagram illustrating a detailed configuration of an apparatus for automatically editing an image by image analysis according to an exemplary embodiment of the present invention.
3 is a flowchart illustrating an automatic image editing procedure by image analysis according to an exemplary embodiment of the present invention.
4 is a flowchart illustrating a detailed procedure of main region detection according to an embodiment of the present invention.
5 is a flowchart illustrating a main region detection procedure by image pattern analysis according to an exemplary embodiment of the present invention.
6A and 6B illustrate an image in which a detected main region is displayed according to an embodiment of the present invention.
7 is a flowchart illustrating a procedure of detecting a margin area according to an embodiment of the present invention.
8A illustrates an image in which a first step margin area is displayed according to an embodiment of the present invention.
8B is a diagram illustrating an image in which a two-step margin area is displayed according to an embodiment of the present invention.
9 is a flowchart illustrating a procedure of cropping an original image according to the first embodiment of the present invention.
10 is a diagram illustrating an image cropped from an original image according to the first embodiment of the present invention.
11 is a flowchart illustrating a procedure of overlaying text on a margin area according to a second embodiment of the present invention.
12 is a diagram illustrating an image in which text is overlaid on a margin area according to a second embodiment of the present invention.
FIG. 13 is a diagram illustrating an image in which a margin area is added to an original image according to a third embodiment of the present invention. FIG.
14 is a view illustrating an image in which a plurality of images are overlaid according to a fourth embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

The present invention discloses an apparatus and method for automatically editing an image by image analysis capable of automatically detecting a position of a main part in an image such as a photograph and automatically editing the image based on the detected main part according to a desired layout. .

For example, after automatically detecting the position of the main part of the picture according to an embodiment of the present invention, crop the detected main part and generate a thumbnail image by adjusting the aspect ratio to the layout to be produced. You can automatically create an image of the layout you want without missing parts.

In addition, according to an embodiment of the present invention, the non-essential areas (eg, margin areas) are automatically analyzed in an image such as a photograph, and caption, title, article content, etc. of content related to the image are automatically analyzed. By overlaying the non-essential areas on the layout or overlaying with another image, various types of layouts such as dynamic magazines can be automatically generated.

Meanwhile, in the present invention described below, an editing target image (ie, an image before editing) for generating a desired image (for example, a thumbnail image) by automatic analysis and editing is called an original image, and the original image is a photograph. Not the broadest concept, including drawings, cartoons, and so on.

In addition, 'crop' used in the present invention to be described later means cropping a selected area (eg, a main area) from all areas of a specific image, and 'overlay' overlays a plurality of images. To edit.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

First, the concept and structure of an apparatus according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2, and the procedure according to an embodiment of the present invention will be described in detail with reference to FIG. 3.

Image auto-editing concept

1 is a view showing the concept of automatic image editing by image analysis according to the present invention. Referring to FIG. 1, an original image (ie, an edit target image) for editing an image to be configured on a web page is stored in a content information (ie, article) database. In the content information database, not only the original image but also a caption, title, article content, etc. of content related to the original image may be linked and stored together.

First, when an image of content to be analyzed stored in the content information database is fed, image analysis is performed in various ways. In this case, the main image is analyzed according to an embodiment of the present invention to detect a main area including a face or an object and a blank area other than the main area.

For example, an object included in the original image is detected by various methods, and an area including the detected object is set as a main area. In this case, the remaining area except for the main area of the entire image may be set as a margin area. On the other hand, when the entire image area is divided into the main area and the margin area, the information such as the location, color, brightness, and optimal aspect ratio of the main area and the location, color, brightness, and optimal width / length of the margin area Information such as ratio may be generated. In this way, the generated information can be referred to later when editing the image.

In addition, the pattern analysis method may be applied as another embodiment for distinguishing the main region and the margin region. For example, a saliency value may be calculated based on a focus value and a contrast value of each pixel in the image, and a main region may be detected according to the calculated distribution of the saliency values.

On the other hand, as a method for detecting the margin area, after detecting the main area as described above, the remaining area may be set as the margin area. As in the method described below, the margin area is set in one step and two steps according to the embodiment of the present invention. You can also implement different editing methods by separating them with spaces.

As such, when the main area and the blank area are divided, the image may be edited according to a layout to be applied to a web page or the like according to various embodiments. Accordingly, the analyzed image is edited by referring to the information of the layout (for example, the size, aspect ratio, resolution, shape, shape, orientation, etc.) of the image to be inserted in the web page for image editing.

For example, according to the first exemplary embodiment of the present invention, an image (for example, a thumbnail image) to be applied to a web page may be produced by cropping the main area. In this case, the cropped image may be further resized to generate an image having a desired size and ratio.

In addition, according to the second embodiment of the present invention, text may be overlaid on the margin area of the analyzed image. In this case, the text may be a caption, title, article content, etc. of the content associated with the original image stored in the content information database linked to the original image, but the present invention is not limited thereto.

In addition, according to the third embodiment of the present invention, if the margin area of the analyzed image is not enough to overlay the text, the margin area may be automatically generated and added. Therefore, an edited image can be generated by overlaying the added text area or the same text or other text as in the second embodiment.

Meanwhile, in the second and third embodiments, as in the first embodiment, an image having a desired size and ratio suitable for a layout may be generated by resizing an image overlaid with text.

In addition, according to the fourth embodiment of the present invention, an edited image may be generated by overlaying a plurality of original images with each other. For example, if two original images are to be overlaid with each other, according to an embodiment of the present invention, the main region of each original image is detected, and the desired edited image is overlaid with each other without overlapping the detected major regions of each original image. Can be generated.

As such, the edited image produced according to each embodiment is delivered to a web server, and the web server arranges the edited image according to a layout to form a web page. That is, according to the screen size of the user terminal (for example, phone, tablet, notebook, PO, etc.) to configure a web page to provide it to the user terminal.

Hereinafter, an apparatus according to an embodiment of the present invention will be described with reference to FIG. 2, and a method according to an embodiment of the present invention will be described with reference to FIG. 3.

<Automatic image editing device by image analysis>

2 is a block diagram illustrating a detailed configuration of an apparatus for automatically editing an image by image analysis according to an exemplary embodiment of the present invention. Referring to FIG. 2, the automatic image editing apparatus according to the embodiment of the present invention may include an image analysis module 220 and an image editing module 230. In addition, the image analysis module 220 may include an object detector 220, a pattern analyzer 220, and a margin detector 223, and the image editing module 230 may include an image crop unit 231. , A text overlay unit 232, an image overlay unit 233, a margin adding unit 234, an image resizing unit 235, and the like.

In addition, the automatic image editing apparatus may include at least one database, such as a content information database 210, an image configuration information database 240, and an edited image database 250.

The content information database 210 stores at least one original image (ie, an editing target image) for editing, and according to an embodiment of the present invention, captions, titles, article contents, etc. of contents related to each original image are stored. It may be mapped and stored together.

The image analysis module 220 receives a source image to be edited from the content information database 210 and performs a function of detecting at least one major area of the entire image area through image analysis. In this case, according to an exemplary embodiment of the present invention, the main region may be selectively detected by the object detector 221, and the main region may be detected by the pattern analyzer 222. In addition, according to an exemplary embodiment of the present invention, the margin detector 223 may detect at least one margin region of the entire image region.

For example, when the image analysis module 220 receives an image of content to be analyzed stored in the content information database 210, image analysis is performed by various methods described below. That is, the main image is analyzed according to an exemplary embodiment of the present invention to detect a main area including a face or an object and a blank area other than the main area.

In more detail, the object detector 221 detects at least one object (eg, face, text, object, etc.) included in the original image by various methods, and detects an area including the detected object. Set to the main area. In this case, the remaining area except the area set as the main area of the entire image may be automatically set as the margin area.

On the other hand, when the entire image area is divided into the main area and the margin area, the information such as the location, color, brightness, and optimal aspect ratio of the main area and the location, color, brightness, and optimal width / length of the margin area Information such as ratio may be generated. In this way, the generated information can be referred to later when editing the image.

In addition, the main area detection by the pattern analyzer 222 may be performed as another method for detecting the main area. For example, a saliency value may be calculated based on a focus value and a contrast value of each pixel in the image, and a main region may be detected according to the calculated distribution of the saliency values. Detailed description thereof will be described later with reference to FIG. 5.

On the other hand, after detecting the main area by the object detector 221 or the pattern analyzer 222 as described above, the remaining area can be set as the margin area, according to another embodiment of the present invention by the margin detector 223 You can also detect the margins further. For example, the margin detection unit 223 may divide the margin area into a first step margin and a second step margin according to an embodiment of the present invention, and implement the first and second step margins to be edited by different editing methods. .

As such, when the main area and the margin area are separated in the image analysis module 220, the image editing module 230 may display the original image in which the area is divided according to various embodiments (eg, according to four embodiments). Edit the image to match the layout of the web page. That is, the image editing module 230 includes layout information (for example, image size, aspect ratio, resolution, shape, shape, shape, orientation, etc.) to be configured in a web page stored in the image configuration information database 240. Reference to) to edit the image analyzed by the image analysis module 220.

For example, the image cropping unit 231 crops main areas separated by the image analyzing module 220 according to the first embodiment of the present invention to produce an image (eg, a thumbnail image) to be applied to a web page. In this case, the cropped image may be resized through the image resizing unit 235 to generate an image having a desired size and ratio.

In addition, the text overlay unit 232 edits the original image by overlaying text on the margin area of the image separated by the image analysis module 220 according to the second embodiment of the present invention. In this case, the text may be a caption, title, article content, etc. of content related to the original image stored and mapped with the original image in the content information database 210 as described above, but the present invention is not limited thereto. .

In addition, when the margin area of the analyzed image is not enough to overlay text according to the third embodiment of the present invention, the margin adding unit 234 may automatically generate and add a margin area. Accordingly, the edited image may be generated by overlaying the text as in the second embodiment on the margin area added by the margin adder 234.

Meanwhile, the image edited by the text overlay unit 232 and the margin adding unit 234 is resized by the image resizing unit 235 as in the first embodiment, so that the image having a desired size and proportion suitable for the layout is resized. Can be generated.

In addition, the image overlay unit 233 may generate an edited image by overlaying a plurality of original images with each other according to the fourth embodiment of the present invention. For example, when two original images are to be overlaid with each other, a plurality of images are provided by the image analysis module 220 according to an embodiment of the present invention to detect a main area of each original image, and the image overlay unit ( In 233, a desired edited image may be generated by overlaying the detected main areas of the original image so that they are not overlapped with each other.

As such, the edited image produced by the image editing module 230 according to each embodiment is stored in the edited image database 250, and the stored edited image is applied to a web server to form various types of web pages. do. That is, according to the screen size of the user terminal (for example, phone, tablet, notebook, PO, etc.) as described above can be configured to provide a web page to the user terminal.

<Automatic image editing by image analysis>

3 is a flowchart illustrating an automatic image editing procedure by image analysis according to an exemplary embodiment of the present invention. Referring to FIG. 3, as described above, a main region is detected by analyzing a fed original image according to an embodiment of the present invention (S301). In this case, as an image editing method, when the main area is to be cropped and edited (S302), the detected main area is cropped to generate an image according to the layout of the corresponding page to form a web page. In this case, the main region cropped by the image generating method may be generated by resizing, and the main region may be cropped in consideration of the layout to which the image is applied. In this way, the generated edited image (eg, thumbnail image) is applied to the corresponding layout of the web page.

Meanwhile, as described above, the text or the image may be overlaid on the margin area instead of the main area without cropping the original image. Therefore, after detecting the margin area other than the main area (S304) in various ways described below, the text or another image may be overlaid.

If the text is to be overlaid on the margin area (S305), it is first determined whether the detected margin area is sufficient to overlay the text (S306). The determination method may calculate the size of the entire text area in consideration of the minimum font size of the text to be overlaid, and determine whether the overlay is possible by comparing with the size of the detected margin area.

As a result of the determination, when the size of the margin area is not sufficient to overlay the text, a margin area is added to the original image or the image in which the main area is cropped according to a new embodiment of the present invention (S307). Then, the text is overlaid on the added margin area (S308).

As a result of the determination, when the size of the margin area is sufficient to overlay the text, the text is overlaid on the detected margin area (S308).

Meanwhile, when it is desired to overlay at least one other original image on the original image according to another embodiment of the present invention (S309), the plurality of images are overlaid with each other. At this time, it is preferable to detect a main region from each original image and to overlay the main regions of each original image so that they do not overlap each other.

For example, when the first image and the second image are to be overlaid according to the layout, the main region and the margin region of the first image and the second image are detected, respectively, and the main region of the second image is cropped to You can overlay on the margin area. By doing so, the main region of the first image and the main region of the second image do not overlap each other, and the main objects on each image can be displayed without missing. Alternatively, the edited image may be generated by cropping the main region of the first image and the main region of the second image, respectively, and rearranging each cropped image to fit the layout.

Hereinafter, a method of detecting the main region and the margin region from the original image will be described in detail with reference to FIGS. 4 to 8.

<Main Area Detection Method>

4 is a flowchart illustrating a detailed procedure of main region detection according to an embodiment of the present invention. Referring to FIG. 4, an area included in the identified object may be detected as a main area by identifying an object included in an original image to be edited. In this case, the object may be a face, text, or other main object.

For example, it may be first detected whether there is a face included in the image (S401). When there is a face detected as a result of the detection (S402), the area including the detected face may be determined as a main area (S403). In this case, a method of detecting a face from an image may be implemented by various known image processing methods, and a detailed description thereof will be omitted. On the other hand, when a plurality of faces are detected as a result of the face detection, the importance of each face (eg, the importance determined based on the size, location, sharpness, etc.) is determined to determine areas including the faces of high importance among the plurality of faces. The main area may be determined as the main area, or one area including all of the detected plurality of faces may be determined as the main area.

In addition, the main region may be determined by analyzing whether the text is included in the image (S404). In this case, the text may be detected using an optical character reader (OCR) technique as a method of detecting text in the image. As such, when text is detected in the image (S405), the detected text area may be determined as the main area (S406).

In addition, according to an embodiment of the present invention, it may be detected (S407) whether at least one predetermined specific object exists. As a result of the detection, when a predetermined specific object exists in the image (S408), the area including the detected object may be determined as the main area (S409).

Meanwhile, in the above-described object detection method, each method of detecting a face, text, and other main objects may be selectively applied, and the present invention is not limited to the detection order shown in FIG. 4.

5 is a flowchart illustrating a main region detection procedure by image pattern analysis according to an exemplary embodiment of the present invention. Referring to FIG. 5, unlike the method described above with reference to FIG. 4, a main region may be detected by analyzing a pattern of the entire image without detecting a specific object.

In other words, when there is a part of the image that is in focus and a part that is not in focus, the area having a relatively high average value of sharpness may be determined as a main area through sharpness analysis. In this case, in the sharpness analysis method, it may be determined that the shorter the period of change of the peripheral pixel value around the specific pixel in the image, the higher the sharpness.

Alternatively, a difference value of color or contrast with respect to neighboring pixels may be obtained for each pixel in the image, and a region having a relatively high average of the difference values may be determined as a main region.

As another method, the main region may be determined by combining the above-described focus value and contrast value. That is, the focus value and the contrast value of each pixel may be calculated (S501 and S502), and the saliency value may be determined (S503) by calculating the focus value and the contrast value. For example, the prominence value may be calculated by multiplying the focus value and the contrast value.

As such, when the prominence value for each pixel is calculated, the distribution of the prominence value in the image is modeled as a two-dimensional Gaussian distribution (S504), and a region in which the prominence value of a predetermined value or more is aggregated is included in the image. It is determined as the main area (S505). In this case, one or more main areas may be detected according to the distribution of the prominence values in the image.

As such, the main region extracted according to the embodiment of the present invention may have a shape 620 of a closed curve (for example, an ellipse) basically representing the size, shape, direction, etc. of the main region, as shown in FIG. 6B. As shown in FIG. 6A, it may be finally determined as a figure 610 such as a quadrangle including the closed curve (s).

<Margin Area Detection Method>

7 is a flowchart illustrating a procedure of detecting a non-salient area according to an embodiment of the present invention. On the other hand, the margin area in the present invention means a region having a relatively low impact in the image, and has a meaning opposite to the main region. Accordingly, the remaining area except for the main area determined as described above may be determined as the blank area, or the blank area may be determined by another method. Meanwhile, some of the main region and the margin region may overlap each other depending on the shape of the main region detected above.

Referring to FIG. 7 as one embodiment for extracting the margin area, as described above, the main area of the image is detected (S701). Next, as shown in FIG. 8A, the remaining area except for the main area 810 is designated as the first step blank area 820 which is a candidate area of the blank area (S702).

Then, image segmentation is performed on the designated first step margin area (S703). In this case, the screen division is an operation of dividing adjacent pixels having similar pixel values (for example, focus value, contrast value, prominence value, etc.) and finally dividing one screen into multiple areas having similar pixel values. For example, the color or contrast of each pixel of the first step margin area is compared with adjacent pixels, and if the difference is less than or equal to a preset reference, the pixels are regarded as similar pixels and grouped into one area. Then, the region having a size or shape higher than the preset second threshold while having the rate of change of the pixel value lower than the preset first threshold value among the divided regions as shown in FIG. 8B. The area 830 is set (S704).

Meanwhile, the first step margin area refers to an area which may be cut out or overlapped with another image, and the second step margin area refers to an area having a flat pixel value with a small screen change rate among the first step blank areas. Each of the margin areas determined as described above may be applied differently in a subsequent overlay editing process (S705). For example, in the following embodiments of the present invention, the first step margin area may be used when overlaying a plurality of images, and the second step margin area may be used when overlaying text on an image.

In the above, the method for detecting the main region and the margin region according to the embodiment of the present invention with reference to FIGS. 4 to 8 in detail. Next, various image editing methods according to embodiments of the present invention will be described in detail with reference to FIGS. 9 to 14.

First Embodiment Crop Main Area

9 is a flowchart illustrating a procedure of cropping an original image according to the first embodiment of the present invention. Referring to FIG. 9, an image may be edited by cropping a main region of the original image according to the first embodiment of the present invention.

That is, as described above, when the main region of the original image is detected (S901), the image is considered in terms of the layout on the web page, and the characteristic value (eg, the thumbnail image) of the region where the edited image (eg, the thumbnail image) is to be placed Image size, aspect ratio, etc.) are received (S902). Then, the image is edited by cropping and resizing the detected main area according to the input characteristic value (S903). In another embodiment, a web page may be configured by calculating an optimal crop ratio and size for a main area of an image for a plurality of contents, and suggesting a content layout most suitable for a combination of the images.

Meanwhile, according to an embodiment of the present invention, the determined main region may be cropped and then resized according to a corresponding size or aspect ratio, and when cropping the main region, it may be implemented to crop in consideration of the corresponding size or aspect ratio. .

10 is a diagram illustrating an image cropped from an original image according to the first embodiment of the present invention. Referring to FIG. 10, as described above in the original image, the cropped image may be disposed on a web page by cropping the main region 1000. For example, by cropping around a main area for use on a relatively small screen such as a smartphone, or cropping to a ratio for a specific layout, an image can be automatically edited effectively.

<Second Embodiment-Image + Text Overlay>

11 is a flowchart illustrating a procedure of overlaying text on a margin area according to a second embodiment of the present invention. Referring to FIG. 11, the main region is detected as described above in the original image (S1101), and the remaining region except the detected main region is set as a margin region (S1102).

Next, the size of the margin area is calculated to determine the length of the text string that can be overlaid (S1103), and text (eg, an article title, a part of content, etc.) corresponding to the corresponding area is automatically overlaid (S1106). In this case, according to an embodiment of the present invention, the color of the margin area to which the text is overlaid is analyzed (S1104), and the color of the text is determined in consideration of the color of the analyzed margin area (S1105). For example, it is desirable to determine the color of the margin area and the complementary color as the color of the text so that the text can be easily seen according to the color of the margin. In addition, when the color of the margin area is a dim color, the color of the text may be set to a font of an opposite color or the text background may be boxed as a separate color.

That is, referring to FIG. 12, the main region 1200 and the margin region 1210 are detected from the original image shown on the left side, and the size and color of the margin region 1210 are analyzed. That is, the length of the text column to be arranged may be determined by analyzing the size of the margin area 1210, and the color of the text to be arranged may be determined by analyzing the color of the margin area 1210.

Therefore, as shown in the edit image shown on the right, the headline 1220 of the text string 'fresh spring interior know-how' may be disposed on the top of the margin area 1210 in a color opposite to the color of the margin area 1210. At the bottom, a text string may be arranged in the form of a text box 1230. The text box 1230 may increase the readability of the text by setting the color of the text in the box and the background color of the text box 1230 to opposite colors.

Example 3-Adding a Margin Region to an Image

FIG. 13 is a diagram illustrating an image in which a margin area is added to an original image according to a third embodiment of the present invention. FIG. Referring to FIG. 13, when a separate blank area is not detected in the original image, the background image complexity outside the main area detected in the original image is determined, and when the image complexity is low, the color of the background is analyzed and the base is analyzed. The color image is extracted and a margin area 1310 of a color (eg, a similar color, complementary color, the same color, a color of a main area, etc.) that matches the extracted basic color according to an embodiment of the present invention is extracted from the original image 1300. Can be added to

As such, an additional title / caption area that can overlay text in a simple background area outside the main area can be created as a margin area. Thus, a new image in which the margin area 1310 is added to the original image 1300 may be generated, or a new caption area may be generated by using HTML / CSS outside the image.

<Example 4-Image + Image Overlay>

14 is a view illustrating an image in which a plurality of images are overlaid according to a fourth embodiment of the present invention. When there are a plurality of images in the content to be placed on a web page or the like, according to the fourth embodiment of the present invention, the plurality of images may be overlaid and automatically edited by comparing the size and position of the main area and the margin area of each image. .

For example, referring to FIG. 14, first, the main region 1411 is determined in the first image 1410 at the top, and the main region 1421 is determined in the second image 1420 at the bottom using the above-described method. .

Then, the main region 1421 of the second image 1420 is overlaid on the margin area of the first image 1410 or the first image 1410 is overlaid on the margin area of the second image 1420. The main area 1411 may be overlaid. This makes it possible to automatically overlay images without having each major area of the plurality of images overlap each other. Meanwhile, although FIG. 14 illustrates a method of overlaying two images, it is also possible to overlay three or more images by the same method.

The various operations and functions as described herein with respect to the various methods may be performed by any of a number of types of functions, such as a particular processing function and / or a processing function implemented therein, and / &Lt; / RTI &gt; For example, such functions may be described herein, as well as performing various operations and processes as described herein, or any other operations and functions as described herein, etc., or the like, And may perform such operations, processes, and the like.

In some embodiments, such functions (which may be implemented on the same device or on distinct devices) may be implemented in such a manner and in accordance with various aspects of the present invention, etc., and / Operations and functions, and their equivalents. In some embodiments, such processing is performed together by the first functional portion in the first device and the second functional portion in the second device. In other embodiments, such processing, operations, etc. are performed entirely by the processing units within one particular apparatus. Even in other embodiments, such processing and operations are performed using at least a first functional portion and a second functional portion within a single device.

Also, in the foregoing description, the terms &quot; system, &quot; &quot; device, &quot; and &quot; section &quot; may be a single processing device or a plurality of processing devices. Such a processing device may be a microprocessor, a microcontroller, a digital signal processor, a microcomputer, a central processing unit, a field programmable gate array, a programmable logic device, a state machine, a logic circuit, an analog circuit, a digital circuit, and / (Analog and / or digital) based on hard coding of operational instructions. The processing module, module, processing circuit, and / or processing unit may be implemented as a single memory device, a plurality of memory devices, and / or a combination of the processing modules, the processing circuitry, and / Lt; / RTI &gt; memory and / or an integrated memory element. Such a memory device may be any device that stores read only memory (ROM), random access memory (RAM), volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and / . If the processing module, the module, the processing circuit, and / or the processing unit include one or more processing devices, the processing devices may be centrally located (i.e., directly or indirectly via a wired and / Connected), distributed (e.g., cloud computing via an indirect connection over a local and / or wide area network). If the processing module, module, processing circuit, and / or processing unit implements one or more of those functions through a static machine, analog circuitry, digital circuitry, and / or circuitry, memory and / It should further be noted that the corresponding operating instructions may be internal or external to the circuit including the static machine, analog circuitry, digital circuitry, and / or logic circuitry. The memory element may be stored and the processing module, module, processing circuitry, and / or processing unit may be configured to store a hard-coded and / or hard-coded data item corresponding to at least a portion of the steps and / or functions depicted in one or more of the figures It should be further noted that it carries out operational instructions. Such a memory device or memory element may be included within the components of the manufacture.

The present invention has been described above with the aim of method steps indicative of the performance of certain functions and their relationships. The boundaries and order of these functional components and method steps have been arbitrarily defined herein for convenience of description. Alternative boundaries and sequences may be defined as long as the specific functions and relationships are properly performed. Any such alternative boundaries and sequences are therefore within the scope and spirit of the claimed invention. In addition, the boundaries of these functional components have been arbitrarily defined for ease of illustration. Alternative boundaries can be defined as long as certain important functions are properly performed. Likewise, the flow diagram blocks may also be arbitrarily defined herein to represent any significant functionality. For extended use, the flowchart block boundaries and order may have been defined and still perform some important function. Alternative definitions of both functional components and flowchart blocks and sequences are therefore within the scope and spirit of the claimed invention. Those skilled in the art will also appreciate that the functional components and other illustrated blocks, modules, and components herein may be implemented as illustrated or as separate components, such as semiconductor integrated circuits (ASICs) And the like, or any combination thereof.

The invention may also be described, at least in part, in terms of one or more embodiments. Embodiments of the invention are used herein to describe the invention, aspects thereof, features thereof, concepts thereof, and / or examples thereof. The physical embodiment of an apparatus, article of manufacture, machine, and / or process for implementing the invention may include one or more aspects, features, concepts, examples, etc., described with reference to one or more embodiments described herein . Moreover, in the entire drawing, embodiments may incorporate the same or similarly named functions, steps, modules, etc. that may use the same or different reference numbers, and as such, the functions, The steps, modules, etc. may be the same or similar functions, steps, modules, etc. or others.

Meanwhile, the term "processing unit" is used in the description of various embodiments of the present invention. A module includes a functional block implemented via hardware to perform one or more functions, such as processing one or more input signals to produce one or more output signals. The hardware implementing the functional block may operate directly by combining software and / or firmware. As used herein, a module may include one or more sub-modules that are themselves modules.

While various combinations of features and specific combinations of features of the present invention are explicitly described herein, other combinations of these features and functions are likewise possible. The present invention is not limited to the specific examples disclosed herein, and obviously incorporates these other combinations.

As described above, the present invention has been described by specific embodiments such as specific components and the like. For those skilled in the art, various modifications and variations are possible from these descriptions.

Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

210: content information database 220: image analysis module
221: object detection module 222: pattern analysis unit
223: margin detection unit 230: image editing module
231: image cropping portion 232: text overlay portion
233: image overlay portion 234: margin addition portion
235: image resizing unit
240: image configuration information database
250: edit image database

Claims (27)

A content information database storing at least one edited image;
An image analysis module configured to receive the at least one image to be edited from the content information database, and detect at least one main region among all areas of the provided image to be edited;
An image configuration information database that stores configuration information of a layout in which the edited image is to be placed; And
And an image editing module for automatically generating an edited image including a main area detected by the image analyzing module according to the configuration information stored in the image configuration information database.
The method according to claim 1, The image analysis module,
And an object detector which detects at least one object from all regions of the editing target image, and determines a region including the detected object as a main region.
The method according to claim 1, The image analysis module,
A focus value and a contrast value are calculated for each pixel in the editing target image, a salient value is calculated by the calculated focus value and the contrast value, and at least one main region according to the distribution of the calculated salient value Automatic image editing apparatus by image analysis comprising a pattern analysis unit for detecting the.
The method according to claim 1, The image analysis module,
And a margin detector configured to determine a region excluding the detected main region among all regions of the editing target image as a margin region.
The method of claim 4, wherein the margin detection unit,
An area excluding the detected main area is set as a first step margin area, a screen is divided into a plurality of areas having a similar pixel value with respect to the first step margin area, and a preset first threshold value of each divided area is set. An apparatus for automatically editing an image by analyzing an image, wherein the region having a change rate of a lower pixel value and having a size or shape higher than a preset second threshold value is set as a two-step margin region.
The apparatus of claim 5, wherein the first step margin area and the second step margin area are differently applied in an overlay editing process.
The method of claim 1, wherein the image editing module,
And an image cropping unit for cropping a main region detected by the image analyzing module from the editing target image.
The method of claim 4, wherein the image editing module,
And an image overlay unit for overlaying text on the margin area.
The apparatus of claim 8, wherein the color of the text to be overlaid on the margin area is set in consideration of the color of the margin area.
The method of claim 8, wherein the content of the text to overlay the margin area is
The automatic image editing apparatus by image analysis, which is any one selected from the caption, title, and article content of the content related to the editing target image.
The method of claim 4, wherein the image editing module,
And an image overlayer configured to overlay the main area of the second image on the margin area of the first image with respect to the plurality of editing target images from which the main area is detected.
The method of claim 1, wherein the image editing module,
And a margin adding unit configured to further add a margin area to the edited image when there is not enough space to overlay text in an area other than the main area of the entire area.
The method of claim 1, wherein the image editing module,
And an image resizing unit for resizing the size or aspect ratio of the edited image including the main area according to the configuration information.
Receiving an image to be edited in the image analysis module;
Detecting at least one major area of the entire area of the provided editing object image in the image analysis module; And
Generating an edited image by automatically editing the image according to configuration information of a preset layout by including the detected main region in an image editing module.
The method of claim 14, wherein the detecting of the main region comprises:
And detecting at least one object from all regions of the editing target image, and determining a region including the detected object as a main region.
The method of claim 14, wherein the detecting of the main region comprises:
Calculating a focus value and a contrast value for each pixel in the editing target image;
Calculating a salient value based on the calculated focus value and contrast value; And
And detecting at least one main area according to the calculated distribution of the salient values.
The method of claim 14, wherein after detecting the major region,
And determining an area excluding the detected main area as a blank area among all areas of the editing target image.
The method of claim 17, wherein the determining of the margin area comprises:
Setting an area excluding the detected main area as a first step margin area;
Splitting the screen into a plurality of areas having similar pixel values with respect to the first blank area; And
Setting an area having a size or shape higher than a preset second threshold value as a second-level margin area among the divided areas having a rate of change of a pixel value lower than a preset first threshold value; Automatic image editing method by analysis.
The method of claim 18, wherein the first step margin area and the second step margin area are differently applied in an overlay editing process.
The method of claim 14, wherein generating the edited image comprises:
Cropping the detected main region from the image to be edited.
The method of claim 17, wherein generating the edited image comprises:
Overlaying text on the margin area; automatic image editing by image analysis.
The method of claim 21, wherein the color of the text to be overlaid on the margin area is set in consideration of the color of the margin area.
The method of claim 21, wherein the content of the text to overlay the margin area is
And caption, title, and article content of the content related to the editing target image.
The method of claim 17, wherein generating the edited image comprises:
Overlaying a main area of the second image on the margin area of the first image with respect to the plurality of editing target images from which the main area has been detected.
The method of claim 14, wherein generating the edited image comprises:
If there is not enough space to overlay the text in the area other than the main area of the entire area, adding a margin area to the editing target image, image auto-editing method by image analysis.
The method of claim 14, wherein generating the edited image comprises:
Resizing the size or aspect ratio of the edited image including the main area according to the configuration information.
A computer-readable recording medium having recorded thereon a program for executing the method of claim 14.

Images