KR102435844B1 - Template providing system based on image data extraction - Google Patents

Abstract

The present invention relates to a system for providing a template based on an image data extraction comprising: an original data receiving part that receives an original document form data comprising at least one among a photograph, an image, a layout, a table, and a text; a template extraction part that extracts a template from the received original data; a template recommendation part that recommends the extracted template based on the user information; and a template editing part that controls to edit at least one among a figure, a color, a composition, a rotation, a size, and a ratio. Therefore, the present invention is capable of enabling a user to conveniently edit.

Description

TEMPLATE PROVIDING SYSTEM BASED ON IMAGE DATA EXTRACTION}

The present invention relates to a technique for editing image data using original document form data, and more particularly, an image data extraction basis for extracting and editing a template based on the original document form data and recommending a template based on user information of the template providing system.

Recently, as non-face-to-face communication technology develops due to the spread of Corona 19, non-face-to-face video conferences, presentations, document data preparation and delivery technologies have developed, and existing online shopping malls have become more active, resulting in forms or forms The importance of creating and extracting pages with existing pages is recognized as a factor that can reduce work time.

In addition, using artificial intelligence to classify photos, images, layouts, tables, and texts in the original document form material, extract them as template elements, freely arrange and edit the extracted template elements, and auto-size more sophisticated and efficient templates You can create a form.

However, when the text is positioned on the background image of the template, it is difficult to delete or change the text on the background image unless the operator has editing skills.

In addition, since all operations such as size, position, color, and spacing depend on visual sense in extracting and composing these templates, people with visual problems or disabilities had a lot of difficulty in recognizing and editing the composition and shape of the template.

Therefore, for the visually impaired, the appearance, size, location, color, and thickness of the template are expressed in Braille or by voice, and the template is extracted from the original document form data through artificial intelligence and at the same time editing the text located on the background image. A study on a template providing system configured to facilitate this is required.

Korean Patent Registration No. 10-21119730

An object of the present invention is to provide a system for easily composing templates of the same or similar form by extracting photos, images, layouts, tables, texts, etc. from original document form data.

In addition, by recommending the extracted or collected templates based on user information, the purpose is to quickly generate the templates according to the preferences and uses of the templates mainly used by the user.

In addition, by editing the text prepared on the background image in the template or original material by reflecting the color of the background image, the purpose of the text deletion, editing and input is natural and convenient for the user to edit.

In addition, an object of the present invention is to provide a system for extracting and editing the template even for the visually impaired by outputting and displaying the components included in the composition of the template in voice or Braille while extracting or editing the template.

An image data extraction-based template providing system according to an embodiment of the present invention includes an original data receiving unit for receiving an original document form data including at least one of a photo, an image, a layout, a table, and a text, from the received original data It may include a template extraction unit for extracting a template, a template recommendation unit for recommending the extracted template based on user information, and a template editing unit for controlling to edit at least one of a figure, color, composition, rotation, size, and ratio. .

In addition, the template extraction unit, the outline extraction unit for extracting the outline of the layout and table from the original data, but deletes the text included in the original data, whether the text is text on a background image or text without a background image In the case of text on the background image, the text is cut out and the cut space is filled with a color matching the background image. In the case of text without a background image, a text processing unit that cuts the text and figures, photos and pictures from the original data It may include an image extraction unit for extracting an image containing.

In addition, the color matching the background image divides an area including text into a plurality of unit blocks, and when a color in a block including text among the divided unit blocks is a single color, the single color is applied to the background image. After determining a matching color and determining whether a closed area exists in a block including text and text passes inside the closed area, the text deletion space included in the closed area may be determined as a color inside the closed area .

In addition, the text processing unit divides an area including text into a plurality of unit blocks, and when a character blank area included in the divided unit blocks has a rectangular shape, one of the vertices of the rectangle does not come into contact with the boundary of the unit block. By setting a non-existent vertex as a reference vertex, a color corresponding to the background image may be filled around the reference vertex.

In addition, the text processing unit collects RGB color code values of two pixels including a background image among the four pixels adjacent to the reference vertex, and calculates the average value of the collected RGB color code values as the RGB color code value of the reference vertex. When two or more pixels among the four pixels adjacent to the pixel of the character blank area adjacent from the reference vertex filled with the color code have a color, the average color code value of the pixels with the color code is can be entered.

Also, the outline extraction unit may include a boundary line extraction unit for extracting a boundary line of a line and a layout object indicating a boundary within a document, and a boundary extraction unit for extracting a border of a table or figure.

In addition, the template editing unit may provide a filter, resizing, cropping, rotation, drawing, text, figure, sticker, frame, corner, background color and merging function.

According to the present invention, it is possible to provide a system for easily composing templates of the same or similar form by extracting photos, images, layouts, tables, texts, and the like from original document form data.

In addition, by recommending the extracted or collected templates based on user information, it is possible to quickly generate the templates according to the preferences and uses of the templates mainly used by the user.

In addition, by editing the text provided on the background image in the template or original material by reflecting the color of the background image, deletion, editing, and input of text can be naturally and conveniently edited by the user.

In addition, by outputting and displaying the components included in the composition of the template in voice or Braille while extracting or editing the template, it is possible to provide a system for extracting and editing the template even for the visually impaired.

1 is a diagram illustrating a block diagram of a template providing system based on image data extraction according to an embodiment of the present invention.
2 is a diagram illustrating an intermediate block diagram of a template extraction unit in a template providing system according to an embodiment of the present invention.
3 is a diagram illustrating original data received in a template providing system according to an embodiment of the present invention.
4 is a diagram illustrating the division of the original data received in the template providing system according to an embodiment of the present invention into unit blocks and displaying the same.
5 is a diagram illustrating a portion in which a closed area exists in a block including a character blank area in the original data divided into unit blocks using the template providing system according to an embodiment of the present invention.
6 is a diagram illustrating a portion in which a background image including text exists in a single color in the text processing unit of the template providing system according to an embodiment of the present invention.
7 is a diagram illustrating a process of inputting RGB color code values into a character blank area in a text processing unit of a template providing system according to an embodiment of the present invention.

Specific details including the problems to be solved for the present invention as described above, means for solving the problems, and the effects of the invention are included in the examples and drawings to be described below. Advantages and features of the present invention, and methods of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings.

The scope of the present invention is not limited to the embodiments described below, and various modifications may be made by those of ordinary skill in the art without departing from the technical gist of the present invention.

Hereinafter, a template providing system based on image data extraction according to the present invention will be described in detail with reference to the accompanying FIGS. 1 to 7 .

First, FIG. 1 is a block diagram of a template providing system based on image data extraction according to an embodiment of the present invention, and FIG. 2 is an intermediate block diagram of a template extraction unit in a template providing system according to an embodiment of the present invention. 3 is a view showing original data received in the template providing system according to an embodiment of the present invention, and FIG. 4 is the original data received in the template providing system according to an embodiment of the present invention. It is a diagram showing the division into unit blocks, and FIG. 5 is a closed area within the block including the character blank area in the original data divided into unit blocks using the template providing system according to an embodiment of the present invention. It is a view showing the existing part, and FIG. 6 is a view showing the part where the background image including characters exists in a single color in the text processing unit of the template providing system according to an embodiment of the present invention, and FIG. 7 is the present invention is a diagram illustrating a process of inputting RGB color code values into the character blank area 310 in the text processing unit of the template providing system according to an embodiment of the present invention.

Referring to FIG. 1 , a template providing system 100 based on image data extraction according to an embodiment of the present invention includes an original data receiving unit 110 , a template extracting unit 120 , a template recommendation unit 130 , and a template editing unit ( 140) may be included.

The original data receiving unit 110 may receive the original document form data including at least one of a photo, an image, a layout, a table, and text.

In this case, the original data receiving unit 110 may receive the text size and font data in the original document form.

In addition, the layout received in the original document form may include all of straight lines, curved lines, and geometrical lines provided in the entire document or a partial area of the document.

The template extraction unit 120 may extract a template from the received original data.

Here, the template may extract a position of a text box indicating a position of a text arranged in the original document form material, a size and a font for each position of the listed text.

Also, it is possible to extract a template including straight lines, curves, geometric lines, figures, and the like, having different thicknesses, colors, sizes and shapes used in the document layout.

Meanwhile, the template extraction unit 120 may be described in more detail with reference to FIG. 2 .

Referring to FIG. 2 , the template extraction unit 120 may include an outline extraction unit 121 , a text processing unit 122 , and an image extraction unit 123 .

The outline extraction unit 121 may extract outlines of layouts and tables from the original data.

Here, the outline extraction unit 121 may further include a boundary line extraction unit and an edge extraction unit.

The boundary line extracting unit may extract a line indicating a boundary within a document and a boundary line of a layout object.

That is, the boundary line extractor may extract a boundary line between text in a document and a layout object or a boundary line between different layout objects.

The border extraction unit may extract the border of the table or figure.

Here, the border extraction unit may further extract the borders of photos, graphs, figures, and images, and when a plurality of different photos or figures are overlapped, it is recognized as a single figure or image, and the overlapped photo or figure is of the whole. Borders can be extracted.

The text processing unit 122 deletes the text included in the original material, but distinguishes whether the text is text on a background image or text without a background image, and cuts out the text when the text is on the background image. You can determine a color that matches the background image in the cut-out space.

In this case, the determined color may mean an RGB color code value.

Here, the original data is defined as a set of units of a predetermined number of pixels as a unit block, and will be described in more detail with reference to FIGS. 3 to 4 .

Referring to FIGS. 3 to 4 , in FIG. 3 , the original data including the background image and the text on the background image may be divided into the unit blocks as shown in FIG. 4 to divide the area including the text.

As an example, in FIG. 4, 48 horizontal and 66 vertical pixels are divided into one rectangular unit block, and the number of horizontal and vertical pixels included in the unit block and the shape of the unit block can be freely modified and represented. have.

In addition, an empty space in which a background image exists in the original data and the text is deleted in an area including text on the background image may be defined as the character blank area 310 .

In this case, when the text blank area 310 is filled with a color matching the background image, the text blank area 310 may be filled with a color matching the background image.

For example, the color corresponding to the background image to be filled in the character blank area 310 is a color corresponding to the background image when the color in the unit block including text is a single color. can decide

That is, as shown in FIG. 6, when the background image color around the empty space for the character blank area 310 from which the text 'ruler' is deleted is a single color with RGB color code values of (0, 0, 0) , it is possible to input the RGB color code value of (0, 0, 0) to the character blank region 310 by determining the character blank region 310 as a single color having an RGB color code value of (0, 0, 0). have.

Meanwhile, again referring to FIGS. 4 to 5 , after determining whether a closed area exists in the block including the text and text passes within the closed area, the text deletion space included in the closed area is located inside the closed area. can be determined by the color of

Here, the closed area may mean an area of one unit block including the character blank area 310 .

At this time, if the background image of the text blank area 310 in the closed area is not a single color, the color of the text blank area 310 may be input using the color of the background image adjacent to the text blank area 310 . have.

For example, if the text area is divided into a plurality of unit blocks, and the character blank area 310 included in the divided unit blocks has a rectangular shape, one of the vertices (1 pixel) of the rectangular unit block By setting a vertex not in contact with the boundary as the reference vertex 710 , a color matching the background image may be input with the reference vertex as the center.

Here, the text processing unit 122 may collect RGB color code values for two pixels having a color by including a background image among four pixels adjacent to the reference vertex.

In this case, the color of the reference vertex 710 may be determined by inputting the average value of the collected RGB color code values as the RGB color code value of the reference vertex 710 .

That is, when two of the four pixels adjacent to the reference vertex 710 have RGB color code values of (46,46,46) and (50,50,50) because the background image is included, the reference The RGB color code value of the vertex may be input as (48, 48, 48), which is the average of the RGB color code values of the two pixels including the background image.

Also, a pixel in the character blank area 310 closest to the reference vertex 710 filled with the RGB color code value may be determined as a color calculation required pixel.

In this case, when RGB color code values exist in two or more of the four pixels adjacent to the determined color calculation required pixel, the average RGB color code value of the pixels in which the RGB color code values exist is calculated, and then the color The calculation required pixel may be re-determined as a pixel closest to the pixel from which the RGB color code value is calculated among the pixels of the character blank area 310 .

Here, the process of inputting the RGB color code value of the character blank area by inputting the RGB color code value of the pixel required for color calculation adjacent to the reference vertex 710 will be described in more detail with reference to FIG. 7 .

Referring to FIG. 7 , starting with the reference vertex 710 , the RGB color of a first pixel 741 including a background image and a second pixel 742 adjacent to the reference vertex 710 and a background image is included. You can input the average of the code values.

At this time, since the RGB color code value of the reference vertex 710 is input, the two adjacent pixels of the first color calculation required pixel 721 and the second color calculation required pixel 722 adjacent to the reference vertex are RGB color code values. is inputted, the average of the RGB color code values of the two adjacent pixels may be calculated and inputted to the first color calculation required pixel 721 and the second color calculation required pixel 72 , respectively. When RGB color code values of the first and second color calculation required pixels 721 and 722 adjacent to the reference vertex 710 are input, the 3-1th color calculation required pixel 731, Since RGB color code values are input to 2 of the 4 pixels adjacent to the 3-2 color calculation required pixel 732 and the 3-3 color calculation required pixel 733, the 3-1 color calculation required pixel ( 731), the RGB color code values of the 3-2 color calculation required pixel 732 and 3-3 color calculation required pixel 733 may be input. Through this process, all RGB color code values of the character blank area 310 can be input.

That is, before the color of the reference vertex 710 is determined, the background image is included among the four pixels adjacent to the pixel of the character blank area 310 except for the reference vertex 710, so that the pixel having an RGB color code value is However, after the color of the reference vertex 710 is filled, two pixels among the four adjacent pixels have RGB color code values, so the average value of the RGB color code values applied to the two pixels is applied to the pixel. It can be applied by determining the color of

In addition, in calculating the average of the RGB color code values of the pixels of the character blank area 310 , in the case of a pixel located immediately below the reference vertex 710 , the four adjacent pixels before the color of the reference vertex 710 are inputted. Although only one pixel among the pixels had the RGB color code value (50, 50, 50), when the color of the reference vertex 710 is input as (48, 48, 48), the RGB values of two adjacent pixels It may be input as (49, 49, 49), which is the average value of the color code values.

Through this process, it is possible to determine and input the RGB color code value of a pixel whose RGB color code value is not determined among the pixels of the character blank area 310 .

On the other hand, as the RGB color code value of the character blank portion 310, the average value of at least two adjacent pixels may be input, and when all four adjacent pixels have RGB color code values, the RGB color code value of the four pixels You can enter the average of

As another example, the average RGB color code value for the colors of the four vertices of the square block by arbitrarily designating a rectangular block of the minimum size that can contain the text of the character blank part 310 is set to the character blank part ( 310) and can be input.

Meanwhile, in the case of text without a background image, the text may be simply cut out.

The image extraction unit 123 may extract an image including a figure, a photograph, and a figure from the original data.

Here, after extracting the figure, photo, and figure, categories of the figure, photo, and figure are automatically recognized and stored separately for each recognized category.

The template recommendation unit 130 may recommend the extracted template based on user information.

That is, when the user creates a template without original data, not in a format that is reconstructed or modified by using the original data, the template may be recommended through the template recommendation unit 130 .

Here, the user information may refer to template format information selected by the user from among templates classified into a report format, a presentation format, an advertisement format, a diary format, a freeform format, and the like.

In addition, in response to the template format selected by the user, a template including examples such as photos, figures, graphs, text boxes (configured by limiting the text input range to a rectangular border), layouts, etc. can be recommended. have.

The template editing unit 140 may control to edit at least one of a figure, a color, a composition, a rotation, a size, and a ratio.

More specifically, the template editing unit 140 may provide a filter, resizing, cropping, rotation, drawing, text, figure, sticker, frame, corner, background color and merging function.

Here, the filter includes a filter function that edits the entire color or texture of the template, such as grayscale, black & white, vivid, invert, vintage, polaroid, film, color enhancement, brown, sepia, suppression, bright, gamma, etc. can be applied to

In addition, in the resizing, the user directly inputs the length and height to adjust the size, but by selecting whether to apply the option to maintain the aspect ratio and use a percentage, the template is edited by adjusting the size from simple resizing to the original ratio can do.

In addition, the cropping is performed by directly inputting the length and height, or freely setting using a mouse, square, 4:3 ratio, 16:9 ratio, 5:3 ratio, 5:4 ratio, 6:4 ratio, 7:5 The template may be edited by selecting at least one of a ratio and a ratio of 10:8 to cut out a figure, a photo, and a part of the picture.

In addition, the rotation uses the upper drag bar to fine-tune parts of figures, photos and pictures from -44° to 44° in 1° increments, clockwise 90° rotation, counterclockwise 90° rotation, vertical symmetry and selecting left-right symmetry to rotate the template to edit the template.

In addition, the drawing is performed by selecting an RGB color code value between (0,0,0) and (256,256,256) and selecting the brush type as circular, spraying, circular spraying, mosaic, horizontal coloring, vertical coloring, grid. The template can be edited by selecting at least one of the pattern coloring formats, selecting a brush size divided into steps 1 to 5, and then drawing on the template.

Also, as with the drawing, the template can be edited by selecting a text color using an RGB color code value, setting a font and text size, and adding text to the template. Here, the font may provide 360 types of Display, 180 types of Hand Writing, 30 types of fixed width (Mono Space), 510 types of Sans Serif and 270 types of Serif.

In addition, the template can be edited by adding a circle, a square, a triangle, an ellipse, an arrow 1, an arrow 2, an arrow 3, a line, a star, a polygon, and a badge provided by default to the figure. In this case, the figure can be used for editing the template by adding photos, pictures, and figures extracted by the template extraction unit 120 .

In addition, the sticker can be divided into emoticon type, graffiti type, landmark type, star type, cloud type, speech bubble type, traffic type, and beach type, and edit can be performed by adding stickers suitable for each theme to the template.

In addition, the frame is provided on the outside of the template and provided as a basic picture frame, a pine picture frame, an oak picture frame, a rainbow, a grunge 1, a grunge 2, an ebony, an art 1 and an art 2, can be added

In addition, the corners, if the template is angled in the form of a rectangle, filleting is performed in the same way based on the four vertices, but the template can be edited by adjusting the radius of the fillet through a drag bar using a mouse. .

Also, as for the background color, the template may be edited by coloring the background color of the outer side that does not include the template content with a color selected using an RGB color code.

Also, in the merge, a template in which all elements edited and applied by the template editing unit 140 are combined may be stored as one unique template.

On the other hand, the template providing system, the original data, figures, pictures, texts, layouts, and all the above-mentioned template components can deliver the shape and size of image data to the visually impaired through Braille or voice.

Here, the template providing system may provide a Braille receiver and a Braille transmitter for transmitting and receiving template information through a Braille keypad to transmit/receive template data to/from the visually impaired.

In addition, information related to dimensions such as line thickness is provided through the distance between the braille keypads, and dimension information in a range smaller than the distance between the braille keypads is expressed as the depth of the key press of the braille keypad to provide dimension information to the visually impaired. can do.

Here, the braille keypad has a shape of six cylinders, and two horizontal and three vertical arrays are provided as one braille keypad, and the braille keypad may be composed of eight horizontal and five vertical braille keypads.

For example, the distance between the cylindrical buttons in the braille keypad is 4 mm between one keypad, the distance between the closest buttons between the braille keypads is 7 mm, and the key pressing depth is 0 to 3 mm when the one cylindrical button is input. In this case, the length of 3 mm or less can express the length to the visually impaired by controlling the depth of the key press of the cylindrical button. In addition, for a length exceeding 3 mm, it can be expressed by combining the distance between buttons in the braille keypad. Here, the object for expressing the length may be a line thickness of the template, a shape of a figure, a character symbol, or the like.

On the other hand, the voice providing function can provide the voice of pronunciation, spacing, punctuation marks, and special symbols of text, and output the shapes of figures, pictures, photos and lines using AI as voices to provide them to the visually impaired.

Through the above process, the image data for the visually impaired is provided by the system extracting photos, images, layouts, tables, texts, etc. from the original data and editing them as templates according to the user's intention, but mixed with the background image in the original data When changing the composition of the text, the text processing unit applies the RGB color code for each pixel depending on whether the background image is a single color to naturally fill the character blank area 310, and edits each template element using the template editing unit. It is possible to increase the convenience of template creation and editing by providing template information through voice and braille keypads for the visually impaired.

According to an embodiment of the present invention, it is possible to provide a system for easily composing a template of the same or similar form by extracting a photo, an image, a layout, a table, text, etc. from an original document form material.

In addition, by recommending the extracted or collected templates based on user information, it is possible to quickly generate the templates according to the preferences and uses of the templates mainly used by the user.

In addition, by editing the text provided on the background image in the template or original material by reflecting the color of the background image, deletion, editing, and input of text can be naturally and conveniently edited by the user.

In addition, by outputting and displaying the components included in the configuration of the template in voice or Braille while extracting or editing the template, it is possible to provide a system for extracting and editing the template even for the visually impaired.

As described above, although one embodiment of the present invention has been described with reference to the limited examples and drawings, one embodiment of the present invention is not limited to the above-described embodiment, which is common knowledge in the field to which the present invention pertains. Various modifications and variations are possible from such a base material. Therefore, one embodiment of the present invention should be understood only by the claims described below, and all equivalents or equivalent modifications thereof will fall within the scope of the spirit of the present invention.

100: Template providing system based on image data extraction
110: original data receiving unit
120: template extraction unit
121: outline extraction unit
122: text processing unit
123: image extraction unit
130: template recommendation unit
140: template editing unit
310: character space
710: reference vertex
721: pixel of the first character blank area adjacent to the reference vertex
722: pixel of the second character blank area adjacent to the reference vertex
731: pixel of 3-1 character blank area
732: pixel of 3-2 character blank area
733: pixel of 3-3 character blank area
741: the first pixel containing the background image
742: second pixel containing the background image

Claims (7)

an original data receiving unit for receiving original document form data including at least one of a photo, an image, a layout, a table, and a text;
a template extraction unit for extracting a template from the received original data;
a template recommendation unit for recommending the extracted template based on user information; and
a template editing unit that controls to edit at least one of a figure, a color, a composition, a rotation, a size, and a ratio;
including,
The template extraction unit,
an outline extraction unit for extracting outlines of layouts and tables from the original data;
Deleting the text included in the original material, but distinguishing whether the text is text on a background image or text without a background image, cuts the text and matches the background image in the cut space if it is text on the background image fill color,
In case of text without a background image, a text processing unit for cutting the text; and
an image extraction unit for extracting an image including a figure, a photograph, and a drawing from the original data;
includes,
The color matching the background image is,
dividing an area including text into a plurality of unit blocks, and when a color in a block including text among the divided unit blocks is a single color, determining the solid color as a color matching the background image;
Image data extraction base, characterized in that after determining whether a closed area exists in a block containing text and text passes inside the closed area, the text deletion space included in the closed area is determined by the color inside the closed area of the template providing system.
delete delete According to claim 1,
The text processing unit,
A text area is divided into a plurality of unit blocks, and when a character blank area included in the divided unit blocks is in the shape of a rectangle, a vertex of the rectangle that does not come into contact with the boundary of the unit block is set as a reference vertex to fill a color matching the background image with the reference vertex as the center.
5. The method of claim 4,
The text processing unit,
Collecting RGB color code values of two pixels including a background image among the four pixels adjacent to the reference vertex,
inputting the average value of the collected RGB color code values as the RGB color code values of the reference vertices;
A pixel in a character blank area closest to the reference vertex filled with the RGB color code value is determined as a color calculation required pixel, and an RGB color code value exists in two or more of the four pixels adjacent to the determined color calculation required pixel. In this case, after calculating the average RGB color code value of the pixels in which the RGB color code value exists,
The color calculation required pixel is recrystallized to be the pixel closest to the pixel from which the RGB color code value is calculated among the pixels in the character blank area, and the process of calculating the average RGB color code value is the RGB color code for the pixel in the character blank area. A template providing system based on image data extraction, characterized in that it is repeated until all values are calculated.
According to claim 1,
The outline extraction unit,
a boundary line extracting unit for extracting a boundary line of a line indicating a boundary within a document and a boundary line of a layout object; and
a border extracting unit for extracting the border of a table or figure;
An image data extraction-based template providing system comprising a.
According to claim 1,
The template editing unit,
A template providing system based on image data extraction, characterized in that it provides filter, resizing, cropping, rotation, drawing, text, shape, sticker, frame, corner, background color and merging functions.

Images