KR20230083971A - A method for translating and editing text contained within an image and a device for performing the same - Google Patents

Abstract

According to one embodiment of the present application, provided is a method for translating target text included in an image, which includes the steps of: obtaining an image, wherein the image is an image including target text; extracting a target text area, which is an area corresponding to at least a portion of the target text, from the obtained image; obtaining translated text by translating the target text included in the target text area using a predetermined algorithm; processing the target text area according to predetermined standards; and inserting and placing the obtained translated text into the target text area.

Description

A method for translating and editing text included in an image and a device for performing the same

The present invention relates to a method for translating text included in an image and editing the translated text, and an apparatus for performing the same.

Recently, the number of users using overseas purchasing agency services is rapidly increasing, and accordingly, the number of overseas purchasing agency online sellers is also increasing. Overseas purchasing agency online sellers search for overseas products, collect information about them, translate the product information, and reprocess it before registering and selling it in the market. There are several limitations in translating information about overseas products. do.

The text translation service is basically supported in the web browser, but the translation service for the text included in the image is not activated, so the image translation task takes a long time and is expensive compared to the text translation task. do.

Accordingly, there is a demand for an image translation solution capable of automatically translating text included in an image and properly editing the translated text.

One object of the present invention is to provide a method of automatically translating text included in an image and editing the translated text, and an apparatus for performing the same.

The problem to be solved by the present invention is not limited to the above-mentioned problems, and problems not mentioned will be clearly understood by those skilled in the art from this specification and the accompanying drawings. .

A method for translating target text included in an image disclosed in this application includes acquiring an image, wherein the image is an image including the target text; extracting a target text area corresponding to at least a part of the target text from the obtained image; obtaining translated text by translating target text included in the target text area through a predetermined algorithm; processing the target text area according to a predetermined criterion; and inserting and arranging the obtained translated text into the target text area.

The solutions to the problems of the present invention are not limited to the above-described solutions, and solutions not mentioned will be clearly understood by those skilled in the art from this specification and the accompanying drawings. You will be able to.

According to an embodiment of the present application, text included in an image is translated in real time, reflected in the image, and a service capable of editing the translated text is provided, so that the image can be translated and edited efficiently and conveniently.

Effects of the present invention are not limited to the above-mentioned effects, and effects not mentioned will be clearly understood by those skilled in the art from this specification and the accompanying drawings.

1 is a diagram for describing a system for translating and editing text included in an image according to an exemplary embodiment.
2 is a diagram for explaining the configuration of a server according to an embodiment.
3 is a diagram for explaining an image translation and editing method according to an exemplary embodiment.
4 is a diagram for illustratively explaining that a server extracts target text from an image.
5 is a diagram for illustratively explaining a method of extracting, by a server, standardized target text and unstructured target text from an image.
6 is a diagram for illustratively describing a method for obtaining translated text by a server according to an exemplary embodiment.
7 is a diagram for illustratively explaining a method for obtaining translated text by a server according to another embodiment.
8 is a diagram for illustratively explaining a method of analyzing product information from an image and then using it to translate text.
9 and 10 are diagrams for illustratively explaining a method for a server to selectively translate text included in an image.
11 and 12 are diagrams illustratively illustrating a specific method for a server to translate target text.
13 is a diagram for illustratively explaining a method of inserting the acquired translated text into an image.
14 and 15 are diagrams for illustratively explaining a method of inserting translated text reflecting feature information (style information) of target text into an image.
16 and 17 are diagrams for explaining a method of obtaining target text feature information according to an exemplary embodiment.
18 is a diagram for exemplarily explaining a method for a server to additionally modify and insert translated text.
FIG. 19 is a diagram for explaining a method for a server to additionally modify translated text when the translated text is inserted out of an area within an image.
20 and 21 are diagrams for illustratively explaining UX/UI of software that enables additional editing of inserted and arranged translated text and storage and management of edited images.
22 is a diagram for exemplarily explaining a method of extracting target text from an image using a text recognition algorithm.
23 is a diagram for illustratively explaining a method of defining text included in an image as one or more groups.
24 is a diagram for exemplarily explaining a method of acquiring symbol information and text area information from an image using an algorithm.
25 is a diagram for exemplarily explaining a method of classifying and determining text included in an image into word groups according to an exemplary embodiment.
26 is a diagram for illustratively explaining a method of determining a plurality of symbols included in an image as at least one word group.
27 is a diagram for exemplarily explaining a method of classifying and determining text included in an image into a line group according to an exemplary embodiment.
28 to 31 are diagrams for explaining conditions for determining a plurality of word groups as a line group by way of example.
32 is a diagram for exemplarily explaining a method of classifying and determining text included in an image into paragraph groups according to an exemplary embodiment.
33 is a diagram for illustratively explaining conditions for determining a plurality of line groups as a paragraph group.

The foregoing objects, features and advantages of the present application will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. However, the present application can apply various changes and can have various embodiments. Hereinafter, specific embodiments will be illustrated in the drawings and described in detail.

Like reference numerals designate essentially like elements throughout the specification. In addition, components having the same function within the scope of the same idea appearing in the drawings of each embodiment will be described using the same reference numerals, and overlapping descriptions thereof will be omitted.

If it is determined that a detailed description of a known function or configuration related to the present application may unnecessarily obscure the subject matter of the present application, the detailed description thereof will be omitted. In addition, numbers (eg, first, second, etc.) used in the description process of this specification are only identifiers for distinguishing one component from another component.

In addition, the suffixes "module" and "unit" for components used in the following embodiments are given or used interchangeably in consideration of ease of writing the specification, and do not have meanings or roles that are distinguished from each other by themselves.

In the following examples, expressions in the singular number include plural expressions unless the context clearly dictates otherwise.

In the following embodiments, terms such as include or have mean that features or components described in the specification exist, and do not preclude the possibility that one or more other features or components may be added.

In the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and the present invention is not necessarily limited to those shown.

If an embodiment is otherwise implementable, the order of specific processes may be performed differently from the order described. For example, two processes that are described in succession may be performed substantially concurrently, or may proceed in an order reverse to that described.

In the following embodiments, when components are connected, a case in which the components are directly connected as well as a case in which components are interposed between the components and connected indirectly is included.

For example, when it is said that components are electrically connected in this specification, not only the case where the components are directly electrically connected, but also the case where the components are interposed and electrically connected indirectly is included.

A method of translating target text included in an image according to an embodiment, comprising: obtaining an image, wherein the image is an image including target text; extracting a target text area corresponding to at least a part of the target text from the obtained image; obtaining translated text by translating target text included in the target text area through a predetermined algorithm; processing the target text area according to a predetermined criterion; and inserting and arranging the obtained translated text into the target text area.

The image includes a plurality of target texts, and the plurality of target texts are classified into at least one target text group according to a predetermined criterion. Translating the target text for each of one or more target text groups may be included.

The plurality of target texts may include a first target text group and a second target text group classified based on any one of font type, font color, size, and language type of the target text.

The acquiring of the translated text may include translating the target text included in the first target text group through a first algorithm and translating the target text included in the second target text group through a second algorithm. can include

In the obtaining of the translated text, one of the first target text group and the second target text group corresponding to a predetermined condition may be selectively translated.

The plurality of target texts include a first type of target text and a second type of target text, wherein the first type of target text is standardized target text among the plurality of target texts, and the second type of target text is unstructured target text among the plurality of target texts, and the obtaining of the translated text includes the steps of translating the target text of the first type through a first algorithm and the target text of the second type through a second algorithm. It may include translating the target text.

The target text of the first type may be target text located in an area corresponding to a background in the image, and the target text of the second type may be target text located in an area corresponding to a product photo in the image.

The obtaining of the translated text may include translating the target text included in the target text area through a predetermined algorithm to obtain a first candidate translation text and a second candidate translation text; and determining one of the first candidate translation text and the second candidate translation text as the translated text according to a predetermined criterion.

The method of translating target text included in the image includes extracting meta data from the image, and obtaining the translated text is included in the target text area based on the extracted meta data. The translated text may be obtained by translating the existing target text.

The method of translating target text included in the image includes extracting meta data from the image, and the obtaining of the translated text includes the first candidate translated text and the first candidate translated text based on the meta data. A step of determining one of two candidate translation texts as the translation text may be included.

The method of translating the target text included in the image includes obtaining user information, wherein the user information includes information about a country to which the user belongs or a language environment of the user, and obtaining the translated text. In the step of translating the target text included in the target text area based on the obtained user information, translated text may be obtained.

The method of translating the target text included in the image includes obtaining user information, wherein the user information includes information about a country to which the user belongs or a language environment of the user, and obtaining the translated text. In the step of translating, one of the first target text group and the second target text group may be selectively translated based on the obtained user information.

The method of translating the target text included in the image includes obtaining user information, wherein the user information includes information about a country to which the user belongs or a language environment of the user, and obtaining the translated text. The doing may include determining one of the first candidate translation text and the second candidate translation text as the translation text based on the obtained user information.

The method for translating target text included in the image includes obtaining product information from the image, wherein the product information is information about a product included in the image, and obtaining the translated text. may acquire translated text by translating the target text included in the target text area based on the obtained product information.

The method for translating target text included in the image includes obtaining product information from the image, wherein the product information is information about a product included in the image, and obtaining the translated text. The may include determining one of the first candidate translation text and the second candidate translation text as the translation text based on the obtained product information.

The obtaining of the translated text may include translating the target text included in the target text area through a predetermined algorithm to obtain a candidate translated text; obtaining a recommended translation text corresponding to synonyms or similar words of the candidate translation text according to a predetermined criterion; and determining one of the candidate translation text and the recommended translation text as the translation text.

The recommended translation text may be a synonym or similar word of the candidate translation text determined based on at least one of metadata obtained from the image, product information obtained from the image, and user information.

The method of translating the target text included in the image includes acquiring style information about the target text, and inserting and arranging the translated text includes the translation of the translated text based on the acquired style information. determining the style; and inserting and arranging the translated text into the target text area by reflecting the determined style of the translated text.

The style information may include at least one of a font type, text color, text size, text arrangement direction, text alignment criterion, text border color, text kerning, and text background color of the target text.

Acquiring style information on the target text may further include analyzing an edge of the target text or analyzing feature points of the target text, and the style information may include analyzing an edge of the target text or the target text. It may be information obtained through feature point analysis.

The step of determining the style of the translated text may include: acquiring translated text to which the acquired style information is reflected; determining a translated text area, which is an area where the translated text is placed in the image, and determining whether to additionally modify the style of the translated text based on the translated text area; and when it is determined that the style of the translated text needs to be additionally modified, further transforming the style of the translated text in consideration of the translated text area and then inserting and arranging the style of the translated text into the image.

The step of determining whether to additionally modify the style of the translated text may include determining that the style of the translated text should be additionally modified when at least some of the translated text regions are out of an area within the image.

The step of inserting and arranging the translated text in the image after additionally modifying the style of the translated text includes the font type, text color, text size, text arrangement direction, text alignment criterion, text border color, text kerning, and text background color of the translated text. After additionally transforming at least one of the above, it may be inserted and arranged in the image.

The step of determining the style of the translated text may include further modifying the style of the translated text and inserting and arranging the style of the translated text when the translated text reflecting the acquired style information is inserted out of an area within the image. can

One full system

FIG. 1 is a diagram for explaining a system for translating and editing text included in an image according to an exemplary embodiment, and FIG. 2 is a diagram for explaining a configuration of a server according to an exemplary embodiment.

Referring to FIG. 1 , a method of translating and editing images according to an embodiment may be performed through a server 1000 . Referring to FIG. 2 , the server 1000 may include a control unit 100 , a storage unit 200 and a communication unit 300 .

The storage unit 200 may store neural network models, algorithms, and/or other data used to translate and edit images. In this case, the neural network model, algorithm, and/or other data stored in the storage unit 200 may be received from an external server or electronic device.

The communication unit 300 may communicate with a personal cloud device, an external server, and/or an external device. The communication unit 300 may receive data input by a user from an external device or server, and may transmit data stored on the server 1000 to an external device or server. To this end, the communication unit 300 may include one or more components enabling communication with an external device, a server, and the like, and may include, for example, at least one of a short-distance communication module, a wired communication module, and a communication module.

The short-range communication module uses a wireless communication network such as a Bluetooth module, an infrared communication module, a Radio Frequency Identification (RFID) communication module, a Wireless Local Access Network (WLAN) communication module, an NFC communication module, and a Zigbee communication module to transmit signals at a short distance. It may include various short-range communication modules that transmit and receive.

Wired communication modules include various wired communication modules, such as Controller Area Network (CAN) communication modules, Local Area Network (LAN) modules, Wide Area Network (WAN) modules, or Value Added Network (VAN) modules. In addition to communication modules, various cable communications such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), DVI (Digital Visual Interface), RS-232 (recommended standard 232), power line communication, or POTS (plain old telephone service) modules may be included.

The controller 100 may include at least one processor. At this time, each processor may execute a predetermined operation by executing at least one instruction stored in the memory. Specifically, the control unit 100 may control overall operations of elements included in the server 1000 . In other words, the server 1000 may be controlled or operated by the controller 100 .

According to an embodiment, the controller 100 may acquire an image, translate text included in the acquired image, and provide the translated text to a user. Also, the controller 100 may perform a function of allowing the translated text to be edited by the user.

A detailed description of a series of steps in which the control unit 100 performs image translation and editing will be described later.

Referring back to FIG. 1 , an image translation and editing system according to an embodiment may include a server 1000 and an electronic device 2000 . In this case, the electronic device 2000 may include a portable information communication device, such as a smart phone or a tablet. In addition, the electronic device 2000 may include various devices that automatically calculate or process data using electronic circuits.

According to an embodiment, the server 1000 may translate and edit text included in the image based on the obtained image. In this case, the server 1000 may perform image translation and editing operations based on data transmitted from the electronic device 2000 . For example, the electronic device 2000 may acquire an image by a user's input. The electronic device 2000 may transmit the acquired image to the server 1000, and the server 1000 may perform a function of translating and editing text included in the received image.

According to another embodiment, the electronic device 2000 may translate and edit text included in the image based on the obtained image. The electronic device 2000 may perform text translation and editing using an image acquired by a user's input. In this case, the electronic device 2000 may perform text translation and editing using an image acquired by a user's input, and then transmit the translated and edited image to the server 1000 . The server 1000 may store the image obtained from the electronic device 2000 in the storage unit 200 and transmit it to another external device.

Hereinafter, a method of translating and editing text included in an image is exemplarily described as being performed by the server 1000, but is not limited thereto, and may be performed by the electronic device 2000 as described above. .

2 whole process

3 is a diagram for explaining an image translation and editing method according to an exemplary embodiment. Referring to FIG. 3 , a method of translating and editing an image according to an exemplary embodiment includes acquiring an image (S1000), extracting target text from the image (S2000), translating the target text (S3000), and performing target text. Obtaining the translated text (S4000), removing the target text and background from the image (S5000), synthesizing the background (S6000), inserting the translated text (S7000), and An editing step (S8000) may be included.

3 image acquisition

The server 1000 may acquire an image through an image acquisition step (S1000). The server 1000 may obtain an image from a user's input. The image may include at least one text. The image may be an image including product-related information, but is not limited thereto, and may be various types of images uploaded on the web.

4 Target text extraction

The server 1000 may extract target text from an image acquired through the target text extraction step (S2000). For example, the target text may be text included in an image. The target text may refer to a region corresponding to text in an image.

4 is a diagram for illustratively explaining that a server extracts target text from an image. Referring to FIG. 4 , an image may include at least one text, and the server 1000 may extract at least one target text from the image.

The server 1000 may recognize target text included in an image through a character recognition algorithm (or optical character recognition (OCR)) and then determine a region corresponding to the target text region in the image as a target text region.

At least one area corresponding to text may exist in the acquired image, and the server 1000 may extract at least one text area existing in the image.

For example, as shown in FIG. 4 , a plurality of target texts may exist in an image, and the server 1000 may extract a plurality of target texts from the image. At this time, the extracted target text includes the first target text T1, the second target text T2, the third target text T3, the fourth target text T4, the fifth target text T5, and the sixth target text. It may be the text T6, the seventh target text T7, the eighth target text T8, and the ninth target text T9.

The plurality of texts may be in one language or may be in a plurality of languages. For example, as shown in FIG. 4 , the image may include target text written in Chinese (second to ninth target text) and target text written in English (first target text).

According to an embodiment, the server 1000 may extract at least one target text area from an image based on a predetermined criterion.

For example, the server 1000 may extract a target text area by classifying a plurality of texts recognized through a character recognition algorithm for each font. More specifically, the server 1000 may determine text having a first font in the image as a first target text area, and may determine text having a second font as a second target text area.

As another example, the server 1000 may extract a target text area by classifying a plurality of texts recognized through a character recognition algorithm by character size. More specifically, the server 1000 may determine text having a first size in the image as a first target text area and text having a second size as a second target text area.

As another example, the server 1000 may extract a target text area based on a position in an image of a plurality of texts recognized through a text recognition algorithm. More specifically, the server 1000 may determine, as one target text area, texts adjacent to each other within a predetermined standard or more in the image. For example, text existing in a first area of the image may be determined as a first target text area, and text existing in a second area may be determined as a second target text area.

5 is a diagram for illustratively explaining a method of extracting, by a server, standardized target text and unstructured target text from an image. Referring to FIG. 5 , the step of extracting target text ( S2000 ) may include a step of extracting standardized target text ( S2100 ) and a step of extracting unstructured target text ( S2200 ).

The image obtained by the server 1000 may include a plurality of texts. The text may include standard text located in the background area of the image and non-standard text located in a product photo in the image.

Referring to the image illustrated in FIG. 4 , the target text area extracted from the image is a standardized target text area, for example, a first target text area T1, a second target text area T2, and a third target text area T3. ), a fourth target text area T4, a fifth target text area T5, and a ninth target text area T9. Also, the target text area may include unstructured target text areas, for example, a sixth target text area T6, a seventh target text area T7, and an eighth target text area T8.

A more detailed invention of extracting target text through a text recognition algorithm will be described later with reference to the drawings.

5 Translate target text (acquire translated text)

5.1 neural network model

The server 1000 may extract standardized target text and non-standardized target text from the obtained image, and then obtain translation text from each of the extracted target texts.

6 is a diagram for illustratively describing a method for obtaining translated text by a server according to an exemplary embodiment. Referring to FIG. 6 , the server 1000 may extract the translated text in which the target text is translated through a translated text obtaining step (S4000). For example, the server 1000 may obtain translated text by acquiring an image, extracting target text, and then translating the extracted target text.

According to an embodiment, the server 1000 may obtain translated text by translating target text extracted through a pretrained neural network model (NN). The server 1000 may obtain the translated text by translating the target text through deep learning-based machine translation. However, it is not limited thereto, and although not shown in the drawing, the server 1000 may obtain the translated text by translating the target text through various known translation algorithms or devices.

Meanwhile, as shown in FIG. 6 , the server 1000 may additionally obtain a user input to obtain translated text. For example, the server 1000 extracts target text from an image, translates the target text through a neural network model to obtain at least one candidate translated text, and then, based on a user input, at least one of the candidate translated texts. can be determined as the translated text.

5.2 How to Improve Translation Quality - Using Metadata

7 is a diagram for illustratively explaining a method for obtaining translated text by a server according to another embodiment. Hereinafter, a method of improving the quality of translation performed by the server 1000 using meta data will be described with reference to FIG. 7 .

Referring to (a) of FIG. 7 , the server 1000 may extract meta data from an image through a meta data extraction step ( S2100 ).

The server 1000 may extract meta data from the image through a meta data extraction step ( S2100 ). The server 1000 may extract meta data by analyzing information included in the image based on the obtained image. For example, the server 1000 may obtain information about objects, people, and/or spaces included in the image by analyzing the obtained image.

The server 1000 may use the extracted metadata when translating target text through a neural network model (NN). The server 1000 may obtain a plurality of candidate translated texts by translating the extracted target text through a neural network model (NN) or the like. In this case, the server 1000 may determine at least one of a plurality of candidate translation texts obtained based on the meta data as the translation text.

For example, when a word and/or sentence to be translated is translated through a neural network model (NN), the word and/or sentence may be translated into at least one version. At this time, a version corresponding to the extracted meta data among at least one translated version may be determined as the final translation.

As a more specific example, the server 1000 extracts target text from an acquired image, and then translates the target text through a neural network model (NN) to form a plurality of candidate translated texts, such as a first candidate translated text and a second candidate translated text. text can be obtained. In this case, the server 1000 may determine one of the obtained first candidate translation text and the second candidate translation text that corresponds most to (or is most related to) the meta data extracted from the image as the final translation text.

5.3 How to Improve Translation Quality - Utilizing User Information

Referring to (b) of FIG. 7 , the server 1000 may additionally acquire user information through a user information acquisition step (S2300) and then obtain translated text using this.

Illustratively, the server 1000 may obtain information about the country to which the user belongs and/or the language environment of the user through the user information acquisition step (S2300). Thereafter, the server 1000 may translate the target text included in the image in consideration of the acquired country to which the user belongs and/or language environment information of the user.

As a more specific example, the server 1000 extracts target text written in the first language and/or the second language from an image, and then converts the target text into a third language determined based on user information (eg, user language environment information). can be translated into languages.

That is, the server 1000 can automatically translate the acquired image into the language used by the country to which the user belongs, taking into account the user's language environment, etc., without any external input or selection.

5.4 How to improve translation quality - Utilize product information

8 is a diagram for illustratively explaining a method of analyzing product information from an image and then using it to translate text. Referring to FIG. 8 , the server 1000 may translate text or provide recommended translation text based on product information included in an image.

When obtaining an image including product information, the server 1000 may analyze the image and obtain information (eg, product name, product option, product specification, etc.) on the product included in the image.

In addition, the server 1000 may extract the target text from the image, obtain a translated candidate text, and recommend synonyms and/or similar words of the candidate translated text to the user based on the product-related information. can do. Thereafter, the server 1000 may finally determine one of the candidate translation text and the recommended translation text provided to the user as the translation text.

As a more specific example, the server 1000 may obtain at least one candidate translation text by translating the target text. In addition, after acquiring product information through image analysis, the server 1000 may obtain at least one recommended translation text corresponding to the candidate translation text based on the product information. At least one of the candidate translation text and the recommended translation text may be provided to the user, and based on a user response thereto, the server 1000 may determine one of the candidate translation text and the recommended translation text as the final translation text. there is.

Meanwhile, in the synonym DB generation step (S4500), the server 1000 translates the target text included in the image to obtain candidate translation text, and synonyms and/or synonyms of the candidate translation text based on product information extracted from the image. A synonym DB can be created by acquiring and storing synonyms.

The server 1000 may perform image translation by utilizing the synonym DB. For example, when product information extracted through an acquired image is the same as or similar to product information stored in the synonym DB, the server 1000 recommends and translates words, sentences, etc. included in the synonym DB when translating images. It can be determined by text and provided to the user.

5.5 Unstructured text translation

As described above, the target text extracted from the image by the server 1000 may include standardized text and unstructured text. In this case, the standardized text may be translated through the above-described text recognition algorithm (or optical character reading device (OCR)).

In addition, unstructured text may also be translated through the character recognition algorithm (or optical character reading device (OCR)), but image translation may be performed through edge region analysis of the detected text to improve accuracy.

For example, the server 1000 may translate unstructured text included in an image through a neural network model. In this case, the neural network model may be trained to translate the text after extracting and analyzing the style of the edge region of the text from the image.

5.6 Selective text translation

The server 1000 according to an embodiment may selectively translate at least one text included in an image. For example, the server 1000 may selectively translate at least one of a plurality of texts included in an image according to a predetermined criterion. Hereinafter, referring to FIG. 8, a method of selectively translating text included in an image by a server will be described.

9 and 10 are diagrams for illustratively explaining a method for a server to selectively translate text included in an image. Referring to FIG. 9 , after extracting target text from an image, the server 1000 may classify the target text according to a predetermined criterion. The server 1000 may classify the text included in the image into one or more groups according to a predetermined criterion, and then perform translation on a text group satisfying a certain condition among the classified text groups.

After extracting the target text from the image, the server 1000 may classify the target text into a first target text group and a second target text group according to a predetermined criterion.

For example, the server 1000 may classify the target text included in the image into a first target text group having a first font, a second target text group having a second font, and the like, based on the font type. In this case, the server 1000 may selectively translate at least one of the first target text group and the second target text group based on a predetermined criterion or a user's input value. For example, when obtaining a user's input to translate a first font, the server 1000 may translate only the first target text group among the first target text group and the second target text group. Alternatively, when obtaining a user's input to translate the third font, the server 1000 may not translate both the first target text group and the second target text group.

As another example, the server 1000 may classify the target text included in the image into a first target text group having a first size, a second target text group having a second size, and the like, based on the character size. In this case, the server 1000 may selectively translate at least one of the first target text group and the second target text group based on a predetermined criterion or a user's input value. For example, when obtaining a user's input to translate the first size, the server 1000 may translate only the first target text group among the first target text group and the second target text group. Alternatively, when the server 1000 obtains the user's input to translate the third size, the server 1000 may not perform translation on both the first target text group and the second target text group.

As another example, the server 1000 may classify the target text included in the image into a first target text group having a first language, a second target text group having a second language, and the like, based on the language type. In this case, the server 1000 may selectively translate at least one of the first target text group and the second target text group based on a predetermined criterion or a user's input value. For example, when obtaining a user's input to translate a first language, the server 1000 may translate only the first target text group among the first target text group and the second target text group. Alternatively, the server 1000 may not perform translation for both the first target text group and the second target text group when obtaining a user's input to translate the third language.

As another example, the server 1000 classifies the target text included in the image into a first target text group having a first language and a second target text group having a second language, and then the first language determined as the representative language. And translation may be performed on a target text group related to any one of the second languages.

In this case, the representative language may be determined based on information obtained by analyzing the image. For example, the target text included in the image is classified into a first target text group written in a first language and a second target text group written in a second language, and the number of texts belonging to the first target text group is the second target text group. If the number of texts belonging to the group is greater than the number of texts belonging to the group, the first language may be determined as the representative language. Alternatively, when the average size of texts belonging to the first target text group is greater than the average size of texts belonging to the second target text group, the first language may be determined as the representative language.

The above methods of classifying the target text are exemplary, and the target text extracted according to various other criteria may be classified into one or more text groups. For example, the extracted target text may be classified into at least one text group based on the coordinates of the text location in the image, the color of the text, and the like.

Meanwhile, at least one target text group classified as described above may be translated through a neural network model. In this case, each text group classified as shown in (a) of FIG. 10 may be translated through one neural network model (NN). Alternatively, each text group classified as shown in (b) of FIG. 10 may be translated through different neural network models.

For example, target text included in an image is divided into a first target text group having a first language, a second target text group having a second language, and a third target text group having a third language, based on the language type. If classified, the first target text group may be translated through a first neural network model NN1 trained to translate a first language, and the second target text group may be translated through a second target text group trained to translate a second language. It can be translated through a neural network model NN2, and the third target text group can be translated through a third neural network model NN3 trained to translate a third language.

5.7 Exemplary Text Translation Methods

11 and 12 are diagrams for illustratively explaining a specific method for a server to translate target text. Referring to FIGS. 10 and 11 , the server 1000 may recognize text included in an image for each character, word, and/or sentence, and then analyze the style of the corresponding text.

Referring to (a) of FIG. 11 , the text included in the image may have different languages, fonts, and sizes. In this case, referring to (b) of FIG. 11 , the server 1000 may acquire text information about each character after recognizing each character included in the image one by one. However, this is just an example, and unlike FIG. 11(b), after recognizing two or more characters at once, text information about the characters can be obtained.

Referring to (a) and (b) of FIG. 12 , the server 1000 may classify and/or reclassify text included in an image into one or more groups based on the obtained text information. For example, the server 1000 may classify and/or reclassify the target text into at least one or more groups based on information on the coordinates of the text in the image.

The server 1000 may translate the target text included in the image in units of groups classified and/or reclassified in the above-described method.

6 Remove target text (Inpaint) and composite background (Rasterize)

The server 1000 acquires an image, recognizes text included in the acquired image, and translates the text, removes a text area to be translated from the image, and removes a text area from the image, and the background of the text area. can perform the function of restoring.

The server 1000 may perform a function of determining a text area where text is located in an image and then removing the text area from the image. In addition, the server 1000 may perform a function of restoring a background of a part corresponding to the text area removed from the image.

For example, the server 1000 removes the text area from the image, extracts a representative color by analyzing an area around the text area, and restores the background of the text area based on the representative color. can be performed.

As another example, the server 1000 may perform a function of removing the text area from the image and additionally removing a text area that is not completely removed (eg, an area corresponding to image noise).

The above-described method of removing a specific region (eg, text region) in an image and restoring the background of the removed portion may be performed through various known algorithms or the like.

7 Insert translated text

7.1 Extract target text information

The server 1000 according to an embodiment may obtain translated text by translating target text included in an image, and then insert the translated text into the image.

13 is a diagram for illustratively explaining a method of inserting the acquired translated text into an image. Referring to FIG. 13 , the server 1000 may insert the translated text obtained through the step of inserting the translated text (S7000) into an image.

The server 1000 may obtain translated text by translating target text extracted from an image, and then insert the translated text into a region corresponding to the target text in the image.

The server 1000 may perform post-correction on the translated text and then insert the post-corrected translated text into the image. For example, the server 1000 may insert the translated text into the image based on feature information (style information on the target text) of the target text.

14 and 15 are diagrams for illustratively explaining a method of inserting translated text reflecting feature information (style information) of target text into an image.

Referring to FIG. 14 , the server 1000 may acquire feature information by analyzing target text extracted from an image through a target text feature information acquisition step ( S7100 ). In the figure, after extracting the target text, it is shown that the target text is analyzed through a neural network model to acquire the feature information, but it is not limited thereto, and the feature information can be obtained through various known image analysis algorithms. .

The server 1000 may reflect the characteristic information to the obtained translation text through the step of applying target text characteristic information to the translated text (S7300), and may insert the translated text reflecting the characteristic information into the image. there is.

Referring to FIG. 15, the feature information is information about the target text, for example, text color information, text size information, font information, text direction information, text border color information, text kerning information, text background color information, and text alignment information. can include

16 and 17 are diagrams for explaining a method of obtaining target text feature information according to an exemplary embodiment. Referring to FIG. 16 , when the target text included in the image is standardized text, the server 1000 obtains the image, detects a target text area where the text is located in the image, and exists in the target text area. Information about the text to be played may be obtained, for example, target text feature information. Thereafter, the server 1000 may obtain translated text reflecting the target text feature information and insert it into an image.

Referring to FIG. 17 , when target text included in an image is unstructured text, the server 1000 detects and analyzes an edge region (S7110) and obtains edge region information (S7130) to perform target text feature information. can be obtained.

In addition, when the target text included in the image is unstructured text, the server 1000 detects a target text area in the image, detects feature points of the text included in the target text area, digitizes the feature points, and The target text feature information may be obtained.

Exemplarily, the server 1000 may determine the font of the target text by obtaining target text characteristic information in the above-described method. Then, when the translated text is inserted into an image, the translated text reflecting the font of the target text can be inserted into the image.

7.2 Further correction of the translated text

18 is a diagram for exemplarily explaining a method for a server to additionally modify and insert translated text. Referring to FIG. 18 , the server 1000 may determine whether the translated text needs to be additionally modified through a step of determining whether the translated text needs to be corrected (S7500) after applying the target text characteristic information to the translated text. there is.

When the server 1000 determines that the translated text reflecting the target text feature information needs to be additionally modified, the translated text may be additionally modified through a text size, alignment, or arrangement change step (S7700).

After the server 1000 reflects target text feature information in the translated text, when the translated text is inserted into an image and does not satisfy a predetermined condition, the server 1000 determines that the translated text needs to be additionally modified. can do.

For example, when the server 1000 translates a plurality of target texts included in an image, obtains first translated text and second translated text, and inserts and arranges them into the image, the first translated text and the second translated text are obtained. When two translated texts are inserted and placed in an overlapping region on the image, the server 1000 may determine that at least one of the first translated text and the second translated text requires additional modification.

In this case, the server 1000 additionally changes the style of at least one of the first translated text and the second translated text (eg, changes in size, arrangement method, alignment method, etc. of the first translated text and the second translated text). ) to insert into the image.

As another example, the server 1000 obtains the translated text by translating at least one target text included in the image, and when inserting and arranging the translated text into the image, at least a part of the translated text is inserted out of an area within the image. In this case, the server 1000 may determine that additional correction is required for the translated text.

In this case, the server 1000 may additionally change the style of the translated text (eg, change the size, arrangement method, arrangement method, etc. of the translated text) and insert it into the image.

FIG. 19 is a diagram for explaining a method for a server to additionally modify translated text when the translated text is inserted out of an area within an image. Hereinafter, referring to FIG. 19, a method of additionally modifying translated text will be described.

Referring to (a) of FIG. 19 , target text may be included in an image, and as described above, the server 1000 classifies the target text into at least one group and may translate the target text, and at the same time (or Sequentially), feature information may be extracted by analyzing the target text.

Thereafter, the server 1000 may reflect the characteristic information on the translated text as shown in (b) of FIG. 19 and then insert and arrange it into the image. In this case, at least a part of the translated text area inserted into the image may be inserted beyond the area within the image. In this case, the server 1000 may additionally change the style of the translated text and insert it into the image.

For example, as shown in (c) of FIG. 19, the server 1000 determines the size, spacing, length and/or arrangement of the translated text when at least some of the regions occupied by the translated text disposed on the image is out of the region within the image. Through a change in the method or the like, all of the area occupied by the translated text may be included in the area within the image.

8 Provide editor service

20 and 21 are diagrams for illustratively explaining UX/UI of software that enables additional editing of inserted and arranged translated text.

Referring to FIGS. 20 and 21 , when the translated text obtained by the above method is inserted into an image, the server 1000 may additionally edit the inserted translated text by the user or store and manage the edited image. function can be provided.

For example, the server 1000 may perform at least one of the font type, text color, text size, text arrangement direction, text alignment criterion, text border color, text kerning, and text background color of the translated text obtained based on the user's input. It is possible to provide a function of additionally transforming the above to be reflected in the image.

As a more specific example, the server 1000 may provide synonyms and/or synonyms of the obtained translation text to the user through the display unit of the electronic device, and may change the translated text by acquiring the user's input.

In addition to this, the server 1000 may provide the user with various text editing functions provided by general text editing tools and software, through which the user may edit and utilize the style of the translated image as desired.

9 Target text extraction method

According to an embodiment, the server 1000 or the control unit 100 may extract text included in an image using an algorithm, classify and determine one or more groups, and then manage the text.

After extracting the text included in the image, the server 1000 or the control unit 100 may classify and manage text into at least one or more groups based on a predetermined criterion, and then, when translating the text included in the image, the When performed based on classified group information, the effect of further improving translation quality can be provided.

For example, compared to performing image translation after simply extracting text included in an image, text included in an image is classified into a plurality of groups (eg word group, sentence group, paragraph group, etc.) and then classified. When image translation is performed based on the received information, the translation quality can be further improved.

An operation of extracting text included in an image, classifying, determining, and managing text into at least one or more groups may be performed by the server 1000 or by the control unit 100 of the electronic device. Hereinafter, it will be described as being performed in the control unit 100 for convenience of description.

22 is a diagram for exemplarily explaining a method of extracting target text from an image using a text recognition algorithm. Referring to FIG. 22, the server 1000 or the control unit 100 obtains symbol information included in an image using an algorithm (S9100), determines a word group using the symbol information (S9300), and selects a line group. By determining (S9500) and determining a Paragraph group (S9700), the text included in the image can be extracted.

23 is a diagram for illustratively explaining a method of defining text included in an image as one or more groups. The image may include at least one text, and the at least one text may be classified into a plurality of groups. For example, the at least one text may be determined in units of symbols, words, lines, and paragraphs.

For example, referring to FIG. 23 , each character can be defined as a symbol, a set of adjacent symbols can be defined as a word, and a set of adjacent words can be defined as a line. Also, a set of adjacent lines can be defined as a Paragraph. However, this is an example, and the text included in the image may be classified and determined into Symbol, Word, Line, and Paragraph groups by various methods and standards. Hereinafter, a method of classifying text included in an image into a plurality of groups will be reviewed.

24 is a diagram for exemplarily explaining a method of acquiring symbol information and text area information from an image using an algorithm. Referring to FIG. 24 , the control unit 100 according to an embodiment may obtain symbol information and text area information using an algorithm.

Referring to (a) of FIG. 24 , the controller 100 may extract a symbol from an image using a first algorithm. Then, the control unit 100 may obtain symbol information from the extracted symbols.

In this case, the symbol may mean each individual character included in the image, as described above with reference to FIG. 23 . In addition, the symbol information is information that can be obtained from the extracted symbol, and may be, for example, a list of all symbols included in an image, location information (coordinate information) where symbols are located in an image, and the like.

The controller 100 may obtain text area information after extracting a text area from an image using a first algorithm. In this case, the text area may refer to an area occupied by texts included in the image in the image. For example, the text area may refer to an area occupied by at least one of individual characters, words, sentences, and paragraphs in an image. As a more specific example, the text area may refer to an area occupied by at least one of symbols, words, lines, and paragraphs in an image.

In (a) of FIG. 24, it has been described that the control unit 100 acquires symbol information and text area information using one algorithm, but is not limited thereto and uses different algorithms to obtain symbol information and text area information. can be obtained.

For example, referring to (b) of FIG. 24 , the controller 100 may extract a symbol from an image through a first algorithm and obtain symbol information based on the extracted symbol. Also, the controller 100 may extract a text area from an image using a second algorithm and obtain information about the extracted text area. In this case, the first algorithm and the second algorithm may be different algorithms. In this case, illustratively, the first algorithm may be a better algorithm than the second algorithm in performance of extracting text from an image, that is, performance of extracting a symbol from an image.

25 is a diagram for exemplarily explaining a method of classifying and determining text included in an image into word groups according to an exemplary embodiment. The control unit 100 may determine each individual symbol as a word group using the acquired symbol information and text area information.

Referring to FIG. 25, the control unit 100 obtains symbol location information, obtains word area information, compares the location of the symbol with the word area, and determines a set of symbols located within the word area as a word group. can

The controller 100 may obtain word area information in the image and simultaneously obtain symbol location information. The location information of the symbol may be coordinate information. When the coordinates of the Symbol are included in the first word area in the image, the controller 100 can classify them as a first word group.

At this time, the coordinate information of the symbol may include the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate. For example, the coordinates of a symbol may include upper left coordinates, lower left coordinates, upper right coordinates, and lower right coordinates for one Symbol.

26 is a diagram for illustratively explaining a method of determining a plurality of symbols included in an image as at least one word group.

Referring to FIG. 26, the controller 100 obtains areas occupied in the image by words included in the image (eg, first word area WA1 and second word area WA2) by using a predetermined algorithm. can do. In addition, the controller 100 may extract a plurality of symbols (S1 to S9) included in the image using a predetermined algorithm.

The control unit 100 may determine symbols (S1 to S9) included in the first word area WA1 among the plurality of extracted symbols as the first word group, and select the second word area WA2 among the plurality of extracted symbols. Symbols included in can be determined as the second word group.

As a more specific example, if the first Symbol S1 is located in the first word area WA1 in the image, the controller 100 may determine that the first Symbol S1 is included in the first word group. .

As another example, the controller 100 may, when at least a part of the first area occupied by the first symbol in the image is included in both the first word area WA1 and the second word area WA2, the first area In consideration of the ratio of the degree to which at least a portion overlaps with the first word area WA1 and the degree to which at least a portion of the first area overlaps with the second word area WA2, the first symbol is selected as a first word group. or the second word group.

For example, the control unit 100 determines the extent to which at least a part of the first area occupied by the first symbol in the image overlaps with the first word area WA1 in the second area occupied by the first symbol in the image (at this time, the The second area may correspond to the first area, may be the same area as the first area, or may be a different area from the first area.) If it is greater than the degree of overlap, it may be determined that the first symbol is included in the first word group.

27 is a diagram for exemplarily explaining a method of classifying and determining text included in an image into a line group according to an exemplary embodiment. The controller 100 may determine a line group, which is a set of two or more word groups, by using the obtained information on one or more word groups.

Referring to FIG. 27 , the controller 100 may acquire the first word group and the second word group in the above-described method. Then, the controller 100 compares the first word group and the second word group, and determines a set of the first word group and the second word group as a line group when a predetermined condition is satisfied.

28 to 31 are diagrams for explaining conditions for determining a plurality of word groups as a line group by way of example. Hereinafter, conditions for determining a plurality of word groups as a line group will be described with reference to each drawing. In this case, a line group may be determined only when a plurality of word groups satisfy all conditions to be described later, but is not limited thereto, and a line group when a plurality of word groups satisfy at least one of a plurality of conditions. may be determined.

Referring to FIG. 28 , the controller 100 obtains a first word group (WG1) and a second word group (WG2), and then, based on information about the area occupied by each group in the image, the first word group ( WG1) and the second word group (WG2) may be determined as a line group.

For example, the controller 100 determines the coordinates of the area occupied by the first height value d1 and the second word group WG2 in the image derived from the coordinate values of the area occupied by the first word group WG1 in the image. Based on the second height value d2 derived through the value, it may be determined whether to determine the first word group WG1 and the second word group WG2 as the line group.

As a more specific example, the controller 100 classifies the first word group WG1 and the second word group WG2 as a Line group when the first height value d1 is greater than or equal to the second height value d2. can decide

Referring to FIG. 29 , the controller 100 determines the first word group (WG1) and the second word group (WG2) by considering the angle value of the first word group (WG1) and the angle value of the second word group (WG2). It can be determined whether to determine as a Line group.

For example, the control unit 100 calculates an allowable angular range from angle information of a region occupied by the first word group WG1 in the image, and then calculates the allowable angular range and the second word group WG2 located in the image. It is possible to determine whether to determine the first word group (WG1) and the second word group (WG2) as a line group by comparing angle values.

As a more specific example, the controller 100 may calculate a first angle value from the coordinate values of the first word group WG1 and determine an allowable angle range based on the first angle value. Thereafter, the controller 100 calculates a second angle value from the coordinate values of the second word group WG2, and when the second angle value is included in the allowable angle range, the first word group WG1 and the second angle value 2 Word group (WG2) can be determined as Line group.

Referring to FIG. 30 , the controller 100 divides the first word group WG1 and the second word group WG2 into a line group based on the width of any one symbol included in the first word group WG1. You can decide whether or not to.

For example, the controller 100 calculates the width d1 of any one symbol included in the first word group WG1, and the distance value between the first word group WG1 and the second word group WG2 ( d2) can be calculated. Thereafter, the controller 100 controls the first word group WG1 and the second word group WG1 and the second word group WG2 when the distance value d2 between the first word group WG1 and the second word group WG2 is equal to or less than the width d1 of the symbol. The word group (WG2) can be determined as a line group.

Referring to FIG. 31 , the controller 100 determines the first word group WG1 and the second word group WG2 as a line group based on the MBR (Minimum Bounding Rectangle) value of the first word group WG1. can decide whether to do it or not.

For example, the controller 100 calculates a Minimum Bounding Rectangle (MBR) value of the first word group WG1, calculates a second height value of the second word group WG1, and then calculates the MBR value and the second height value of the second word group WG1. Based on the height value, it may be determined whether to determine the first word group (WG1) and the second word group (WG2) as the line group.

In this case, the MBR (Minimum Bounding Rectangle) may mean a rectangle having a minimum size capable of including all symbols included in a Word group, a Line group, or a Paragraph group.

As a more specific example, the controller 100 may calculate a Minimum Bounding Rectangle (MBR) value of the first word group WG1 using a predetermined algorithm, and determine the maximum allowable height value d1 from the MBR value. Thereafter, the controller 100 calculates the second height value d2 of the second word group WG2, and when the second height value d2 is equal to or less than the maximum allowable height value d1, the first word It is possible to determine whether to determine the group WG1 and the second word group WG2 as a line group.

32 is a diagram for exemplarily explaining a method of classifying and determining text included in an image into paragraph groups according to an exemplary embodiment. The control unit 100 may determine a paragraph group, which is a set of two or more line groups, by using the acquired information about one or more line groups. Referring to FIG. A line group is obtained, a second line group is obtained, information of the first line group and information of the second line group are compared, and when a predetermined paragraph integration condition is satisfied, the first line group and the second line group are obtained. A Line group can be determined as a Paragraph group.

33 is a diagram for illustratively explaining conditions for determining a plurality of line groups as a paragraph group. Referring to FIG. 33 , the controller 100 may obtain a first line group G1 and a second line group G2. In this case, the controller 100 compares the first line group (G1) and the second line group (G2), and when a predetermined condition is satisfied, the first line group (G1) and the second line group (G2) A set of can be determined as a Paragraph group.

The controller 100 may extract the first height value d1 derived from the coordinate value of the area occupied by the first line group G1 in the image, and the second line group G2 occupied in the image. A second height value derived through the coordinate values of the region may be extracted.

The controller 100 may calculate a distance value d2 between the first line group G1 and the second line group G2. At this time, the distance value may be the distance between the first line group G1 and the second line group G2, but is not limited thereto, and the MBR of the first line group G1 and the second line group G2 ) may be a value determined based on the MBR of

When the distance value d2 between the first line group G1 and the second line group G2 is equal to or less than the first height value d1 of the first line group G1, the control unit 100 determines the first The line group G1 and the second line group G2 may be determined as one paragraph group.

The features, structures, effects, etc. described in the embodiments above are included in at least one embodiment of the present invention, and are not necessarily limited to only one embodiment. Furthermore, the features, structures, effects, etc. illustrated in each embodiment can be combined or modified with respect to other embodiments by those skilled in the art in the field to which the embodiments belong. Therefore, contents related to these combinations and variations should be construed as being included in the scope of the present invention.

In addition, although the embodiment has been described above, this is only an example and does not limit the present invention, and those skilled in the art to the present invention pertain to the above to the extent that does not deviate from the essential characteristics of the present embodiment. It will be appreciated that various modifications and applications not exemplified are possible. That is, each component specifically shown in the embodiment can be implemented by modifying it. And differences related to these modifications and applications should be construed as being included in the scope of the present invention as defined in the appended claims.

Claims (18)

a storage unit for storing an image including text; and
Including; a control unit for extracting text included in the image;
The control unit,
Classifying and extracting text included in the image into at least one group,
Obtaining symbol information on a plurality of symbols included in the image through a first algorithm, wherein the plurality of symbols are individual unit characters constituting the text;
obtaining text area information about a text area in the image through a second algorithm, wherein the text area is an area corresponding to the text in the image;
Classifying text included in the image into at least one group based on the symbol information and the text area information;
A device for extracting and grouping text contained within images.
According to claim 1,
The symbol information includes at least one of the type, list, number, coordinates, and location of a plurality of symbols included in the image.
A device for extracting and grouping text contained within images.
According to claim 1,
The first algorithm and the second algorithm are different algorithms,
A device for extracting and grouping text contained within images.
According to claim 1,
The control unit,
From the image, word area information, which is an area corresponding to a word included in the image, is obtained through the second algorithm, and the word is a region of the symbol that satisfies a predetermined condition. Set Im -,
determining at least one or more of the plurality of symbols as a word group based on the symbol information and the word region information;
A device for extracting and grouping text contained within images.
According to claim 4,
The control unit,
Determine at least one or more of the plurality of symbols as a word group based on positions or coordinates occupied by the plurality of symbols included in the symbol information in the image;
Determining at least one symbol located in the word area among the plurality of symbols as a word group,
A device for extracting and grouping text contained within images.
According to claim 4,
The symbol information includes information on coordinates at which a first symbol, which is any one of the plurality of symbols, is located in the image, wherein the coordinates include a first coordinate that is an upper left coordinate of the first symbol and a lower left corner of the first symbol. Including a second coordinate that is a coordinate, a third coordinate that is an upper right coordinate, and a fourth coordinate that is a lower right coordinate,
The control unit,
Determining the first symbol as a word group when any one of the first to fourth coordinates is located in the word area,
A device for extracting and grouping text contained within images.
According to claim 4,
The word area includes a first word area and a second word area,
The control unit,
determining a symbol located in the first word area among the plurality of symbols as a first word group, and determining a symbol located in the second word area as a second word group;
A device for extracting and grouping text contained within images.
According to claim 7,
The control unit,
When a first symbol, which is any one of the plurality of symbols, is included in both the first word area and the second word area, the area occupied by the first symbol in the image is the first word area and the second word area. determining the first symbol as a first word group when the degree of overlap is greater than the degree of overlap with the second word region;
A device for extracting and grouping text contained within images.
According to claim 4,
The control unit,
Based on the symbol information and the word region information, at least one of the plurality of symbols is determined as a word group, the word group including a first word group and a second word group;
When a predetermined condition is satisfied, the first word group and the second word group are determined as a line group;
The line group refers to a set of words that satisfy a predetermined condition.
A device for extracting and grouping text contained within images.
According to claim 9,
The control unit,
A first height value of an area occupied by the first word group in the image is compared with a second height value of an area occupied by the second word group in the image, and the first word group and the second height value are compared. determining whether to determine a word group as the line group;
A device for extracting and grouping text contained within images
According to claim 9,
The control unit,
Calculating a first angle value related to the arrangement of the first word group in the image from the coordinate values of the first word group;
After determining the allowable angle range based on the first angle value,
After calculating a second angle value related to the arrangement of the second word group in the image from the coordinate value of the second word group,
determining the first word group and the second word group as the line group when the second angle value is included in the allowable angle range;
A device for extracting and grouping text contained within images.
According to claim 9,
The control unit,
determining whether to determine the first word group and the second word group as the line group based on the width of any one symbol included in the first word group;
A device for extracting and grouping text contained within images.
According to claim 12,
The control unit,
Calculate a width of a first symbol, which is any one symbol included in the first word group, and calculate a spacing value between the first word group and the second word group,
determining the first word group and the second word group as the line group when an interval value between the first word group and the second word group is less than or equal to a width of the first symbol;
A device for extracting and grouping text contained within images.
According to claim 9,
The control unit,
Calculate a Minimum Bounding Rectangle (MBR) value of the first word group, wherein the Minimum Bounding Rectangle (MBR) is a rectangle having a minimum size that can include all symbols included in the first word group;
After determining the maximum allowable height value based on the MBR value,
determining the first word group and the second word group as the line group when a height value of an area occupied by the second word group in the image is less than or equal to the maximum allowable height value;
A device for extracting and grouping text contained within images.
According to claim 9,
The control unit,
When a predetermined condition is satisfied, the first word group and the second word group are determined as a line group, the line group including a first line group and a second line group;
determining the first line group and the second line group as a paragraph group when a predetermined condition is satisfied;
A device for extracting and grouping text contained within images.
According to claim 15,
The control unit,
A first height value of an area occupied by the first line group in the image is compared with a second height value of an area occupied by the second line group in the image, and the first line group and the second height value are compared. determining whether to determine a line group as the paragraph group;
A device for extracting and grouping text contained within images.
According to claim 15,
The control unit,
Calculate a first height value for an area occupied by the first line group in the image, and calculate a distance value between the first line group and the second line group,
determining the first line group and the second line group as the paragraph group when a distance value between the first line group and the second line group is less than or equal to the first height value;
A device for extracting and grouping text contained within images.
A method for extracting text included in an image,
obtaining an image, wherein the image is an image including text;
obtaining symbol information on a plurality of symbols included in the image through a first algorithm, wherein the plurality of symbols are individual unit characters constituting the text;
acquiring text area information about a text area in the image through a second algorithm, wherein the text area corresponds to the text in the image;
classifying text included in the image into at least one group based on the symbol information and the text area information; and
Classifying and extracting text included in the image into the at least one or more groups;
How to extract and group text contained within images.

Images