KR20240025766A - Method and system for recognizing document that including table structure - Google Patents

Abstract

A document recognition method including a table structure according to an embodiment of the present invention is a method in which a document recognition application executed by at least one processor of a terminal recognizes a document including a table structure, and includes a predetermined table structure. Obtaining a table document image; Obtaining type information specifying the type of table structure in the obtained table document shooting video; performing a customized table item detection process according to the obtained type information; Obtaining table structure analysis information including boundary and item data for the table structure based on the performed customized table item detection process; Obtaining character recognition information for each acquired item; and providing table structuring information in which the table structure is structured in the form of a text file based on the obtained character recognition information and the table structure analysis information.

Description

Document recognition method and system including table structure {METHOD AND SYSTEM FOR RECOGNIZING DOCUMENT THAT INCLUDING TABLE STRUCTURE}

The present invention relates to a document recognition method and system including a table structure. More specifically, it relates to a method and system for automatically recognizing and digitizing a physical document including a predetermined table structure in a manner specialized for the table structure.

As we enter the digital age, document processing automation solutions that automate document-based tasks are increasing.

Here, the document processing automation may refer to a process of performing optical character recognition (OCR) based on a predetermined image and digitizing the result.

Specifically, in existing document processing, people directly view documents in the form of printed documents or image files and manually enter them into the system.

At this time, if an error occurs, such as incorrectly entering content in another item or entering incorrect information due to a typing mistake, time and financial damage may occur.

The document processing automation recognizes characters in imaged documents through optical character recognition (OCR) instead of manually working with people, and automatically inputs the characters into a digital system.

Through such document processing automation, the fatigue of workers performing the relevant work can be reduced and work efficiency can be increased by reducing mistakes that occur when people directly process documents.

However, when only optical character recognition (OCR) is applied to the corresponding document image, the characters are recognized well, but there is a problem that it is difficult to apply to actual document processing automation because the structure of the document is not considered.

In particular, documents containing tables, which are often used in business, contain characters representing specific titles or contents in each item within the table. If optical character recognition (OCR) is applied as is, these items can be distinguished from each other. The problem is that the information that can be obtained is ignored.

If the structure or form of the relevant document is unified, document processing automation based on the unique location of each item can be implemented. However, if the format of various documents needs to be processed (for example, invoice documents issued by various companies need to be processed) In this case, there is a limitation that the location of each item is different for each document type, making it difficult to automate document processing based on this.

In addition, there are various ways to distinguish each of the above items depending on the type of document (e.g., a format that separates all items with line segments, a format that separates rows with horizontal lines and columns with spaces, or characters that indicate a predetermined title and content). In order to accurately detect each item area of the corresponding table (a format that separates items into a single box, etc.), it is necessary to consider these various cases.

KR 0259804 B1

The present invention was created to solve the problems described above, and its purpose is to provide a method and system for automatically recognizing and digitizing a physical document including a predetermined table structure in a manner specialized for the table structure.

In detail, an embodiment of the present invention seeks to implement a method and system for performing automatic recognition optimized for the table structure based on the arrangement form according to vertical and horizontal lines and/or spacing between characters.

Additionally, in an embodiment of the present invention, a method and system for providing a structured text file based on the automatically recognized table structure are intended to be implemented.

However, the technical problems to be achieved by the present invention and embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

A document recognition method including a table structure according to an embodiment of the present invention is a method in which a document recognition application executed by at least one processor of a terminal recognizes a document including a table structure, and includes a predetermined table structure. Obtaining a table document image; Obtaining type information specifying the type of table structure in the obtained table document shooting video; performing a customized table item detection process according to the obtained type information; Obtaining table structure analysis information including boundary and item data for the table structure based on the performed customized table item detection process; Obtaining character recognition information for each acquired item; and providing table structuring information in which the table structure is structured in the form of a text file based on the obtained character recognition information and the table structure analysis information.

At this time, the type information is either a line segment table type that divides at least one item included in the table structure into line segments, or a blank table type that divides the at least one item into blank space or blank space and line segments. Includes.

In addition, the step of performing a customized table item detection process according to the type information includes performing a line segment-based table item detection process of detecting the table structure analysis information based on vertical and horizontal lines if the type information is the line segment table type. and, if the type information is the blank table type, performing a structure analysis-based table item detection process of detecting the table structure analysis information based on a character array type.

Additionally, performing the line segment-based table item detection process includes detecting the table structure analysis information based on at least one of a predetermined line segment detection algorithm and a predetermined vertex detection algorithm.

Additionally, performing the structure analysis-based table item detection process includes detecting the table structure analysis information based on a predetermined binarization algorithm.

In addition, the step of detecting the table structure analysis information based on the predetermined binarization algorithm includes determining whether a character output pixel exists within a certain first pixel distance in the vertical and horizontal directions with respect to a predetermined pixel in the table structure captured image. It includes a step of determining and, if it is determined that the character output pixel exists, performing a first binarization of converting the pixel value of the pixel to 255.

In addition, the step of detecting the table structure analysis information based on the predetermined binarization algorithm further includes detecting the entire area displaying the table structure based on the table structure captured image on which the first binarization has been performed. do.

In addition, the step of detecting the table structure analysis information based on the predetermined binarization algorithm includes the step of detecting the table structure analysis information within the first pixel distance in the horizontal direction with respect to a predetermined pixel in an area corresponding to the entire area within the table structure captured image. A step of determining whether a text output pixel exists, performing a second binarization of converting the pixel value of the pixel to 255 if it is determined that the text output pixel exists, and based on the corresponding area on which the second binarization was performed. It further includes detecting the table structure analysis information.

In addition, performing the structural analysis-based table item detection process further includes dividing columns or rows of the table structure based on a predetermined reference point and reference line.

Additionally, the step of acquiring character recognition information for each table item includes acquiring the character recognition information based on optical character recognition (OCR) based on the image for each table item.

Meanwhile, a document recognition system including a table structure according to an embodiment of the present invention includes at least one memory storing a document recognition application; and at least one processor that reads the document recognition application stored in the memory to recognize a document including a table structure, wherein the command of the document recognition application acquires a captured image of a table document including a predetermined table structure. A step of obtaining type information that specifies the type of table structure in the obtained table document shooting video, performing a customized table item detection process according to the obtained type information, and Performing a customized table item detection process Obtaining table structure analysis information including boundary and item data for the table structure based on an item detection process, acquiring character recognition information for each obtained item, the obtained character recognition information and It includes a command for performing the step of providing table structuring information in which the table structure is structured in the form of a text file based on table structure analysis information.

At this time, the command of the document recognition application includes performing a line segment-based table item detection process of detecting the table structure analysis information based on vertical and horizontal lines if the type information is a line segment table type, and if the type information is a blank table type. If it is a type, it further includes a command for performing a structure analysis-based table item detection process for detecting the table structure analysis information based on the character array type.

The document recognition method and system including a table structure according to an embodiment of the present invention automatically recognize and digitize a physical document including a predetermined table structure in a manner specialized for the table structure, thereby identifying what type of document the physical document is. Even if a table structure is included, there is an effect of implementing document processing automation that accurately reflects the structural characteristics of the table structure.

In addition, the method and system for recognizing a document including a table structure according to an embodiment of the present invention perform automatic recognition optimized for the table structure based on the arrangement form according to vertical and horizontal lines and/or spacing between characters, Information that can be obtained by distinguishing each item included in various types of table structures can be obtained clearly and easily, and it enables digitization while maintaining the structural characteristics of the table structure intact, enabling document processing automation. It has the effect of improving performance and quality.

In addition, the document recognition method and system including a table structure according to an embodiment of the present invention provide a predetermined structured text file based on the automatically recognized table structure as described above, thereby improving work efficiency associated with document processing automation. It has the effect of increasing quality and reliability, while also improving the quality of various services that utilize it (for example, a certain database construction service, etc.).

However, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood from the description below.

1 is a conceptual diagram of a document recognition system including a table structure according to an embodiment of the present invention.
Figure 2 is an internal block diagram of a terminal according to an embodiment of the present invention.
Figure 3 is a flowchart illustrating a method for recognizing a document including a table structure according to an embodiment of the present invention.
Figure 4 is an example of a drawing for explaining type information of a table structure according to an embodiment of the present invention.
Figure 5 is an example of a diagram for explaining a line segment-based table item detection process according to an embodiment of the present invention.
Figure 6 is an example of a diagram for explaining a structure analysis-based table item detection process according to an embodiment of the present invention.
Figure 7 is a flowchart illustrating a reference point-based column classification process according to an embodiment of the present invention.
Figure 8 is an example of a diagram for explaining a reference point-based column classification process according to an embodiment of the present invention.

Since the present invention can be modified in various ways and can have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first and second are used not in a limiting sense but for the purpose of distinguishing one component from another component. Additionally, singular expressions include plural expressions unless the context clearly dictates otherwise. Additionally, terms such as include or have mean that the features or components described in the specification exist, and do not preclude the possibility of adding one or more other features or components. Additionally, in the drawings, the sizes of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are shown arbitrarily for convenience of explanation, so the present invention is not necessarily limited to what is shown.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. When describing with reference to the drawings, identical or corresponding components will be assigned the same reference numerals and redundant description thereof will be omitted. .

1 is a conceptual diagram of a document recognition system including a table structure according to an embodiment of the present invention.

Referring to FIG. 1, a document recognition system 1000 (hereinafter referred to as a table document recognition system) including a table structure according to an embodiment of the present invention, A table document recognition service that automatically recognizes and digitizes a physical document containing a predetermined table structure in a manner specialized for the table structure can be provided.

In an embodiment, the table document recognition system 1000 that implements the above table document recognition service may include a terminal 100, a document recognition server 200, and a network 300.

At this time, the terminal 100 and the document recognition server 200 may be connected through the network 300.

Here, the network 300 according to the embodiment refers to a connection structure capable of exchanging information between each node, such as the terminal 100 and/or the document recognition server 200. This network 300 Examples include the 3rd Generation Partnership Project (3GPP) network, Long Term Evolution (LTE) network, World Interoperability for Microwave Access (WIMAX) network, Internet, Local Area Network (LAN), and Wireless Local Area Network (Wireless LAN). , WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc., but are not limited thereto.

Hereinafter, the terminal 100 and document recognition server 200 that implement the table document recognition system 1000 will be described in detail with reference to the attached drawings.

- Terminal (100: Terminal)

The terminal 100 according to an embodiment of the present invention may be a computing device installed with a document recognition application (hereinafter referred to as application) that provides a table document recognition service.

In detail, from a hardware perspective, the terminal 100 may include a mobile type computing device 100-1 and/or a desktop type computing device 100-2 on which an application is installed.

Here, the mobile type computing device 100-1 may be a mobile device such as a smart phone or tablet PC with an application installed.

For example, the mobile type computing device 100-1 includes smart phones, mobile phones, digital broadcasting devices, personal digital assistants (PDAs), portable multimedia players (PMPs), and tablet PCs. may be included.

In addition, the desktop type computing device 100-2 provides a table document recognition service based on wired/wireless communication, such as a fixed desktop PC with an application installed, a laptop computer, or a personal computer such as an ultrabook. It may include devices where programs to run are installed.

Additionally, depending on the embodiment, the terminal 100 may further include a server computing device that provides a table document recognition service environment.

Figure 2 is an internal block diagram of the terminal 100 according to an embodiment of the present invention.

Meanwhile, referring to FIG. 2, from a functional perspective, the terminal 100 includes a memory 110, a processor assembly 120, a communication processor 130, an interface unit 140, an input system 150, and a sensor system 160. ) and a display system 170. These components may be configured to be included within the housing of the terminal 100.

In detail, the application 111 is stored in the memory 110 , and the application 111 may store one or more of various applications, data, and commands for providing a table document recognition service environment.

That is, the memory 110 can store commands and data that can be used to create a table document recognition service environment.

Additionally, the memory 110 may include a program area and a data area.

Here, the program area according to the embodiment may be linked between the operating system (OS) and functional elements that boot the terminal 100, and the data area is data generated according to the use of the terminal 100. can be saved.

Additionally, the memory 110 may include at least one non-transitory computer-readable storage medium and a temporary computer-readable storage medium.

For example, the memory 110 may be a variety of storage devices such as ROM, EPROM, flash drive, hard drive, etc., and web storage that performs the storage function of the memory 110 on the Internet. may include.

The processor assembly 120 may include at least one processor capable of executing instructions of the application 111 stored in the memory 110 to perform various tasks for creating a table document recognition service environment.

In an embodiment, the processor assembly 120 may control the overall operation of components through the application 111 of the memory 110 to provide a table document recognition service.

This processor assembly 120 may be a system-on-chip (SOC) suitable for the terminal 100 that includes a central processing unit (CPU) and/or a graphics processing unit (GPU), and the operating system stored in the memory 110. (OS) and/or application programs can be executed, and each component mounted on the terminal 100 can be controlled.

Additionally, the processor assembly 120 may internally communicate with each component through a system bus and may include one or more bus structures, including a local bus.

In addition, the processor assembly 120 includes application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and controllers. ), micro-controllers, microprocessors, and other electrical units for performing functions.

The communication processor 130 may include one or more devices for communicating with external devices. This communication processor 130 can communicate through a wireless network.

In detail, the communication processor 130 can communicate with the terminal 100 that stores content sources for implementing a table document recognition service environment, and can communicate with various user input components such as a controller that receives user input.

In an embodiment, the communication processor 130 may transmit and receive various data related to the table document recognition service to another terminal 100 and/or an external server.

This communication processor 130 uses technical standards or communication methods for mobile communication (e.g., Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), 5G New Radio (NR), WIFI). Alternatively, data can be transmitted and received wirelessly with at least one of a base station, an external terminal 100, and an arbitrary server on a mobile communication network established through a communication device capable of performing short-distance communication.

The sensor system 160 may include various sensors such as an image sensor 161, a position sensor (IMU) 163, an audio sensor 165, a distance sensor, a proximity sensor, and a contact sensor.

Here, the image sensor 161 can capture images (that is, images and/or videos, etc.) of the physical space around the terminal 100.

In an embodiment, the image sensor 161 may capture and acquire various images (eg, a table document photographed image, etc.) related to the table document recognition service.

In addition, the image sensor 161 is disposed on the front or/and rear of the terminal 100 and can acquire an image by photographing the direction in which the terminal 100 is placed. You can photograph space.

This image sensor 161 may include an image sensor device and an image processing module. In detail, the image sensor 161 can process still images or moving images obtained by an image sensor device (eg, CMOS or CCD).

Additionally, the image sensor 161 may use an image processing module to process still images or moving images obtained through the image sensor device to extract necessary information and transmit the extracted information to the processor.

This image sensor 161 may be a camera assembly including at least one camera. The camera assembly may include a general camera that photographs a visible light band, and may further include a special camera such as an infrared camera or a stereo camera.

In addition, the image sensor 161 as described above may be included and operated in the terminal 100, depending on the embodiment, or may be included in an external device (e.g., an external server, etc.) and may be included in the above-described communication processor 130 and/or Alternatively, it may operate through linkage based on the interface unit 140.

The position sensor (IMU, 163) can detect at least one of the movement and acceleration of the terminal 100. For example, it may consist of a combination of various position sensors such as an accelerometer, gyroscope, or magnetometer.

Additionally, the location sensor (IMU) 163 may recognize spatial information about the physical space around the terminal 100 in conjunction with the location communication processor 130, such as the GPS of the communication processor 130.

The audio sensor 165 can recognize sounds around the terminal 100.

In detail, the audio sensor 165 may include a microphone capable of detecting voice input from a user using the terminal 100.

In an embodiment, the audio sensor 165 may receive voice data required for a table document recognition service from the user.

The interface unit 140 can connect the terminal 100 to communicate with one or more other devices. In detail, the interface unit 140 may include a wired and/or wireless communication device compatible with one or more different communication protocols.

Through this interface unit 140, the terminal 100 can be connected to various input/output devices.

For example, the interface unit 140 may be connected to an audio output device such as a headset port or a speaker to output audio.

As an example, the audio output device is connected through the interface unit 140, but embodiments in which the audio output device is installed inside the terminal 100 may also be included.

Additionally, for example, the interface unit 140 may be connected to an input device such as a keyboard and/or mouse to obtain user input.

This interface unit 140 connects devices equipped with a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, and an identification module. Port, audio I/O (Input/Output) port, video I/O (Input/Output) port, earphone port, power amplifier, RF circuit, transceiver and other communication circuits It may be configured to include at least one of:

The input system 150 may detect user input (e.g., a gesture, voice command, button operation, or other type of input) related to the table document recognition service.

In detail, the input system 150 may include a predetermined button, a touch sensor, and/or an image sensor 161 that receives a user motion input.

Additionally, the input system 150 is connected to an external controller through the interface unit 140 and can receive user input.

The display system 170 can output various information related to the table document recognition service as a graphic image.

In an embodiment, the display system 170 may display various user interfaces for table document recognition services, table document images, table structure analysis information, character recognition information, and/or table structure information.

These displays include liquid crystal display (LCD), thin film transistor-liquid crystal display (TFT LCD), organic light-emitting diode (OLED), and flexible display. , a 3D display, or an e-ink display.

The above components may be disposed within the housing of the terminal 100, and the user interface may include a touch sensor 173 on the display 171 configured to receive user touch input.

In detail, the display system 170 may include a display 171 that outputs an image and a touch sensor 173 that detects a user's touch input.

For example, the display 171 may be implemented as a touch screen by forming a mutual layer structure or being integrated with the touch sensor 173. This touch screen can function as a user input unit that provides an input interface between the terminal 100 and the user, and at the same time, can provide an output interface between the terminal 100 and the user.

Meanwhile, depending on the embodiment, the terminal 100 may further perform at least part of the functional operations performed by the document recognition server 200, which will be described later.

- Document recognition server (200: Character recognition server)

The document recognition server 200 according to an embodiment of the present invention can perform a series of processes to provide a table document recognition service.

In detail, in the embodiment, the document recognition server 200 provides the table document recognition service by exchanging data necessary to drive the table document recognition process in an external device such as the terminal 100 with the external device. You can.

In more detail, in the embodiment, the document recognition server 200 uses the application 111 in an external device (in the embodiment, a mobile type computing device 100-1 and/or a desktop type computing device 100-2, etc.). An environment in which this can operate can be provided.

To this end, the document recognition server 200 may include an application program, data, and/or commands for operating the application 111, and may transmit and receive various data based thereon with the external device.

Additionally, in the embodiment, the document recognition server 200 may acquire a captured image of a predetermined table document.

Here, the table document captured image according to the embodiment may mean an image captured of an actual document including a predetermined table structure.

Additionally, in the embodiment, the document recognition server 200 may obtain table structure type information based on the acquired table document photographed image.

Here, the type information according to the embodiment may be information that specifies the type of the table format according to the method in which the table structure is implemented, that is, the method of disclosing the shape or form of the table structure.

Additionally, in the embodiment, the document recognition server 200 may perform a customized table item detection process according to the obtained table structure type information.

Here, the customized table item detection process according to the embodiment detects each of the boundary area of the table structure and at least one table item included in the table structure in a manner optimized for the table structure type information obtained as above. It can mean a process that does.

Additionally, in the embodiment, the document recognition server 200 may obtain character recognition information based on the detected table item.

Here, the character recognition information according to the embodiment may be information obtained through optical character recognition of characters included in a predetermined table item.

Additionally, in the embodiment, the document recognition server 200 may provide table structure information based on the obtained character recognition information.

Here, the table structured information according to the embodiment may mean information converted from a predetermined table structure into a structured text file format according to a preset method (eg, CSV or JSON format, etc.).

Additionally, in the embodiment, the document recognition server 200 may store and manage various application programs, commands, and/or data for implementing a table document recognition service.

In an embodiment, the document recognition server 200 includes at least one table document captured video, table structure type information, table structure analysis information, reference point information, baseline information, character recognition information, table structuring information, and various data processing algorithms (implementation). For example, customized table item detection algorithms, etc.) and/or various user interfaces, etc. may be stored and managed.

However, the functional operations that the document recognition server 200 can perform in the embodiment of the present invention are not limited to those described above, and other functional operations may also be performed.

Meanwhile, referring further to FIG. 1, in the embodiment, the document recognition server 200 as described above includes at least one processor module 210 for data processing and at least one processor module 210 for data exchange with an external device. It is a predetermined computing device including the above communication module (220) and at least one memory module (230) that stores various applications, data, and/or commands for providing a table document recognition service. It can be implemented.

Here, the memory module 230 may store one or more of an operating system (OS), various application programs, data, and commands for providing a table document recognition service.

Additionally, the memory module 230 may include a program area and a data area.

Here, the program area according to the embodiment may be linked between the operating system (OS) that boots the server and functional elements, and the data area may store data generated according to the use of the server.

In an embodiment, the memory module 230 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and may be a web device that performs the storage function of the memory module 230 on the Internet. It could also be storage (web storage).

Additionally, the memory module 230 may be a recording medium that is removable from the server.

Meanwhile, the processor module 210 can control the overall operation of each unit described above in order to implement a table document recognition service.

This processor module 210 may be a system-on-chip (SOC) suitable for a server that includes a central processing unit (CPU) and/or graphics processing unit (GPU), and the operating system (OS) stored in the memory module 230 ) and/or application programs, etc., and control each component mounted on the server.

Additionally, the processor module 210 may internally communicate with each component through a system bus and may include one or more bus structures, including a local bus.

In addition, the processor module 210 includes application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and controllers. ), micro-controllers, microprocessors, and other electrical units for performing functions.

In the above description, it has been described that the document recognition server 200 according to the embodiment of the present invention performs the functional operations as described above. However, depending on the embodiment, at least part of the functional operation performed by the document recognition server 200 is externally performed. Various embodiments may be possible, such as performing the function in the device (e.g., terminal 100, etc.), and at least part of the functional operation performed in the external device may be further performed in the document recognition server 200. .

- Document recognition method including table structure

Hereinafter, a method for the application 111 executed by at least one processor of the terminal 100 according to an embodiment of the present invention to recognize a document including a table structure will be described in detail with reference to FIGS. 3 to 8 attached. do.

In an embodiment of the present invention, at least one processor of the terminal 100 may execute at least one application 111 stored in at least one memory 110 or operate in a background state.

Hereinafter, the method of providing the above-described table document recognition service by operating the at least one processor to execute instructions of the application 111 will be briefly described as being performed by the application 111.

Figure 3 is a flowchart illustrating a method for recognizing a document including a table structure according to an embodiment of the present invention.

Referring to FIG. 3, in the embodiment, the application 111 executed by at least one processor of the terminal 100 or operating in the background may acquire a table document captured image. (S101)

Here, the table document captured image according to the embodiment may mean an image captured of an actual document (hereinafter referred to as a table document) including a predetermined table structure.

In an embodiment, the table document may include at least one table structure implemented based on various different shapes or styles.

For example, the table structure includes the position (coordinates), size, boundary determination method with other items, and/or the category and number of internal text (e.g., title category, It can be implemented in a variety of different forms depending on the content category (content category or title category + content category, etc.).

In an embodiment, the application 111 captures a table document captured image in conjunction with the image sensor 161 and/or an external device (e.g., document recognition server 200, etc.). It can be obtained.

Additionally, in the embodiment, the application 111 may obtain table structure type information based on the acquired table document photographed image. (S103)

That is, in the embodiment, the application 111 may obtain type information for at least one table structure included in the acquired table document captured image.

Figure 4 is an example of a drawing for explaining type information of a table structure according to an embodiment of the present invention.

Here, referring to FIG. 4, the type information according to the embodiment is information that specifies the type of the table format according to the manner in which a given table structure is implemented, that is, the manner in which the shape or form of the table structure is disclosed. It can be.

In an embodiment, this type information may be a line segment table type that divides at least one item (hereinafter referred to as table item) included in the corresponding table structure into line segments, or the at least one table item is divided based on a space or a space and a line segment. It can include separator blank table types, etc.

In detail, in the embodiment, the application 111 may obtain a user input for setting type information for each table structure in the acquired table document captured image.

Accordingly, the application 111 may determine type information for each table structure in the table document captured image based on the obtained user input.

In another embodiment, the application 111 may perform optical character recognition (OCR) based on the acquired table structure captured image.

Additionally, in this embodiment, the application 111 can detect an area (hereinafter referred to as a character distribution area) where the positional coordinates of each character detected based on the optical character recognition (OCR) are distributed according to a predetermined distribution chart. .

And in this embodiment, the application 111 can determine whether a predetermined line segment exists on the area between each character (hereinafter, the gap area) within the detected character distribution area.

Additionally, in this embodiment, if a predetermined line segment exists on all spacing areas within the character distribution area, the application 111 may determine the type information for the table structure on the character distribution area as a line segment table type.

On the other hand, in this embodiment, if a predetermined line segment does not exist on at least part of the spacing area within the character distribution area, the application 111 may determine the type information for the table structure on the character distribution area as a blank table type. .

That is, depending on the embodiment, the application 111 may automatically obtain type information for each table structure in the table structure captured image using a simple OCR-based algorithm even without separate user input.

Additionally, in the embodiment, the application 111 may perform a customized table item detection process according to the obtained table structure type information. (S105)

Here, the customized table item detection process according to the embodiment refers to the boundary area of the table structure (hereinafter referred to as It may refer to a process of distinguishing and detecting table boundaries) and at least one table item included in the corresponding table structure.

At this time, depending on the embodiment, each of the detected table items may be matched with location coordinates for an area corresponding to each table item.

In an embodiment, this customized table item detection process may include a line segment-based table item detection process (hereinafter referred to as a line segment table detection process), which is executed if the table structure type information is a line segment table type, and/or if the table structure type information is a blank table type. It may include a structural analysis-based table item detection process (hereinafter referred to as a blank table detection process) that is executed in the background.

Figure 5 is an example of a diagram for explaining a line segment-based table item detection process according to an embodiment of the present invention.

In detail, referring to FIG. 5, in the embodiment, if the table structure type information is a line segment table type, the application 111 determines the above-described table boundary and each table item (hereinafter referred to as table structure analysis information) based on vertical and horizontal lines. ) can be executed.

Referring to (a) of FIG. 5, in more detail, in the embodiment, the application 111 may: 1) obtain table boundary data for the corresponding table structure based on a predetermined line segment detection algorithm and/or vertex detection algorithm; there is.

In detail, in an embodiment, the application 111 may perform image analysis on the table-structured captured image using the predetermined line segment detection algorithm and/or vertex detection algorithm.

And in the embodiment, the application 111 may detect the outline and/or vertex of the corresponding table structure based on the results of the image analysis performed above.

Illustratively, the application 111 detects the outline of the table structure in the table structure captured image using the Canny edge detection algorithm and based on the Douglas-Peucker Algorithm. Vertices forming the outline can be detected.

Additionally, in the embodiment, the application 111 may obtain table boundary data for the table structure based on the detected outline and/or vertex.

In an embodiment, the application 111 may obtain table boundary data forming each boundary area based on the detected outlines and/or line segments (i.e., vertical and horizontal lines) forming the vertices.

Referring to (b) of FIG. 5, in the embodiment, the application 111 2) creates a corresponding table structure based on the outline and/or vertex detected based on the line segment detection algorithm and/or vertex detection algorithm. Data for each table item included can be obtained.

In detail, in the embodiment, when there are four detected vertices for the detected outline as described above, the application 111 uses the corresponding outline (hereinafter referred to as a square box outline) as a separator for each table item included in the table structure. You can set it.

And in an embodiment, the application 111 may obtain table item data for the table structure based on the area included in each of the set table item separators, that is, a square box area with the table item separator as an outline.

That is, in the embodiment, the application 111 may obtain the table item data by setting each square box area with the table item separator as an outline as a table item area of the corresponding table structure.

Accordingly, the application 111 implements analysis information about a table structure having a line segment table type (i.e., boundary and item data for the corresponding table structure in the embodiment) within a predetermined table structure captured image with relatively little data processing. It can be obtained quickly and accurately by using the algorithm that is used.

At this time, in the embodiment, the application 111 may determine the inclusion relationship between the square box areas in order to avoid duplicate detection between each table item.

And in the embodiment, the application 111 may search for an area containing at least two or more square box areas (hereinafter referred to as an overlapping box area) as a result of the determination of the inclusion relationship.

Additionally, in the embodiment, the application 111 may exclude the outline of a square box forming the identified overlapping box area from the table item separator.

That is, in the embodiment, the application 111 can distinguish and detect each table item in the corresponding table structure without duplication by excluding an area containing at least two or more table items from the table item data.

Returning again, also in the embodiment, the application 111 may 3) generate the table structure analysis information based on the table boundary and table item data obtained as described above.

As such, in the embodiment, when the table structure type information is a line segment table type, the application 111 may perform a line segment table detection process to detect table structure analysis information for the table structure according to the above-described process.

Figure 6 is an example of a diagram for explaining a structure analysis-based table item detection process according to an embodiment of the present invention.

Meanwhile, referring to FIG. 6, in the embodiment, if the table structure type information is a blank table type, the application 111 detects the table structure analysis information based on the character arrangement type based on the spacing between characters. The process can be executed.

In detail, in the embodiment, the application 111 may: 1) obtain each table item data included in the table structure based on a predetermined binarization algorithm.

In more detail, in the embodiment, the application 111 may perform binarization on the table-structured captured image using a predetermined binarization algorithm such as a Run-Length Smoothing Algorithm (RLSA).

For reference, the run length smoothing algorithm refers to a pixel that outputs a predetermined character within a random distance L pixel distance in the vertical and/or horizontal direction with respect to a white pixel in a predetermined image (in the embodiment, a table structure captured image). (i.e., a pixel displaying a color other than white, etc.) is determined to exist, and if it is determined that a pixel that outputs a certain character (hereinafter referred to as a character output pixel) exists within the L pixel distance, the pixel of the corresponding character output pixel It may be an algorithm that implements binarization by converting the pixel value to 255, that is, a white pixel.

Specifically, referring to (a) of FIG. 6, in the embodiment, the application 111, based on a predetermined binarization algorithm, divides L pixels at a random distance in the vertical and horizontal directions with respect to the white pixels in the table structure captured image. If the text output pixel exists within the distance, first binarization can be performed to convert the pixel value of the text output pixel to 255.

Additionally, in the embodiment, the application 111 may detect the entire area displaying the table structure based on the first binarized table structure captured image (hereinafter, first binarized image) as described above.

In detail, in an embodiment, the application 111 may group at least one activated pixel (i.e., a pixel whose pixel value is converted to 255) included in an adjacent area within a predetermined distance within the first binarized image into one. there is.

And in the embodiment, the application 111 may extract a rectangular box area (hereinafter referred to as total rectangular box area) based on the minimum area including all activated pixel areas grouped as above.

Additionally, in the embodiment, the application 111 may detect the entire area of the table structure based on the first binarized image by setting the extracted total square box area as the entire area occupied by the table structure.

In addition, referring to (b) of FIG. 6, in the embodiment, the application 111 creates a table structure based on the area corresponding to the entire detected table structure area (hereinafter, table corresponding area) within the table structure captured image. 2 Binarization can be performed.

In detail, in the embodiment, based on a predetermined binarization algorithm, if the text output pixel exists within a certain distance L pixel distance in the horizontal direction with respect to the white pixel in the table corresponding area, the corresponding text output pixel is selected. A second binarization can be performed to convert the pixel value of to 255.

And the application 111 may obtain each table item data included in the table structure based on the second binarized table corresponding area (hereinafter, second binarized image).

In detail, in the embodiment, the application 111 may group at least one activated pixel (i.e., a pixel whose pixel value is converted to 255) included in an adjacent area within a predetermined distance within the second binarized image into one. there is.

Additionally, in the embodiment, the application 111 may obtain the table item data by designating each activated pixel area grouped as above as each table item.

At this time, in the embodiment, the application 111 may determine whether table items included in one table structure overlap in the vertical direction.

Through this, the application 111 can detect and integrate table item elements belonging to the same row.

In detail, in the embodiment, the application 111 provides coordinate information for each of the table items located at the top (hereinafter referred to as upper bounding box) and the table items located at the bottom (hereinafter referred to as lower bounding boxes) for at least two or more table items. Based on this, it can be determined whether there is overlap in the vertical direction.

In more detail, assuming that the vertical coordinate information of the upper bounding box is (ymin(A), ymax(A)) and the vertical coordinate information of the lower bounding box is (ymin(B), ymax(B)), It can be determined whether the conditions according to [Formula 1] are met.

[Formula 1]

Additionally, in the embodiment, if the conditions according to [Equation 1] are met, the application 111 may determine that the corresponding table items overlap in the vertical direction.

Then, the application 111 can integrate the table items determined to be overlapping into one table item.

That is, in the embodiment, the application 111 may integrate the table items into one when at least some of the obtained table items overlap in the vertical direction, and through this, table items belonging to the same row can be combined with higher accuracy. It can be detected and clearly recognized.

Additionally, in the embodiment, the application 111 may 2) obtain table boundary data for the corresponding table structure based on the table item data obtained as above.

In detail, in the embodiment, the application 111 provides item arrangement information that provides row/column information of the table structure based on the arrangement type (i.e., location coordinates) of each table item according to the acquired table item data. It can be obtained.

In an embodiment, the item arrangement information may include the total number of rows/columns of the corresponding table structure and/or location coordinate information for each row/column.

In more detail, in an embodiment, the application 111 may detect at least one table item listed below and above a predetermined first horizontal line and set it as the first row group.

Additionally, the application 111 may detect at least one table item located below the second horizontal line and listed on a predetermined third horizontal line and set it as the second row group.

In the same manner, the application 111 repeats the row group setting process described above until at least one table item listed and located below the predetermined Nth horizontal line and located above the predetermined N+1th horizontal line is retrieved. It can be done.

And when the table item is issued, the application 111 can determine row information of the table structure based on the first to Nth row groups set up to that point.

Meanwhile, in the embodiment, the application 111 is located on one side of a predetermined first vertical line (hereinafter described with reference to the right side) and on the other side of a predetermined second vertical line (hereinafter described with reference to the left side) and is located at least as listed. You can detect one table item and set it as the first column group.

Additionally, the application 111 may detect at least one table item listed to the right of a second vertical line and to the left of a third vertical line and set it as the second column group.

In the same manner, the application 111 repeats the above-described column group setting process until at least one table item listed and located to the right relative to the predetermined N-th vertical line and to the left relative to the predetermined N+1-th vertical line is retrieved. It can be done.

And when the table item is issued, the application 111 can determine the column information of the table structure based on the first to Nth column groups set up to that point.

Additionally, the application 111 can generate and obtain item arrangement information for the table structure, including the row information and column information determined as above.

Subsequently, in the embodiment, the application 111 may obtain the table boundary data based on the obtained item arrangement information.

That is, the application 111 can obtain table boundary data that forms a boundary area between each corresponding table item based on row/column information according to the obtained item arrangement information.

At this time, in the embodiment, the application 111 may perform a reference point-based column classification process to more accurately distinguish table item elements belonging to the same column when deriving the above column information.

Here, the reference point-based column classification process according to the embodiment refers to a table structure based on the premise that table item elements belonging to the same column have the same alignment direction (e.g., all aligned to the left, center, right, or both sides, etc.). This can refer to the process of deriving column information about my table items.

In the following description, for effective explanation, the alignment direction will be described based on center alignment, but is not limited thereto.

FIG. 7 is a flowchart for explaining a reference point-based column classification process according to an embodiment of the present invention, and FIG. 8 is an example of a drawing for explaining a reference point-based column classification process according to an embodiment of the present invention.

In detail, referring to FIGS. 7 and 8, in the embodiment, the application 111: 1] at least one table item (hereinafter referred to as first) included in the first row (i.e., first row group) according to the row information. Row item element: FRE) can be read. (S201)

That is, the application 111 selects each of at least one table item containing a predetermined character (e.g., a word and/or sentence block, etc.) in the horizontal direction on the first row group as the first row item element (FRE). It can be obtained.

Additionally, in the embodiment, the application 111 may set a central reference point (CBP) for each of the read first row element elements (FRE). (S203)

In detail, in an embodiment, the application 111 determines the left side for a given first first row element based on the leftmost point on the bounding box (hereinafter referred to as component bounding box) representing the boundary area of the first first row element. You can set a reference point (FLP).

Additionally, in the embodiment, the application 111 may set a right reference point (FRP) for the first first row item element based on the rightmost point on the component bounding box of the first first row item element.

And in the embodiment, the application 111 sets the center reference point (CBP) of the first first row element based on the left reference point (FLP) and right reference point (FRP) set for the first first row element. You can.

In an embodiment, the application 111 determines the distance between the left reference point (FLP) and the right reference point (FRP) based on the position coordinates for each of the left reference point (FLP) and right reference point (FRP) of the first first row item element. The exact center point can be derived.

And the application 111 may set the derived exact center point as the central reference point (CBP).

Additionally, in the embodiment, the application 111 may set a reference line (hereinafter, central reference line: CBL) extending vertically downward from the 3] set central reference point (CBP). (S205)

At this time, the central reference line (CBL) may be implemented in a shape that crosses at least one table item area located on the lower area of the first first row item element.

Additionally, in the embodiment, the application 111 may 4] detect at least one dependent item element (DE) for each first row item element (FRE). (S207)

Here, the dependent item element (DE) according to the embodiment may mean at least one table item located along the vertical direction on the lower area (hereinafter, dependent area) of a predetermined first row item element (FRE). there is.

In detail, in an embodiment, the application 111 may, based on the above-described table item data, add at least one table item arranged vertically on a dependent area of the first first row item element to the first first row item element. It can be detected as a dependent element (DE).

Additionally, in the embodiment, the application 111 configures 5] a left reference point (hereinafter referred to as dependent left point: SLP) for each detected at least one dependent element element (DE) and a right reference point (hereinafter referred to as dependent left point) for each dependent element element (DE). , dependent right point: SRP) can be set. (S209)

In detail, in the embodiment, the application 111 determines the dependent left point for the first dependent element based on the leftmost point on the bounding box (hereinafter referred to as dependent bounding box) representing the boundary area of the first dependent element. (SLP) can be set.

Additionally, the application 111 may set a dependent right point (SRP) for a first dependent element based on the rightmost point on the dependent bounding box of the first dependent element.

In addition, in the embodiment, the application 111: 6] If the horizontal distance between the set dependent left point (SLP) and dependent right point (SRP) and the central reference line (CBL) is less than a predetermined distance (d), The corresponding dependent element element (DE) can be determined as a table item belonging to the same column as the first row element element (FRE). (S211)

That is, in an embodiment, the application 111 determines that the horizontal distance between the dependent left point (SLP) and dependent right point (SRP) of the first dependent item element and the central reference line (CBL) according to the first first row item element is If it is determined that the distance d is less than a predetermined distance, the first dependent item element may be determined to be a table item belonging to the same column as the first first row item element.

In this way, application 111 can create a table structure for a given table structure with a blank table type (in an embodiment, a table structure with columns separated by blanks, etc.) by implementing a reference point-based column separation process that is implemented as described above. When deriving column information, table items belonging to the same column can be more clearly distinguished and detected.

According to the embodiment, the application 111 may set a dependent center point indicating the exact center point for each of the at least one dependent item element based on the dependent left point (SLP) and the dependent right point (SRP).

Additionally, in this embodiment, the application 111 may determine whether the set dependent center point is included in the central baseline (CBL).

At this time, in this embodiment, if a predetermined dependent center point is not included on the central reference line (CBL), the application 111 operates the dependent item element (hereinafter, other sort item) that does not include the dependent center point on the central reference line (CBL). element) can be determined.

In detail, in this embodiment, the application 111 is based on at least one of the left reference point (FLP) for the first first row element and/or the right reference point (FRP) for the first first row element. 2 You can set a sub baseline.

In more detail, in this embodiment, the application 111 may include a second left sub-baseline extending vertically downward from the left reference point (FLP) and/or a second right sub-reference line extending vertically downward from the right reference point (FRP). You can set a sub baseline.

Additionally, in this embodiment, the application 111 may determine the alignment direction for the other alignment item elements based on the set second sub reference line.

In detail, in this embodiment, the application 111 may determine whether the dependent left point (SLP) of the other sorted item element is included in the second left sub reference line.

And, if the dependent left point (SLP) is included in the second left sub reference line, the application 111 may determine the alignment direction of the corresponding other aligned item element as left alignment.

In the same way, in this embodiment, the application 111 may determine whether the dependent right point (SRP) of the other sorted item element is included in the second right sub reference line.

And, if the dependent right point (SRP) is included in the second right sub reference line, the application 111 may determine the alignment direction of the corresponding other aligned item element to be right aligned.

Additionally, in this embodiment, the application 111 may determine whether the corresponding other sorted item element belongs to the same column as the first first row item element based on the sorting direction determined as above.

In detail, in this embodiment, if the other-aligned item element is determined to be left-aligned, the application 111 can calculate the horizontal distance between the second left sub reference line and the dependent right point (SRP) of the other-aligned item element. there is.

Additionally, in this embodiment, if the calculated horizontal distance is less than or equal to a predetermined distance d, the application 111 may determine the other sorted item element to be a table item belonging to the same column as the first first row item element. there is.

In the same way, in this embodiment, the application 111 calculates the horizontal distance between the second right sub reference line and the dependent left point (SLP) of the other alignment item element when the other alignment item element is determined to be right aligned. can do.

Additionally, in this embodiment, if the calculated horizontal distance is less than or equal to a predetermined distance d, the application 111 may determine the other sorted item element to be a table item belonging to the same column as the first first row item element. there is.

Meanwhile, depending on the embodiment, the application 111 may set the 2-2 left sub-baseline based on the left reference point (FLP) for the second first row element (i.e., the first row element of the second column). .

In addition, in this embodiment, the application 111 is configured to use a second left sub-baseline for the above-described first first row element (i.e., first row element of the first column) and the second first row element (i.e., first row element of the first column). Based on the 2-2 left sub-baseline for the first row element of 2 columns, the first first row element and at least one dependent element belonging to the same column as the first first row element (hereinafter, the first row element) Center alignment for 1 column item elements can be implemented.

In detail, in this embodiment, the application 111 may set a new center reference line for the first first row element based on the second left sub reference line and the 2-2 left sub reference line.

In detail, in this embodiment, the application 111 may set a new central reference line extending vertically downward from the exact center point of the second left sub-reference line and the 2-2 left sub-reference line.

In addition, in this embodiment, the application 111 determines the central reference point (CBP) of the first first row element based on the set new central baseline and at least one dependent belonging to the same column as the first first row element. Item elements can be aligned so that all dependent center points match.

In detail, in this embodiment, the application 111 may perform the center alignment by arranging the central reference point (CBP) and each of the dependent center points so that they are located on the new central reference line.

On the other hand, according to the embodiment, the application 111 may configure the second left sub-baseline for the above-described first first row element (i.e., the first row element of the first column) and the second first row element (i.e., Based on the 2-2 left sub reference line for the first row element of the second column, bilateral alignment of the first row element can be implemented.

In detail, in this embodiment, the application 111 configures the left reference point (FLP) of the first first row element on the second left sub baseline and at least one dependent belonging to the same column as the first first row element. The letter spacing and/or size for each of the first row item elements may be adjusted so that the dependent left point (SLP) of each item element is arranged.

Additionally, in this embodiment, the application 111 sets a right reference point (FRP) of the first first row element on the 2-2 left sub-baseline and at least one column belonging to the same column as the first first row element. The letter spacing and/or size for each of the first row item elements can be adjusted so that the dependent right point (SRP) of each dependent item element is arranged.

Accordingly, in this embodiment, the application 111 may perform alignment on both sides of the first column element based on the second left sub reference line and the 2-2 left sub reference line.

In the embodiment of the present invention, the method of implementing the reference point-based column classification process has been described as described above, but the above-described functional operations are applied according to the level at which a person skilled in the art can perform the reference point-based row ( A method of implementing a row) classification process may also be included in embodiments of the present invention.

Returning again, also in the embodiment, the application 111 may 3) generate the table structure analysis information based on the table items and table boundary data obtained as described above.

That is, in the embodiment, when the table structure type information is a blank table type, the application 111 may perform a blank table detection process to detect table structure analysis information for the table structure according to the above-described process.

Therefore, the application 111 collects analysis information (i.e., boundary and item data for the corresponding table structure in the embodiment) about the table structure having a blank table type within a predetermined table structure captured image with high accuracy according to a systematic algorithm. It can be obtained with

As described above, in the embodiment, the application 111 processes the table through an optimized process for each type of table structure included in the captured image of a predetermined table document (i.e., line segment table type or blank table type, etc. in the embodiment). By detecting the boundary and item data of the table structure, no matter what type of table structure the table document video contains, the boundary and item data of the table structure can be classified and detected with high accuracy to determine the structural characteristics of the table structure ( In the embodiment, row/column information, etc.) can be clearly identified, and through this, document processing based on the captured image of the table document is later performed (i.e., optical character recognition (OCR) based on the captured image of the table document is performed). The performance of the process of digitizing the results can be improved.

Additionally, in the embodiment, the application 111 may obtain character recognition information based on the detected table item. (S107)

Here, the character recognition information according to the embodiment may be information obtained through optical character recognition (OCR) on characters included in a given table item.

In detail, in the embodiment, the application 111 may recognize the detected character for each table item using a predetermined optical character recognition (OCR) engine (eg, DocTr, etc.).

In an embodiment, the application 111 may input the detected image for each table item into a predetermined optical character recognition (OCR) engine.

Then, the application 111 can obtain data (i.e., character recognition information) as a result of recognizing characters for each table item from the optical character recognition (OCR) engine that receives the image.

At this time, depending on the embodiment, the application 111 may store and manage the obtained character recognition information for each table item by matching the location coordinates for each table item.

Additionally, in the embodiment, the application 111 may provide table structuring information based on the obtained character recognition information. (S109)

Here, the table structured information according to the embodiment may mean information converted from a predetermined table structure into a structured text file format according to a preset method (eg, CSV or JSON format, etc.).

In detail, in the embodiment, the application 111 may include the table structure analysis information (i.e., table boundary and each table item data in the embodiment) and the character recognition information (i.e., table item-specific information) acquired for the corresponding table structure. The table structured information can be created and provided based on character data).

In more detail, in the embodiment, the application 111 may match the character recognition information of the table item corresponding to each area (separate location coordinates) divided according to the table boundary data of the table structure analysis information.

Additionally, in the embodiment, the application 111 may perform structuring to separate each area (hereinafter, cell area) matching the character recognition information using commas and/or line breaks.

In an embodiment, the application 111 may separate the different cell regions with commas when the different cell regions are arranged in the horizontal (row) direction.

Additionally, in the embodiment, the application 111 may distinguish the different cell regions using line breaks when the different cell regions are arranged in the vertical (column) direction.

Therefore, in the embodiment, the application 111 may provide the structured information as above in a predetermined file format (eg, CSV or JSON file format, etc.).

In another embodiment, the application 111 may generate and provide the table structured information based on the above-described location coordinates for each table item and the character recognition information.

In detail, in this embodiment, the application 111 may arrange the character recognition information stored by matching the position coordinates of each table item according to the position coordinates of each table item.

Additionally, in this embodiment, the application 111 may perform structuring to separate the character recognition information for each table item arranged above using commas and/or line breaks.

In an embodiment, when the character recognition information of different table items is arranged in the horizontal (row) direction, the application 111 may separate the different character recognition information with commas.

Additionally, in the embodiment, when the character recognition information of different table items is arranged in a vertical (column) direction, the application 111 may separate the different character recognition information with a line break.

Therefore, in the embodiment, the application 111 may provide the structured information as above in a predetermined file format (eg, CSV or JSON file format, etc.).

In this way, the application 111 structures the table structure according to a preset method (e.g., CSV or JSON format method, etc.) based on table structure analysis information derived by a customized detection method according to the type of specific table structure. By providing it in the form of a text file, it is possible to perform digitization that reflects the form information of the table structure within the captured video of a specific table document with high accuracy, and through this, document processing for images that can be implemented including various types of table structures. By improving the quality of automation (i.e., the process of performing optical character recognition (OCR) based on the image and digitizing the results), work efficiency and reliability can also be increased, and various services that utilize the digitized tabular documents can be provided. The quality of (for example, certain database construction services, etc.) can also be improved.

As described above, the document recognition method and system including a table structure according to an embodiment of the present invention automatically recognize and digitize a physical document including a predetermined table structure in a manner specialized for the table structure, so that the physical document is Even if a tangible table structure is included, there is an effect of implementing document processing automation that accurately reflects the structural characteristics of the table structure.

In addition, the method and system for recognizing a document including a table structure according to an embodiment of the present invention perform automatic recognition optimized for the table structure based on the arrangement form according to vertical and horizontal lines and/or spacing between characters, Information that can be obtained by distinguishing each item included in various types of table structures can be obtained clearly and easily, and it enables digitization while maintaining the structural characteristics of the table structure intact, enabling document processing automation. It has the effect of improving performance and quality.

In addition, the document recognition method and system including a table structure according to an embodiment of the present invention provide a predetermined structured text file based on the automatically recognized table structure as described above, thereby improving work efficiency associated with document processing automation. It has the effect of increasing quality and reliability, while also improving the quality of various services that utilize it (for example, a certain database construction service, etc.).

Meanwhile, the embodiments according to the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable by those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. A hardware device may be converted into one or more software modules to perform processing according to the invention and vice versa.

The specific implementations described in the present invention are examples and do not limit the scope of the present invention in any way. For the sake of brevity of the specification, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connections or connection members of lines between components shown in the drawings exemplify functional connections and/or physical or circuit connections, and in actual devices, various functional connections or physical connections may be replaced or added. Can be represented as connections, or circuit connections. Additionally, if there is no specific mention such as “essential,” “important,” etc., it may not be a necessary component for the application of the present invention.

In addition, although the detailed description of the present invention has been described with reference to preferred embodiments of the present invention, those skilled in the art or those skilled in the art will understand the spirit of the present invention as described in the patent claims to be described later. It will be understood that the present invention can be modified and changed in various ways without departing from the technical scope. Therefore, the technical scope of the present invention should not be limited to what is described in the detailed description of the specification, but should be defined by the scope of the patent claims.

Claims (12)

A method in which a document recognition application executed by at least one processor of a terminal recognizes a document including a table structure, comprising:
Obtaining an image of a table document including a predetermined table structure;
Obtaining type information specifying the type of table structure in the obtained table document shooting video;
performing a customized table item detection process according to the obtained type information;
Obtaining table structure analysis information including boundary and item data for the table structure based on the performed customized table item detection process;
Obtaining character recognition information for each acquired item; and
Providing table structuring information in which the table structure is structured in the form of a text file based on the obtained character recognition information and the table structure analysis information.
A method for recognizing documents containing a table structure. According to claim 1,
The type information above is,
A line segment table type in which all at least one item included in the table structure is separated by a line segment, or a blank table type in which the at least one item is separated by a space or a space and a line segment.
A method for recognizing documents containing a table structure. According to clause 2,
The step of performing a customized table item detection process according to the type information is,
If the type information is the line segment table type, performing a line segment-based table item detection process to detect the table structure analysis information based on vertical and horizontal lines;
If the type information is the blank table type, performing a structure analysis-based table item detection process to detect the table structure analysis information based on the character array type.
A method for recognizing documents containing a table structure. According to clause 3,
The step of performing the line segment-based table item detection process is,
Comprising the step of detecting the table structure analysis information based on at least one algorithm of a predetermined line segment detection algorithm and a predetermined vertex detection algorithm.
A method for recognizing documents containing a table structure. According to clause 3,
The step of performing the structural analysis-based table item detection process is,
Including detecting the table structure analysis information based on a predetermined binarization algorithm.
A method for recognizing documents containing a table structure. According to clause 5,
The step of detecting the table structure analysis information based on the predetermined binarization algorithm is,
determining whether a text output pixel exists within an arbitrary first pixel distance in the vertical and horizontal directions with respect to a predetermined pixel in the table structure captured image;
When it is determined that the character output pixel exists, performing first binarization to convert the pixel value of the pixel to 255.
A method for recognizing documents containing a table structure. According to clause 6,
The step of detecting the table structure analysis information based on the predetermined binarization algorithm is,
Further comprising the step of detecting the entire area displaying the table structure based on the table structure captured image on which the first binarization has been performed.
A method for recognizing documents containing a table structure. According to clause 7,
The step of detecting the table structure analysis information based on the predetermined binarization algorithm is,
determining whether a text output pixel exists within the first pixel distance in the horizontal direction with respect to a predetermined pixel in an area corresponding to the entire area within the table structure captured image;
If it is determined that the character output pixel exists, performing a second binarization to convert the pixel value of the pixel to 255;
Further comprising detecting the table structure analysis information based on the corresponding region on which the second binarization was performed.
A method for recognizing documents containing a table structure. According to clause 5,
The step of performing the structural analysis-based table item detection process is,
Further comprising dividing columns or rows of the table structure based on predetermined reference points and baselines.
A method for recognizing documents containing a table structure. According to claim 1,
The step of acquiring character recognition information for each table item is,
Comprising the step of acquiring the character recognition information based on optical character recognition (OCR) based on the image for each table item.
A method for recognizing documents containing a table structure. at least one memory storing a document recognition application; and
At least one processor that reads the document recognition application stored in the memory to recognize a document including a table structure,
The command of the document recognition application is,
Obtaining a table document image containing a predetermined table structure;
Obtaining type information specifying the type of table structure in the obtained table document shooting video;
performing a customized table item detection process according to the obtained type information;
Obtaining table structure analysis information including boundary and item data for the table structure based on the customized table item detection process performed;
Obtaining character recognition information for each acquired item;
A command for performing the step of providing table structuring information in which the table structure is structured in the form of a text file based on the obtained character recognition information and the table structure analysis information.
A document recognition system with a tabular structure. According to claim 11,
The command of the document recognition application is,
If the type information is a line segment table type, performing a line segment-based table item detection process to detect the table structure analysis information based on vertical and horizontal lines;
If the type information is a blank table type, it further includes a command for performing a structure analysis-based table item detection process for detecting the table structure analysis information based on the character array type.
A document recognition system with a tabular structure.