JP6719862B2 - PDF data retrieval system and program for PDF data retrieval system - Google Patents

Description

The present invention relates to a system for correctly extracting data from a PDF (portable document format) file which is a document file, and particularly to various analyzers used for instrumental analysis (eg, chromatograph, spectrophotometer, mass spectrometer, etc.). The present invention relates to a system for correctly extracting the data from a PDF file such as an analysis report including various data obtained by the above.

Nowadays, it is general to control analysis devices and process analysis data obtained as a result of analysis using analysis control software that operates on a general computer.
Many of such analysis control software have a function of creating a report (for example, refer to Non-Patent Document 1). That is, it is a function of arranging a plurality of data related to a certain analysis side by side and finally outputting the data as an analysis report.
Usually, the PDF file format is mainly used because the analysis report is hard to be tampered with and easy to browse.

In such an analysis report, not only the conclusion as the analysis result but also various data for identifying the sample to be analyzed and the data for supporting the conclusion are often described together. It may also be required to describe the analysis conditions that led to the acquisition of such data. That is, it is required to ensure reproducibility that the same data can be obtained when the same sample is analyzed under the same analysis conditions.

"Lab Solutions Multi-Data Report", Shimadzu Corporation, Internet <http://www.an.shimadzu.co.jp/data-net/labsolutions/dbcs/mdr.htm>, [March 12, 2015 Search] ]

These data for specifying the sample, the data for the analysis conditions, the data for the analysis results, etc., are used in the form of spreadsheet software such as Excel (registered trademark) when it is assumed that they will be used next. It is generally stored in. However, if the analysis device and the analysis data analysis device are manufactured by the same manufacturer, they can be easily used because the data storage formats of both are the same or known. If the manufacturers are different, it is generally not possible to use such data in the form of spreadsheet software. Alternatively, such data may not be obtained (output) in the first place.

On the other hand, each analysis device or analysis data analysis device issues a report of analysis results, and most of them are in the PDF format as described above. Therefore, even if the manufacturers are different, the analysis result report can be used. Since the analysis condition data and the analysis result data are often included in the data, it is possible to perform the reproduction analysis or the analysis with some conditions changed by using the analysis condition data and the analysis result data.

The analysis condition data and the analysis result data are generally described in a table format in the PDF file. To create a PDF file from a word processor or spreadsheet software, Acrobat software called (registered trademark) of middleware, also has the ability to retrieve data from a PDF file. Therefore, data can be retrieved from the table in the PDF file by using the middleware.

However, currently available middleware such as Acrobat have the following problems. For example, when a table part is extracted as text data from a PDF file containing a table as shown in FIG. 1(a), there is a blank cell in the table as shown in FIG. 1(b). Then, the data of the subsequent part is taken out in the state where it was packed before. Further, since the character data before the table and the character data after the table are taken out continuously with the data in the table, only the table cannot be reconstructed. Furthermore, middleware generally cannot acquire the data of the ruled lines forming the table included in the PDF file. Therefore, it is difficult to accurately reproduce the table. Therefore, even if an attempt is made to perform a similar analysis using a PDF file that describes the results of an analysis that has already been performed, it cannot be performed automatically, and the person who performs the subsequent analysis can re-read the condition data for the previous analysis. There was a problem that the data had to be reconstructed, or even if an attempt was made to perform further analysis based on the data obtained as a result of the previous analysis, manual input was required again.

The present invention has been made to solve the above problems, and a main object thereof is to provide a system and a program capable of correctly extracting data from a table included in a PDF file.

The PDF data extraction system according to the present invention made to solve the above problems is
A PDF data extraction system for acquiring a character string constituting a table included in a PDF file using PDF middleware and outputting the character string in a format reproducible in a table format,
a) an extraction unit setting unit that sets the extraction of the character string from the PDF file in the PDF middleware in line units based on the attribute of the character string and the horizontal threshold that is the threshold of the horizontal movement amount between the character strings,
b) a lateral threshold value setting unit that sets the lateral threshold value to a predetermined value,
c) a character string data acquisition unit that extracts a character string for each line from a specified PDF file according to the settings of the extraction unit setting unit and the horizontal threshold setting unit,
d) Among the coordinate values of each character string for each line extracted by the character string data acquisition unit , the position in the row direction of the character string is determined by the x coordinate value, and the position in the column direction of the character string is determined by the y coordinate value. and table forming section which determines and their form more reconstruction table to place in tabular form,
e) An output unit that outputs the reconstruction table in a predetermined data format.

In the present invention, the “character string” means one or a plurality of letters, numbers, symbols, etc. arranged side by side, and may include a space (blank). When the middleware extracts the character string from the PDF file, the extraction unit can be an object unit, a line unit, a block unit, or the like. In the PDF data extraction system according to the present invention, the extraction unit setting unit sets this for each row. When extracting character strings on a line-by-line basis, the middleware uses a same font, has the same font size, etc. However, if the lateral movement amount between them is larger than a predetermined lateral threshold value, it is treated as another row. Therefore, by appropriately setting the horizontal shift amount in the horizontal threshold setting unit, when the table is included in the specified PDF file, the data (characters, numbers, symbols) in each cell of each row of the table is set. Etc.) will be extracted as separate lines.

When the middleware extracts the character string from the PDF file, each character string is given a coordinate value. Therefore, based on the coordinate value attached to the character string of each row thus extracted, it can be determined whether or not the extracted row is a character string separated by the lateral movement amount equal to or more than the lateral threshold value. The table forming unit sequentially inspects the coordinate values of the character strings of the extracted rows, and if a plurality of rows of character strings separated in the horizontal direction are continuous, by arranging them in the horizontal direction, the original It is possible to reconstruct one row of the table (hereinafter, referred to as "original row"). At that time, the position (column position) of each cell can also be reconstructed by the coordinate value (x coordinate value) attached to each character string. Further, when a plurality of such original rows are continuous, the table can be reconstructed by using the x coordinate value for the cells in each column in one row and the y coordinate value for each cell in the multiple rows. .. In addition, when blank lines are included in consecutive lines, it is possible to consider the surroundings as consecutive lines to form one table as a whole, or to separate tables with blank lines and form separate tables. Good. This can be dealt with by setting in advance.

However, in the target PDF table, if there is a cell that contains multiple lines of character strings in one cell, the y-coordinate skipping interval of the multiple lines is small (half or an integer fraction). Become. Using this and the x coordinate value, the table forming unit reconstructs a cell including such a plurality of rows.

Accordingly, the table forming unit can specify the table included in the target PDF file with high probability. Even if there is a cell without a character string in the table, it can be recognized as a blank cell, so that the table can be reconstructed correctly.

The output unit outputs the table formed by the table forming unit in a predetermined format, such as a csv format, a tab delimited format, an xml format, or an Excel format.

As a preferred embodiment of the present invention,
The table forming unit may further include a delimiter input unit that allows the user to input text immediately before or after the table, and the table forming unit may determine the start or end of the table based on the text thus input. it can.

With the PDF data extraction system according to the present invention, the table can be detected from the PDF file including the table, and the character string included in the table can be acquired. Furthermore, even if there is a blank cell that does not contain a character string in the table, the table can be reconstructed in a form that reflects the blank cell.

Therefore, even if the analysis report obtained as a result of the control of the analysis device or the analysis is output in the PDF format, the table included in the analysis report can be reconstructed extremely easily and accurately. When it is necessary to perform reanalysis or additional analysis under the same or similar conditions as those used, it is possible to greatly reduce the user's labor. In addition to this, the PDF data retrieval system according to the present invention significantly reduces manual input processing by the user. As a result, errors and mistakes can be reduced.

The figure which shows the example (a) of one display (output) of a PDF file, and the example (b) of the result output when the text data of the table contained in this PDF file is copied and pasted (prior art). The schematic block diagram which shows one Example of the PDF data extraction system which concerns on this invention. The flowchart which shows the example of operation|movement of a PDF data extraction system. An example of a file output as a result of the operation of the PDF data extraction system. Another screen example in the operation of the PDF data extraction system.

Hereinafter, an example of an embodiment of a PDF data extraction system according to the present invention will be described in detail with reference to the drawings.

FIG. 2 shows an embodiment of the PDF data extraction system 1 according to the present invention. The present PDF data extraction system 1 is actually a computer, and includes a CPU (Central Processing Unit) 10 which is a central processing unit, a memory 12, a monitor (display unit) 14 including an LCD (Liquid Crystal Display), a keyboard, a mouse, and the like. An input unit 16 including a storage unit and a storage unit 20 including a large-capacity storage device such as a hard disk or an SSD (Solid State Drive) are connected to each other. The storage unit 20 stores a PDF file extracting system program 21 according to the present invention, and a PDF file middleware (PDF middleware) 22 such as Adobe Acrobat, for example. The storage unit 20 also stores an OS (Operating System) 29. Note that the PDF middleware 22 may be commercially available from other companies as long as it has a function of extracting a character string from a PDF file.

The PDF data extraction system 1 according to the present embodiment is an interface (I/F) for controlling direct connection with an external device or connection with an external device via a network such as a LAN (Local Area Network). 18 and is connected from the I/F 18 to an analyzer A1 which is a chromatograph mass spectrometer through a network cable NW (or wireless LAN). In the PDF data extraction system according to the present invention, the number of analyzers connected to the outside via the I/F 18 is not limited to one, and a plurality of analyzers may be used. Further, the PDF data extracting system may be integrated with the analyzer.

As shown in FIG. 2, the PDF data extraction system program 21 includes an extraction unit setting unit 31, a lateral threshold value setting unit 32, a character string data acquisition unit 33, a table forming unit 34, and an output unit 35. .. All of these are functional means basically realized by software by the CPU 10 executing the PDF data fetching system program 21. The PDF data extraction system program 21 does not necessarily have to be a single program, and may have a function incorporated in a part of a program for controlling the analyzer, for example, and its form is not particularly limited.

Hereinafter, the operation of the PDF data extraction process will be specifically described with reference to the flowchart of FIG.

First, the user starts the PDF data fetch system program 21 (hereinafter, simply referred to as a program). At this time, the user designates the analysis report 5 which is a PDF file as an operation target. In the present embodiment, the analysis report 5 will be described as the analysis report of FIG.

[Step S21] Since the analysis report 5 is designated, the extraction unit setting unit 31, the lateral threshold value setting unit 32, and the character string data acquisition unit 33 cooperate with the PDF middleware 22 so that the characters are extracted from the analysis report 5. Extract columns by row.
That is, the extraction unit setting unit 31 sets the character string extraction unit of the PDF middleware 22 in units of lines, and the horizontal threshold setting unit 32 further sets the horizontal threshold in that case. When the distance between the characters in the character string included in one line is equal to or larger than the horizontal threshold value, the PDF middleware 22 determines that the character strings before and after the character string belong to another line. This lateral threshold value is usually set to a preset value (for example, two or more half-width spaces of a normal size (for example, 11 points), etc.). After the PDF middleware 22 is set by the extraction unit setting unit 31 and the horizontal threshold setting unit 32, the character string data acquisition unit 33 uses the PDF middleware 22 to extract each character in line units from the PDF file of the analysis report 5. Get a column. Thus, the data output from the PDF middleware 22 includes the extracted character strings and the coordinate values of the character strings. In this example, the following character strings and their coordinate values (represented by "x:y") are obtained for each of the lines <L1> to <L31>. In addition, in the following example, the delimiter of the character string is represented by a comma (,), but if there is a possibility that the character string contains a comma, you can use other symbols or tabs. it can. Alternatively, the coordinate value (x:y) itself or a combination of the coordinate value and a comma may be used as a delimiter between the character strings.

<L1> Analysis Report (40:1)
<L2> Analysis result (0:3)
<L3> Sample 1 (2:5)
<L4> Sample 2 (25:5)
<L5> Sample 3 (50:5)
<L6> Sample 4 (70:5)
<L7> 311 (10:7)
<L8> 353(30:7)
<L9> 322 (60:7)
<L10> 399 (80:7)
<L11> 2598 (7:9)
<L12> 2283 (26:9)
<L13> 2033 (77:9)
<L14>
<L15> 11.3 (26:13)
<L16> 13.6 (60:13)
<L17> 0 (85:13)
<L18> 115(10:15)
<L19> 203 (26:15)
<L20> 268 (60:15)
<L21> 183 (80:16)
<L22> *1 (10:17)
<L23> *2 (26:17)
<L24> *3 (60:17)
<L25> 7973 (7:19)
<L26> 6088 (77:19)
<L27> 15 (60:21)
<L28> 3 (85:21)
<L29> Comment (0:23)
<L30> Of the samples 1-4 analyzed this time, sample 4 (0:25)
Abnormal pH was observed only in <L31>. This is sample 4 (0:27)

[Step S22] Next, the table forming unit 32 checks whether or not a plurality of lines separated by the horizontal movement amount equal to or larger than the horizontal threshold value are consecutive in the acquired character string and coordinate value for each line. When a character string having the same y-coordinate in each row extends over a plurality of rows, it can be determined to be that. In the case of this example, the third line <L3> to the sixth line <L6>, the seventh line <L7> to the tenth line <L10>, and the like correspond thereto. Then, those lines are reconstructed into one line (the “original line”) based on the x coordinate value. When these original rows extend over a plurality of rows, the table forming unit 34 determines that they are data forming one table. In the above example, it is determined that the third row <L3> to the 28th row <L28> including the 14th row <L14> which is an empty row are data forming one table. On the other hand, the 1st line <L1>, the 2nd line <L2>, the 30th line <L30> and the 31st line <L31> are all composed of one character string, so normal text data Is determined to be.

The table forming unit 34 detects that the x coordinate of the character string from the third line <L3> to the 28th line <L28> is divided into four at substantially equal intervals, that is, there are four columns. In addition, the y coordinate of the 21st line <L21> is between the y coordinate of the 18th line to the 20th line and the y coordinate of the 22nd line <L22> to the 24th line <L24>. It is detected that the jump width in the y direction is smaller than that between the two, and it is determined that the 18th row <L18> to the 24th row <L24> constitute one cell in the y direction.

The above judgment, that is,
-The third line <L3> is the first line of the table, and the 28th line <L28> is the last line.
・The number of rows is four.
-The 14th line <L14> has no character string and is a blank line.
-The 18th row <L18> to the 24th row <L24> is one cell in the y direction.
・A table with 8 rows and 4 columns.
Based on the above, the table forming unit 34 forms the table so that each character string is arranged at the correct position in the table.

[Step S23] Finally, the output unit 35 outputs the table formed by the table forming unit 34 in a predetermined file format designated in advance or designated by the user at this timing (FIG. 4, FIG. 4). , This is an example output in csv format.) This completes the PDF data extraction process.

FIG. 5 shows an example of a screen in operation in another embodiment of the PDF data extraction system 1 according to the present invention. In this embodiment, a title input unit 36 and a division input unit 37 are further provided as functional means realized by software when the CPU 10 executes the analysis PDF data extraction system program 21.

When the user inputs "Sample 1, Sample 2, Sample 3, Sample 4" as the column title list on this input screen (title input unit 36), the table forming unit 34 displays these character strings in the first row of the table. It is possible to determine that the number of columns in the table is four, which is a component of the (original row). Further, although the title list of rows is blank in this embodiment, the number of rows in the table can be determined by inputting a character string in this field.

Further, in this input screen (delimiter input section 37), the user designates "comment", which is the character string immediately after the last character string in the table, as the end character string. Accordingly, the table forming unit 34 can determine that the table ends with "3" which is the character string immediately before the character string "comment". In the example of FIG. 5, only the terminating character string can be entered, but the user may be allowed to enter the character string immediately before the first character string in the table in the PDF file as the starting character string in the table. .. Accordingly, the table forming unit 34 can determine the start character string of the table.

1...PDF data retrieval system 5...Analysis report (PDF file)
10... CPU
12...Memory 14...Monitor 16...Input unit 18...I/F
20... Storage unit 21... PDF data retrieval system program 22... PDF middleware 29... OS
31... Extraction unit setting unit 32... Horizontal threshold value setting unit 33... Character string data acquisition unit 34... Table forming unit 35... Output unit 36... Title input unit 37... Break input unit A1... Analytical device

Claims (4)

A PDF data extraction system for acquiring a character string constituting a table included in a PDF file using PDF middleware and outputting the character string in a format reproducible in a table format,
a) an extraction unit setting unit that sets the extraction of the character string from the PDF file in the PDF middleware in line units based on the attribute of the character string and the horizontal threshold that is the threshold of the horizontal movement amount between the character strings,
b) a lateral threshold value setting unit that sets the lateral threshold value to a predetermined value,
c) a character string data acquisition unit that extracts a character string for each line from a specified PDF file according to the settings of the extraction unit setting unit and the horizontal threshold setting unit,
d) Among the coordinate values of each character string for each line extracted by the character string data acquisition unit , the position in the row direction of the character string is determined by the x coordinate value, and the position in the column direction of the character string is determined by the y coordinate value. and table forming section which determines and their form more reconstruction table to place in tabular form,
e) A PDF data extraction system, comprising: an output unit that outputs the reconstruction table in a predetermined data format. Furthermore,
f) A delimiter input unit that allows a user to input text immediately before or after the table, wherein the table forming unit determines the start or end of the table based on the text. The PDF data retrieval system described. Furthermore,
g) A title input unit for allowing a user to input a title of a column or row of the table, and the table forming unit determines the number of columns or rows of the table based on the title. Alternatively, the PDF data extraction system according to item 2. A program used in a computer for a PDF data extraction system that acquires a character string forming a table included in a PDF file using PDF middleware and outputs the character string in a format that can be reproduced in a table format. , The computer,
a) an extraction unit setting unit that sets the extraction of the character string from the PDF file in the PDF middleware in line units based on the attribute of the character string and the horizontal threshold that is the threshold of the horizontal movement amount between the character strings,
b) a lateral threshold value setting unit that sets the lateral threshold value to a predetermined value,
c) a character string data acquisition unit that extracts a character string for each line from a specified PDF file according to the settings of the extraction unit setting unit and the horizontal threshold setting unit,
d) Among the coordinate values of each character string for each line extracted by the character string data acquisition unit , the position in the row direction of the character string is determined by the x coordinate value, and the position in the column direction of the character string is determined by the y coordinate value. and table forming section which determines and their form more reconstruction table to place in tabular form,
e) A program for a PDF data extraction system, which causes the reconstruction table to function as an output unit that outputs the data in a predetermined data format.

Images