JP7471802B2 - Archive Support System - Google Patents

Description

The present invention relates to an archive support system, for example, an archive support system that supports the task of extracting metadata from a digital archive (group of images) of historical architectural drawings.

Digital archiving of various materials (books, documents, paintings, drawings, etc.) has been underway in libraries, archives, museums, etc. both in Japan and abroad, and image materials are being digitized and are becoming searchable along with metadata (catalog information, bibliographic information) indexes (e.g., IIIF, etc.).

Amid this, image analysis and machine learning techniques are being developed to analyze and extract content from digital images. For example, the ROIS-DS Center for Open Data in the Humanities and the National Institute of Japanese Literature at the National Institutes for the Humanities have announced a system that recognizes cursive writing in ancient documents and converts them into text.

In addition, at the École Polytechnique Fédérale de Lausanne in Switzerland, as part of their digital humanities research, they are conducting research using deep learning to extract segments from materials that contain a mixture of various layouts and content, such as decorative text and charts (see, for example, non-patent document 1).

Oliveira, Sofia Ares, Benoit Seguin, and Frederic Kaplan. "dhSegment: A generic deep-learning approach for document segmentation." In 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 7-12. IEEE, 2018. Hideko Matsunami, Yusuke Kawakami, Yuji Katsuki, Minako Okuyama, "Changes in the Drawing Name Column of Blueprints," in "Mansions of the Meiji and Taisho Eras: The World of Colored Drawings Created by Shimizu Gumi," Kashiwa Shobo, 2009, 268-261

The importance of archives is also being recognized in the field of architecture. The National Museum of Modern Architecture recently held its first archivist seminar to promote the archiving of architectural documents such as architectural drawings, and introduced examples of how original documents are handled.

Archiving and digitally archiving architectural drawings requires the receipt and organization of a huge amount of material, followed by many manual processes such as cataloguing, inputting the data into a database, digitizing the data, and publishing it, all of which is done manually, creating an enormous burden.

Currently, digital archiving and the release of digital archives are carried out on the premise that documents such as drawings stored by archival institutions must first be manually collected by a person, who then creates a vast catalog and transcribes the metadata. However, by taking advantage of the characteristics of machine learning as described above, it is possible to first digitize the drawings and then mechanically extract metadata from the documents; the more archived documents there are, the more accurate the extraction can be. Creating such a machine learning system could significantly reduce the amount of work required for archiving and analysis.

On the other hand, in the case of historical drawings of modern buildings, whose historical value is being rediscovered and whose analysis for preserving and recording cultural properties, as well as repairs for utilization, are urgently needed, each designer and contractor has created their own drawing style, and metadata is generally written in that style. These appear as columns and marks on the drawings, and even if the same person created the drawings, the style may differ depending on the time period or department, as shown in Figures 5 (1) and (2). As can be seen in Non-Patent Document 2, styles differ depending on the person who created the drawings (department, etc.). There may also be structural commonalities between different styles. Therefore, architectural drawings have a variety of styles and a structure (for example, a drawing name column), and the contents written are defined as metadata depending on that structure.

The inventors focused on this issue and came up with the invention, which uses machine learning to analyze the layout of a large number of digitized drawing images and extracts the style parts in which metadata is written.

The present invention has been made in consideration of the above, and aims to provide an archive support system that reduces the labor required for archiving and analysis.

In order to solve the above problems and achieve the objective, the archiving support system of the present invention is a support system for supporting the archiving of drawings, and is characterized by having a learning result recording unit that uses training data related to the style of the drawing included in the image of the drawing to learn the style of the drawing included in the image and records a drawing model for identifying the style of the drawing, and an extraction unit that uses the drawing model to extract the style of the drawing included in the image and extracts data related to metadata from the extracted style.

Another archive support system according to the present invention is characterized in that in the above-mentioned invention, the style of the drawing is a style related to the drawing name column.

The archive support system according to the present invention is a support system for supporting the archiving of drawings, and includes a learning result recording unit that uses training data related to the style of the drawing included in the image of the drawing to learn the style of the drawing included in the image and records a drawing model for identifying the style of the drawing, and an extraction unit that uses the drawing model to extract the style of the drawing included in the image and extracts data related to metadata from the extracted style, thereby providing an archive support system that reduces the labor required for archiving and analysis.

In addition, another archive support system according to the present invention has the effect of being able to extract data related to metadata based on the information written in the drawing name column, since in the above-mentioned invention, the drawing style is a style related to the drawing name column.

FIG. 1 is a schematic diagram showing an embodiment of an archive support system according to the present invention. FIG. 2 is an explanatory diagram of this embodiment. FIG. 3 is a schematic flow chart of this embodiment. FIG. 4 is an explanatory diagram of this embodiment. FIG. 5 is a diagram showing an example of a design drawing.

Below, an embodiment of the archive support system according to the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment.

As shown in FIG. 1, the archive support system 100 according to the present invention includes a database 10, a support server 12, and a user terminal 14. This archive support system 100 is connected to similar archive support systems of other institutions via a network.

The database 10 records a large number of digitalized images of architectural drawings. The images of architectural drawings are input by scanning paper drawings with a scanner or the like. In the example shown in the figure, drawing material groups A, B, and C of different styles are each recorded in the database 10 as multiple digital format files. As shown in FIG. 2, drawing material group A is composed of No. 1 to No. N, drawing material group B is composed of A-1 to A-X and E-1 to E-X, and drawing material group C is composed of 1.jpg to X.jpg. In addition, this database 10 also records character groups, image objects, metadata information, data structured documents, etc. extracted from images by a method described below. In addition, metadata that does not appear as information in the images (for example, the date of collection, notes by donors, researchers, etc.) may be recorded in the database 10. In the example shown in FIG. 2, metadata of the production year is recorded in each file of drawing material group C. In addition, drawing material groups that are publicly available through the archive support system of another institution may be temporarily recorded and used for learning and analysis in the archive support system 100.

The support server 12 is a computer that supports the creation of archives, and is connected to the database 10 and the user terminal 14. This support server 12 includes a control unit 16, a teacher data recording unit 18, and a learning result recording unit 20.

The control unit 16 is used to analyze the layout from a group of images of many drawings using machine learning, and to extract style parts in which metadata is written. This control unit 16 is configured to perform various processes such as learning processing, layout analysis, character image extraction, and item content extraction using a CPU, RAM, ROM, etc. This processing is executed by the learning processing unit 22, layout analysis unit 24, character image extraction unit 26, and item content extraction unit 28.

The learning processing unit 22 executes a learning process that uses training data including images with a set drawing style to generate a drawing model for identifying the drawing style in an unknown image. Specifically, the learning processing unit 22 performs machine learning using images and training data indicating the drawing style (e.g., the drawing name field) contained in the images. In this learning, the learning processing unit 22 analyzes, for example, patterns that appear repeatedly in multiple images, generates a drawing model for identifying the same style between images, and registers the generated drawing model in the learning result recording unit 20.

The layout analysis unit 24 uses the drawing model recorded in the learning result recording unit 20 to analyze the layout of the image to be analyzed. Specifically, the layout analysis unit 24 analyzes the layout of the image to be analyzed by image analysis, and separates the figures and characters (contents) in the image from the style. The style is represented by, for example, lines or a drawing name field. The layout analysis unit 24 may, for example, analyze which part of the image to be analyzed corresponds to the drawing name field, and perform a process to determine the degree of match with the drawing model. As a result, drawings with a degree of match equal to or greater than a predetermined threshold may be determined to have the same (common) style, and drawings with a degree of match less than the predetermined threshold may be determined to have different styles.

The character image extraction unit 26 executes recognition and extraction processing of characters and images contained in each item area in the style. In this embodiment, it is assumed that drawings having the same style part are made by the same drawing creator, and characters and image objects are extracted from each item area in the style (e.g., drawing name column) separated by the layout analysis unit 24. Here, character groups and image objects are extracted by OCR for type, by image analysis for handwritten characters, and as image data for stamps. The extracted character groups and image objects may be recorded in the database 10. Note that a recognition model may be generated by machine learning using teacher data consisting of text characters associated with character images of handwritten characters and teacher data associated with image objects such as seal impressions, and this recognition model may be used to extract characters cut out from each item information in the style, image objects such as seal impressions, etc.

The item content extraction unit 28 extracts information on the type of each item described in the style (e.g., project name, specifications, scale, date, approval stamp, etc.) from the characters and images extracted by the character image extraction unit 26. The extracted information is recorded in the database 10. Note that a recognition model may be generated by machine learning using training data associated with the type of each item described in the style, and the type of each item described in the style may be extracted from the characters and images using this recognition model. The item content extraction unit 28 may also data-structure the drawing based on the extracted information, and create a data-structured document that describes the style and the content of each item.

The teacher data recording unit 18 records multiple pieces of teacher data used in machine learning. The teacher data may be recorded in advance before machine learning is performed, or may be recorded or updated by the user via the user terminal 14.

The learning result recording unit 20 records drawing models for analyzing drawing styles. Drawing models are recorded when machine learning is performed using training data.

The user terminal 14 is a computer terminal used by a user. This user terminal 14 is equipped with an editing interface consisting of a control unit, an input unit (keyboard, etc.), and an output unit (display, etc.), all of which are not shown. The user can use the database 10 and the support server 12 via the user terminal 14.

Using the archive support system 100 configured as described above, the layout between images with a common drawing style is learned, the accuracy of distinguishing characters and image objects in each item in the style is improved, and the information of each item extracted from there is recorded in the database 10. By estimating and providing examples of the contents of each item (e.g. date and time, property name, person in charge, etc.) using machine learning, it is possible to simplify confirmation and input by the archive worker. It is also possible to analyze the relevance of metadata information between images with different drawing styles, so that it is possible to distinguish the same items between different styles. In addition, it is also possible to group groups of materials with different drawing styles into separate sets, while sharing metadata items common to different styles between the groups, thereby improving the accuracy of distinction.

Next, an example of a processing procedure performed by the above-mentioned archive support system 100 will be described.
In the following process, machine learning is performed using initial training data, and then the layout is analyzed.
As shown in Fig. 3, first, the drawing materials (material group A) are scanned, and a group of image files is stored in the database 10 (step S1). Next, the learning processing unit 22 analyzes patterns that appear repeatedly in multiple images, and identifies the same style between the images (step S2). Next, the layout analysis unit 24 performs image processing to separate the figures/text and the style in the images (step S3).

Next, the character image extraction unit 26 extracts items such as character groups and seal impressions written in the style (step S4). Then, the item content extraction unit 28 extracts the type of each item written in the style.

Next, the image of the drawing is data-structured to create a data-structured document that describes the style and the contents of each item (step S5). The format of the data-structured document may be, for example, an XML (extensible Markup Language) format in which the text describing the style and each item is enclosed in tags to create a structure.

Next, the user modifies and assigns meaning to the data structured document on the editing interface of the user terminal 14 (step S6).

Next, steps S1 to S6 are similarly performed for the other drawing material group (material group B) (step S7). Next, as shown in FIG. 4, for example, the relevance between images and the relevance of metadata information between different drawing material groups (material groups A and B) are analyzed, and identical items between different styles are identified (step S8). Next, the user integrates and corrects items of similar information in the data structured document on the editing interface of the user terminal 14 (step S9). If more materials are added, return to step S1 and repeat steps S1 to S9 to improve the accuracy of identification. Note that the accuracy of machine learning may be improved by obtaining drawing material groups published by archive support systems of other institutions via a network or the like and having the learning processing unit 22 learn them.

This makes it easier to input data that can be read from a huge amount of drawings. In addition, based on the metadata information for each identified item, users can easily classify, search, extract, list, and output any drawings from the collection of drawings (for example, those made by the same design firm, those with the same stamp, those created during a specific period, etc.).

According to this embodiment, in drawings that contain a variety of information such as characters, figures, pictures, and marks, it is possible to separate style and content, analyze the similarity of styles, and extract items for each style. In addition, by learning the same items between different styles and creating a list of items and styles for each drawing, it is also possible to sort a group of drawings by item or search by content.

This dramatically reduces the effort required to organize and analyze archived materials, and eases the labor required for archiving and analysis. It also makes it possible to extract relationships between drawing materials that were previously difficult to find manually.

In the above embodiment, the application to architectural drawings has been described as an example, but the drawings of the present invention are not limited to this. For example, the present invention can also be applied to drawings related to machines, etc. Even when applied to such drawings, the same effects as those described above can be achieved.

As described above, the archive support system of the present invention is a support system for supporting the archiving of drawings, and includes a learning result recording unit that uses training data related to the style of the drawing included in the image of the drawing to learn the style of the drawing included in the image and records a drawing model for identifying the style of the drawing, and an extraction unit that uses the drawing model to extract the style of the drawing included in the image and extracts data related to metadata from the extracted style, thereby providing an archive support system that reduces the labor required for archiving and analysis.

In addition, in another archive support system according to the present invention, in the above-mentioned invention, since the style of the drawing is a style related to the drawing name column, data related to metadata can be extracted based on the information written in the drawing name column.

As described above, the archive support system of the present invention is useful for extracting and analyzing metadata from archives of drawings not only within construction companies, but also in public archive institutions such as public archives and libraries that hold vast amounts of drawings, and in private archive institutions such as private museums and corporate history archives, and is particularly suitable for reducing the labor required for work and analysis.

REFERENCE SIGNS LIST 10 Database 12 Support server 14 User terminal 16 Control unit 18 Teacher data recording unit 20 Learning result recording unit 22 Learning processing unit 24 Layout analysis unit 26 Character image extraction unit 28 Item content extraction unit (extraction unit)
100 Archive Support System

Claims (1)

A support system for supporting a drawing archiving operation, comprising:
a learning result recording unit that uses teacher data related to the style of the drawing name field included in the image of the drawing to learn the style of the drawing name field included in the image of the drawing and records a drawing model for identifying the style of the drawing name field ;
an extraction unit that uses the drawing model to extract a style of a drawing name field included in an image of the drawing , and extracts data related to metadata from the extracted style ;
An archive support system characterized by structuring the extracted data and creating a data-structured document that describes the style and the contents of the data .

Images