IL313293A - Ai-based system method for detecting repetitive appearances of elements in multiple views of building plans - Google Patents

Description

47749/ AI-BASED SYSTEM METHOD FOR DETECTING REPETITIVE APPEARANCES OF ELEMENTS IN MULTIPLE VIEWS OF BUILDING PLANS Field of the Invention The present invention relates to the field of construction. More particularly, the invention relates to an AI-based system and method for detecting repetitive appearances of elements in multiple views of building plans. Background of the Invention A building plan is a set of drawings (typically 2-D) that provides a graphical illustration, and textual descriptions, of what a building or a complex will look like after construction. The plan is made by designers (architects, engineers, and other consultants) based on the client’s and building code requirements and is then shared with the builders and contractors for budgeting and execution. A drawing set includes different plans such as a foundation plan that represents the general design intent of the foundation (a slab, crawl space, basement, post, and beams, the extent of structural slabs, and more), a floor plan (that illustrates the layouts of the rooms, walls, doors, and windows), a roof plan (that typically illustrates ridges, valleys, and hips indicating the roofing material and slopes of roof surfaces), an elevation plan (that illustrates a view of a building seen from one side and includes two dimensional, flat, representation of one facade). A drawing set may also include some of the other important features, like the location of walls (both interior and exterior) with their corresponding coating, wall openings like doors and windows, vertical circulation including stairs, structural elements, including columns, fixture locations like sinks dimensions, reference symbols, notes, scales and legends. A typical drawing set can be lengthy and includes a plurality of sheets, representing multiple layers, such as various architectural plans, air-conditioning plan, electrical plan and more. Drawing sets may also include partial plans, Enlargement plans and unit plans, to further elaborate on the overall floor plan. For example, the architect can provide a general view of a 47749/ portion of a building (e.g., a floor) or a selected area in that portion in one sheet , and an enlarged view of this selected area in another sheet , with each view depicting different types of required information and different levels of details. Hence, the nature of a drawing set is to split the information of every specific area of the building across several views (in several sheets), with each view providing different information or level of details, such as: overall plan, enlarged plan, mechanical plan, reflected ceiling plan, lighting plan, etc. The primary challenge is to have all the information coordinated across all related views, so as to avoid discrepancies between various elements represented in different views. Fig. 1A illustrates such a discrepancy. Region 101 in view 100 is enlarged in view 103. However, door 102 in region 101 is missing in view 103. This kind of discrepancy originates from a clerical error or miscommunication during the design stage. Fig. 1B illustrates the differences between views of similar areas in different sheets of the drawing set. It can be seen that the views have different drawing scales, different rotation angles and different details, such that the views are not sufficiently distinctive. Conventional methods involve manual review by engineers, design teams, estimators, or other professional teams, comparing various types of information in the set, in order to detect discrepancies. This is a highly mundane, time-consuming, expensive and error prone process. Failing to detect discrepancies has far-reaching consequences regarding the cost of construction. Before starting the building process, the contractor enters a pre-construction stage, in which he reviews the drawing set and submits a proposal, which should reflect the expected construction costs and timeline. According to a study conducted by the Construction Industry Institute (CII), design errors account for up to 70% of all construction rework, which also lead to delays and significant cost overruns. Moreover, large discrepancies between pre-tender estimates and actual bids for construction have a serious impact on the viability of a project. Owners, architects, engineers, cost consultants, contractors and subcontractors all 47749/ have a vested interest in ensuring a high degree of cost predictability. The later a discrepancy is detected, the more costly it is to fix it. In one example, a discrepancy between the architectural and interior design plans was detected in the field - the bathrooms were finished with paint as per the architect and not with tiles as per the interior designer. This had to be corrected, and the rework attached to this correction cost 1.5 million dollars to the contractor. If the same issue was caught after the bid but before construction, it could cost 700,000 dollars for the contractor for the missed scope in the original budget. But, if the issue was discovered even before the bid submission, the correction would only require a quick clarification on the documents. Therefore, it is critical to detect any critical error at an early stage, such that the contractor will have correct information before providing any estimated cost, or before starting the construction works. In order to detect such errors, it is required to compare geometric features as well as related information (such as callout notes, tags, etc.) in different views that appear in different sheets of the drawing set , as well as comparing text between different sheets . In addition, there is a need to unify the appearances of a particular element (e.g., a door or a window) in all the sheets , in order to avoid counting the same element twice in quantity takeoff estimation. It is therefore an object of the present invention to provide a method for automatically detecting the repetition of elements in different appearances and views in a drawing set.

It is another object of the present invention to provide a method for accurately detecting the repetition of elements in different sizes and orientations in various views (floor plans, enlargement plans, etc.) of a drawing set .

Other objects and advantages of the invention will become apparent as the description proceeds. Summary of the Invention 47749/ A method for detecting repetitive appearances of elements in multiple views of a building plan, comprising the steps of: a) detecting the repetition of views by identifying views in each sheet of the building plan; classifying views, sheet by sheet , while assigning a unique identifier to each view; b) for each classified view in a sheet , searching the each classified view in all other sheets ; and c) performing a dedicated template matching using an AI-based matching model, for finding matches between pairs of input images and corresponding template images. The input images may be in PDF or image 2-D formats. The dedicated template matching may be performed by the following steps: a) receiving a drawing set of the building plan, in a 2-D format; b) identifying and isolating specific plan views within the drawing set; c) identifying the drawing scale (a textual object) of each plan view; d) assigning an estimated a drawing scale to each view that lacks a drawing scale, or change the scale to potentially match the scale of the same view in another sheet ; e) identifying candidate views for match; f) for each match, performing feature extraction and using a classification model to disqualify false matches; g) setting a threshold for legitimate matches; h) produce a list of consolidated views that reflects views that are contained in other views; i) outputting all the discovered transformation matrices that transform each view to its corresponding containing view; and j) outputting the coordinates of the contained view in the corresponding containing view. A dataset for training the matching model is created by identifying all the connections between 47749/ views. The AI-based matching model may be trained using supervised learning, to extract a set of defined features. The AI-based matching model may be a binary model, which decides if there is a match or not. The method may further comprise the step of avoiding searching a large view within a smaller view by using the pixels size and the scale of each view, thereby reducing noise and ineffective matching operations. The method may further comprise the step of avoiding unnecessary searching by calculating the actual area of each view using the pixel map and the scale. The matching model may be trained by: a) conducting template matching, to create a plurality of candidate views; b) checking each candidate view against a template view, to identify a potential match; and c) calculating and features that provide indications regarding the level of match between the candidate view and the template. The features that should be used may be predetermined to the matching model. The predetermined features may be: a) mutual words that appear in a candidate view and a template view; b) the amount of pixels in a candidate view with respect to a template view. Identifying and isolating specific plan views may be performed by: a) following receiving the set of drawings of the building plan as an input, identifying and 47749/ isolating specific plan views within the drawing set; b) identifying sheets within the drawing set, that include floor plans; c) identifying, using a deep learning model, individual views within all the sheets in the set , while ignoring the title block; and d) outputting all identified individual views. Identifying the scale of each view and bringing different plans to a uniform scale may be performed by: a) following receiving the set of all identified individual views, identifying the drawing scale of each view and assigns a scale to the view; b) identifying all drawing scale annotation components within each sheet; c) assigning a drawing scale component to each view, using rule-based matching of views to drawing scale annotations; d) checking if each view has an assigned drawing scale annotation and if it does not have a scale annotation, uses alternatives to determine an estimated drawing scale of the view, while considering known standard scales; and e) outputting all the views with their corresponding assigned drawing scales. Identifying candidate views for match may be performed by: a) following receiving the set of all the views with their corresponding assigned drawing scales, for each view, finding all the views that might contain the view, based on view size and scale; b) resizing views to a uniform average scale; c) rotating the views to match orientation with the potential containing view; d) applying a template-matching algorithm between the two views, while selecting the top N results, from all performed matches; and e) outputting a list of views, each view with a list of matches. Disqualifying false matches may be performed by: 47749/ a) following receiving a list of views with a list of matches for each view, performing feature extraction, according to a predefined list of features; b) disqualifying false matches, based on the prediction of a dedicated AI-based classification model; and c) outputting a list of pairs of matching views. Consolidating the pairs of views into a full hierarchical structure may be performed by: a) following receiving a list of pairs of matching views, creating a hierarchical tree for all views which contains other views; and b) computing the transformation matrix between each view to other views that is required to position any view to the root view of its hierarchy, thereby overlaying any pair of views on each other, even if a direct match could not be found. Brief Description of the Drawings The above and other characteristics and advantages of the invention will be better understood through the following illustrative and non-limitative detailed description of preferred embodiments thereof, with reference to the appended drawings, wherein: - Fig. 1A illustrates a discrepancy between the information originating from different sheets of the building plan; - Fig. 1B illustrates the same area in the building as it’s depicted in two different views (different floor plans of different scale and orientation); - Fig. 2 is a flowchart of a dedicated template matching process, according to an embodiment of the invention; - Fig. 3 is a flowchart of the process of identifying and isolating specific plan views, according to an embodiment of the invention; - Fig. 4 is a flowchart of the process of identifying the scale of each view and bringing different plans to a uniform scale, according to an embodiment of the invention; 47749/ - Fig. 5 is a flowchart of the process of identifying candidate views for match, according to an embodiment of the invention; - Fig. 6 is a flowchart of the process of disqualifying false matches, according to an embodiment of the invention; and - Fig. 7 is a flowchart of the process of consolidating the pairs of views into a full hierarchical structure, according to an embodiment of the invention.

Detailed Description of the Present Invention Definitions Drawing Set (Construction drawing set) - A compilation of drawings, notes, tables, schedules, references and other graphical or worded descriptions, that allow general contractors and workers to understand and construct any project. The drawing set (also, Contract Documents) are created by the design team, such as architects, engineers and other consultants. The information in the drawing set is the design intent - what the building should look like after construction. Sheets - The drawing set is composed of sheets (pages). Each sheet should have a sheet name and a unique sheet number. A typical sheet will have on the right hand side (or the bottom) a titleblock that describes the project and the sheet. It will include information, such as the architect, project name and address, sheet name and number, issuance information, etc. Views (drawing views) - a view is a "basic" unit of graphic information representing an aspect of the proposed building. Typical views are: plans, sections, elevations, details and diagrams. A typical view would have a title, drawing scale and often include various annotations such as dimensions, callout notes, tags, etc. The same area of the building can be represented by different views in different disciplines. The present invention provides an AI-based method for detecting repetitive appearances of regions in multiple views in a drawing set (also called "AI Drawings Overlay"). For example, the method can identify a room which appears in different views of the set. After identifying 47749/ repetitive regions, another method can be implemented to identify elements and objects in these regions, as well. The typical input to the system is the drawing set , which may be submitted in PDF format (which is standard in the industry). At the first step, the system identifies views on each sheet of the drawing set . At the next step, the system performs classification of views, sheet by sheet , while assigning a unique identifier to each view. At the next stage, the system searches for each classified view in a sheet , in all other sheets to find a potential match between an input image and a template image, using an AI-based matching model. Using a unique template matching A conventional technique for finding a potential match is template matching, which is a method for searching and finding the location of a template image in a larger image. It slides the template image over the input image (as in 2D convolution) and compares the template and patch of input image under the template image. Several comparison methods may be implemented to find a match. It returns a grayscale image, where each pixel denotes how much the neighborhood of that pixel matches the template . However, the system of the present invention requires adaptations in the conventional template matching techniques, since (1) each view appears in a different scale and different rotation and (2) by definition, the different views are not identical, even when describing the same area, thus, there is an expected level of dissimilarity which could be too high for template matching. These differences between views generate a substantial amount of false positive indications (considered as "noise") which should be filtered out. Given the above constraints, the system proposed by the present invention provides a dedicated template matching process (illustrated in Fig. 2), which is described by the following steps: The first step 201 is to receive a drawing set of the building plan, in a 2D format such as PDF or image files; 47749/ The next step 202 is to identify and isolate specific plan views within the drawing set, using object detection models; The next step 203 is to identify the drawing scale (a textual object) of each plan view; If there is no drawing scale, the next step 204 is to assume a drawing scale to each view without a drawing scale or change the scale such that it will potentially match the scale of the same view in another sheet ; The next step 205 is to identify candidate views for match using template matching; The next step 206 is to extract features for each match in order to disqualify false matches using a classification model; The next step 207 is to disqualify candidates using a proprietary classification model; The next step 208 is to produce a list of consolidated views (i.e., if view A is within view B, and view B is within view C, then view A is within view C) that reflects views that are contained in other views; The output is all discovered transformation matrices from each view to its corresponding containing view. The system defines which features should be used, in order to increase the accuracy. For example, text that appears in a specific view and in a candidate view or the size of a view. An AI-based matching model is trained (using supervised learning) to make a binary classification and decide if there is a match or not. In order to reduce noise and ineffective matching operations, the scale of each view is used to avoid searching a large view within a smaller view. For example, there is no need to search for a view of a floor within a view of a room. The AI-based matching model The matching model is trained by conducting template matching to create a plurality of candidate views (each candidate view will be checked against a template view to identify a potential match) and features that provide indications regarding the level of match between 47749/ the candidate view and the template, are calculated. Since creating a dataset is a slow process that requires skilled personnel, there is only little data to train the matching model (which may be for example, a neural network). Generally, existing models for finding image similarity select and extract the features independently and therefore, require a large amount of data for training. This problem is overcome by determining to the matching model, which features should be used, such as mutual words that appear in a candidate view and a template view, as well as the amount of pixels in a candidate view with respect to a template view. This can be done by using metrics that provide indications regarding how much a feature affects the result, such as the mutual information score (which expresses the extent to which observed frequency of co-occurrence differs from what should statistically would have been expected. Statistically, it is a measure of the strength of association between variables x and y). The matching model outputs a score (typically between 0 and 1), such that a higher score reflects a better match. The dedicated template matching process uses several computational metrics to generate a single score used to make a decision whether or not there is a match. The selection of features is also affected by the time and resources required to calculate each feature. However, if the elimination of a feature (to save computation time) deteriorates the result, this feature should remain. Also, there are different combinations of features that improve the result, even though each one of them received a relatively lower score. Fig. 3 is a flowchart of the process of identifying and isolating specific plan views, according to an embodiment of the invention. After receiving the set of drawings of the building plan as an input (from the preceding process), the system identifies and isolates specific plan views within the drawing set. At the next step 301, the system identifies sheets within the drawing set that 47749/ include floor plans. At the next step 302, the system identifies, using a deep learning model, individual views within each sheet, while ignoring the title block (the sheet template background that contains information related to the design, such as the design name, the company, and the designer). At the next step 303, the system outputs all identified individual views. Fig. 4 is a flowchart of the process of identifying the scale of each view and bringing different plans to a uniform scale, according to an embodiment of the invention. After receiving the set of all identified individual views (from the preceding process), the system identifies the drawing scale of each view and assigns a scale to the view. At the next step 401, the system identifies all drawing scale annotation components within each sheet. At the next step 402, the system assigns a drawing scale component to each view using rule-based matching of views to drawing scale annotations. At the next step 403, the system checks if each view has an assigned drawing scale annotation. If it does not have a scale annotation, at the next step 404 the system uses alternatives to determine an estimated drawing scale of the view, while considering known standard scales. These alternatives may include a classification model, shared gridlines as an anchor, PDF metadata (PDF has the ability to assign scaling factor to an area within the PDF. This is usually used by some software programs to allow users to take measurements), and extrapolation based on common size of identified elements within the view. If it does have, at the next step 405 the system outputs all the views with their corresponding assigned drawing scales. Fig. 5 is a flowchart of the process of identifying candidate views for match, according to an embodiment of the invention. After receiving the set of all the views with their corresponding assigned drawing scales (from the preceding process), for each view, at the next step 501, the system finds all the views (based on view size and scale) that might contain that view. At the next step 502, the system resizes views to a uniform average scale. At the next step 503, the system rotates the views to match orientation with the potential containing view. At the next step 504, the system applies a template-matching algorithm between the two views, while 47749/ selecting the top N results, from all performed matches. At the next step 505, the system outputs a list of views, each view with a list of matches, such that view A is included within view B at coordinates C. Fig. 6 is a flowchart of the process of disqualifying false matches, according to an embodiment of the invention. After receiving a list of views with a list of matches for each view (from the preceding process), at the next step 601, for each match, the system performs feature extraction, according to a predefined list of features. At the next step 602, the system disqualifies false matches, based on the prediction of a dedicated AI-based classification model (all matches with a score lower than a predetermined threshold are disqualified ). At the next step 603, the system outputs a list of pairs of matching views. Fig. 7 is a flowchart of the process of consolidating the pairs of views into a full hierarchical structure, according to an embodiment of the invention. After receiving a list of pairs of matching views (from the preceding process), at the next step 701, the system creates a hierarchical tree for all views which contains other views. At the next step 702, the system computes the transformation matrix between each view to other views that is required to position any view to the root view of its hierarchy (if A is in B and B is in C, then C would be the root). This allows overlaying any pair of views on each other, even if a direct match could not be found (e.g., if view A was found in view B and view B was found in view C, the location of view A within view C can therefore be deducted). It should be noted that although the examples and illustrations above relate to the field of construction, the method of the present invention can be implemented to other fields that require the detection of repetitive appearances of objects in multiple views, such as the management of storerooms, logistic warehouses, IC design, inventory counting, electronic circuit design / digital circuitry design, etc. The above examples and description have of course been provided only for the purpose of 47749/ illustration, and are not intended to limit the invention in any way. As will be appreciated by the skilled person, the invention can be carried out in a great variety of ways, employing more than one technique from those described above, all without exceeding the scope of the claims.

Claims (16)

47749/ Claims

1. A method for detecting repetitive appearances of elements in multiple views of a building plan, comprising: a) detecting the repetition of views by identifying views in each sheet of said building plan; classifying views, sheet by sheet , while assigning a unique identifier to each view; b) for each classified view in a sheet , searching said each classified view in all other sheets ; and c) performing a dedicated template matching using an AI-based matching model, for finding matches between pairs of input images and corresponding template images.

2. A method according to claim 1, wherein the input images are in PDF or image 2-D formats.

3. A method according to claim 1, wherein the dedicated template matching is performed by the following steps: a) receiving a drawing set of the building plan, in a 2-D format; b) identifying and isolating specific plan views within said drawing set; c) identifying the drawing scale (a textual object) of each plan view; d) assigning an estimated a drawing scale to each view that lacks a drawing scale, or change said scale to potentially match the scale of the same view in another sheet ; e) identifying candidate views for match; f) for each match, performing feature extraction and using a classification model to disqualify false matches; g) setting a threshold for legitimate matches; h) produce a list of consolidated views that reflects views that are contained in other views; i) outputting all the discovered transformation matrices that transform each view to its corresponding containing view; and 47749/ j) outputting the coordinates of the contained view in said corresponding containing view.

4. A method according to claim 1, wherein a dataset for training the matching model is created by identifying all the connections between views.

5. A method according to claim 1, wherein the AI-based matching model is trained using supervised learning, to extract a set of defined features.

6. A method according to claim 1, wherein the AI-based matching model is a binary model, which decides if there is a match or not.

7. A method according to claim 1, further comprising avoiding searching a large view within a smaller view by using the pixels size and the scale of each view, thereby reducing noise and ineffective matching operations.

8. A method according to claim 1, further comprising avoiding unnecessary searching by calculating the actual area of each view using the pixel map and the scale.

9. A method according to claim 1, wherein the matching model is trained by: d) conducting template matching, to create a plurality of candidate views; e) checking each candidate view against a template view, to identify a potential match; and f) calculating and features that provide indications regarding the level of match between said candidate view and said template.

10. A method according to claim 1, wherein the features that should be used, are predetermined to the matching model. 47749/

11. A method according to claim 1, wherein the predetermined features are: c) mutual words that appear in a candidate view and a template view; d) the amount of pixels in a candidate view with respect to a template view;

12. A method according to claim 1, wherein identifying and isolating specific plan views are performed by: a) following receiving the set of drawings of the building plan as an input, identifying and isolating specific plan views within the drawing set; b) identifying sheets within the drawing set, that include floor plans; c) identifying, using a deep learning model, individual views within all the sheets in the set , while ignoring the title block; and d) outputting all identified individual views.

13. A method according to claim 1, wherein identifying the scale of each view and bringing different plans to a uniform scale are performed by: f) following receiving the set of all identified individual views, identifying the drawing scale of each view and assigns a scale to the view; g) identifying all drawing scale annotation components within each sheet; h) assigning a drawing scale component to each view, using rule-based matching of views to drawing scale annotations; i) checking if each view has an assigned drawing scale annotation and if it does not have a scale annotation, uses alternatives to determine an estimated drawing scale of said view, while considering known standard scales; and j) outputting all the views with their corresponding assigned drawing scales.

14. A method according to claim 1, wherein identifying candidate views for match is performed by: a) following receiving the set of all the views with their corresponding assigned drawing 47749/ scales, for each view, finding all the views that might contain said view, based on view size and scale; b) resizing views to a uniform average scale; c) rotating the views to match orientation with the potential containing view; d) applying a template-matching algorithm between the two views, while selecting the top N results, from all performed matches; and e) outputting a list of views, each view with a list of matches.

15. A method according to claim 1, wherein disqualifying false matches is performed by: a) following receiving a list of views with a list of matches for each view, performing feature extraction, according to a predefined list of features; b) disqualifying false matches, based on the prediction of a dedicated AI-based classification model; and c) outputting a list of pairs of matching views.

16. A method according to claim 1, wherein consolidating the pairs of views into a full hierarchical structure is performed by: a) following receiving a list of pairs of matching views, creating a hierarchical tree for all views which contains other views; and b) computing the transformation matrix between each view to other views that is required to position any view to the root view of its hierarchy, thereby overlaying any pair of views on each other, even if a direct match could not be found.