WO2021157009A1 - 画像解析装置、画像解析方法及び画像解析プログラム - Google Patents
画像解析装置、画像解析方法及び画像解析プログラム Download PDFInfo
- Publication number
- WO2021157009A1 WO2021157009A1 PCT/JP2020/004551 JP2020004551W WO2021157009A1 WO 2021157009 A1 WO2021157009 A1 WO 2021157009A1 JP 2020004551 W JP2020004551 W JP 2020004551W WO 2021157009 A1 WO2021157009 A1 WO 2021157009A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- input
- attribute information
- similar
- unit
- Prior art date
Links
- 238000010191 image analysis Methods 0.000 title claims abstract description 30
- 238000003703 image analysis method Methods 0.000 title claims description 4
- 238000001514 detection method Methods 0.000 claims abstract description 39
- 239000000284 extract Substances 0.000 claims abstract description 16
- 239000000470 constituent Substances 0.000 abstract description 10
- 238000000605 extraction Methods 0.000 description 19
- 238000010586 diagram Methods 0.000 description 15
- 238000000034 method Methods 0.000 description 4
- 102200094740 rs1085307938 Human genes 0.000 description 4
- 102220065616 rs533231493 Human genes 0.000 description 4
- 102220353539 c.41G>A Human genes 0.000 description 3
- 102220002646 rs104894310 Human genes 0.000 description 3
- 102220126636 rs886044323 Human genes 0.000 description 3
- 101001139126 Homo sapiens Krueppel-like factor 6 Proteins 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 102220045929 rs587782499 Human genes 0.000 description 2
- 102220606803 Gap junction beta-1 protein_G21D_mutation Human genes 0.000 description 1
- 101000710013 Homo sapiens Reversion-inducing cysteine-rich protein with Kazal motifs Proteins 0.000 description 1
- 101000661807 Homo sapiens Suppressor of tumorigenicity 14 protein Proteins 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 102200006539 rs121913529 Human genes 0.000 description 1
- 102220052829 rs727503049 Human genes 0.000 description 1
- 102220067427 rs765472726 Human genes 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/42—Document-oriented image-based pattern recognition based on the type of document
- G06V30/422—Technical drawings; Geographical maps
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
- G06F16/532—Query formulation, e.g. graphical querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5854—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using shape and object relationship
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/19007—Matching; Proximity measures
- G06V30/19093—Proximity measures, i.e. similarity or distance measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
Definitions
- the present disclosure relates to an image analysis device, an image analysis method, and an image analysis program.
- Patent Document 1 describes that a component such as a utility pole or a manhole is recognized and attribute information indicating an attribute is associated with the recognized component.
- the components shown in the drawing image are shown in a predetermined shape such as a symbol, and the predetermined shape and the attribute information are uniquely associated with each other. It was premised that there was. Therefore, in the technique disclosed in Patent Document 1, when a component is shown in a general-purpose shape such as a circle or a square, the attribute information of the component cannot be uniquely specified from the shape, and the component is configured. There was a problem that the attribute information of the element could not be output.
- the present disclosure has been made to solve the above-mentioned problems, and the object shown in the drawing image and / or the component constituting the object is shown in a general-purpose shape. Even if there is, the purpose is to be able to output the attribute information of the object and / or the constituent elements constituting the object.
- the image analysis device is an image analysis device that outputs attribute information of the input input image, and is an input image input from a drawing image and a knowledge database including attribute information indicating the attribute of the drawing image.
- the similar image search unit that searches for a drawing image similar to It is provided with an image correspondence detection unit that detects as, and an attribute output unit that extracts attribute information corresponding to the corresponding point detected by the image correspondence detection unit from the knowledge database and outputs it as attribute information of the input image. It is a feature.
- the present disclosure discloses an object and / or a component constituting the object even when the object and / or a component constituting the object shown in the drawing image is shown in a general-purpose shape. It has the effect of being able to output the attribute information of.
- FIG. It is a block diagram which shows the structure of the image analysis apparatus which concerns on Embodiment 1.
- FIG. It is a figure which shows an example of the drawing image input from the outside. It is a figure which shows an example of the drawing image stored in the knowledge database. It is a block diagram which shows the structure of the similar image search part. It is a figure explaining the example of extracting the feature amount of the drawing image.
- It is a flowchart which shows the operation of the similar image search unit.
- FIG. 1 is a block diagram showing a configuration of an image analysis device 100 according to the first embodiment.
- the image analysis device 100 is a device in which a drawing image (an example of an input image) corresponding to the drawing data is input and the attribute information of the input input image is output.
- the image analysis device 100 is a device that outputs the attribute information of the object shown in the drawing image and / or the attribute information of the constituent elements constituting the object.
- the attribute information of the drawing image when it is not necessary to separately explain the attribute information of the object and the attribute information of the constituent elements constituting the object, it is simply referred to as the attribute information of the drawing image.
- the output attribute information of the drawing image includes the name of the object and / or the components constituting the object, the model number, the creation date of the drawing image, the update date of the drawing image, and the name of the person in charge of creating the drawing image.
- the text indicating the name of the matter for which the drawing image was created may be included.
- the attribute information of the output drawing image may include the dimensions, materials and / or weights of the object and / or the constituent elements constituting the object.
- the image analysis device 100 includes a similar image search unit 1, an image correspondence detection unit 2, an attribute output unit 3, and a knowledge database 4.
- the knowledge database 4 stores a drawing image and knowledge data including attribute information indicating the attributes of the drawing image.
- the knowledge database 4 may be included in the image analysis device 100 or may be externally attached.
- FIG. 2 is a diagram showing an example of a drawing image input from the outside.
- G1 is a drawing image (an example of the input image) input from the outside.
- the drawing image shows the shape, structure, arrangement, and the like of an object such as a machine or an arrangement and / or a component constituting the object. Further, the drawing image may target a plant, factory or store as an object, and may show the arrangement of equipment, machines, etc. in the object.
- the design drawing of the elevator will be described as a drawing image. Further, in the present embodiment, the elevator will be the object, and the hoistway, the car, the guide rail, the control panel, and / or the ladder device constituting the elevator will be described as components.
- the attribute information is not associated with the object and / or all the components constituting the object shown in the drawing image, but the object and / or Attribute information may be associated with some of the components that make up the object.
- association means that at least the correspondence between the object and / or the component constituting the object shown in the drawing image and the attribute information is uniquely shown.
- the attribute information may be displayed by using a leader line from the object and / or the component constituting the object shown in the drawing image.
- FIG. 3 is a diagram showing an example of a drawing image stored in the knowledge database 4.
- G2 and G3 are drawing images stored in the knowledge database 4.
- G21 to G24 are components shown in the drawing image G2.
- G31 to G34 are components shown in the drawing image G3.
- the components G21 and G31 are associated with a "hoistway” as attribute information.
- the components G22 and G32 are associated with a "basket” as attribute information.
- “guide rails” are associated with the components G23 and G33 as attribute information.
- the component G24 is associated with a "control panel” as attribute information.
- the component G34 is associated with the "ladder device" as the attribute information.
- the attribute information is associated with all the constituent elements in the drawing image, but the attribute information may be associated with some of the constituent elements.
- an image showing an object and / or a component constituting the object and attribute information may be stored in association with each other.
- FIG. 4 is a block diagram showing the configuration of the similar image search unit 1.
- the similar image search unit 1 includes a feature amount extraction unit 11 and a search unit 12, and searches the knowledge database 4 for a drawing image similar to the input drawing image.
- the feature amount extraction unit 11 extracts the feature amount of the drawing image G1 input by using HOG (Histograms of Oriented Grades) or the graph kernel algorithm, and outputs the extracted feature amount to the search unit 12.
- HOG Heistograms of Oriented Grades
- FIG. 5 is a diagram illustrating an example of extracting a feature amount of a drawing image.
- the feature amount extraction unit 11 assigns an identifier to each vertex of the object and / or the component constituting the object shown in the drawing image. As shown in FIG. 5, for example, the feature amount extraction unit 11 may assign a number as an identifier indicating features such as an angle of each vertex, a length of a line segment connecting each vertex, and a thickness.
- an identifier is given only to each vertex, but a node may be generated for all the pixels of the line segment or the thinned out pixels, and the identifier may be given to the node. It should be noted that assigning identifiers to many pixels can improve the accuracy of identifying the characteristics of the components, but since the load of subsequent processing increases, it is determined according to the hardware resources and the like. Just do it.
- the search unit 12 calculates the similarity between the feature amount extracted from the drawing image G1 and the feature amount extracted from the drawing image stored in the knowledge database 4.
- the search unit 12 sorts the drawing images stored in the knowledge database 4 by the degree of similarity, and outputs the drawing images having the highest degree of similarity to the image correspondence detection unit 2 in order. In the present embodiment, it is assumed that the search unit 12 outputs G2 in FIG. 3 to the image correspondence detection unit 2 as a drawing image having a high degree of similarity.
- the feature amount extracted from the drawing image stored in the knowledge database 4 may be included in the knowledge data in advance, or may be sequentially extracted by the feature amount extraction unit 11.
- Non-Patent Document 1 N. Shervashidze et al., “Weisfeiler-Lehman Graph Kernels,” JMLR, vol. 12, pp.2539-2561, 2011.
- FIG. 6 is a block diagram showing the configuration of the image correspondence detection unit 2.
- the image correspondence detection unit 2 includes a feature point extraction unit 21 and a correspondence point matching unit 22, and corresponds between the feature points of the drawing image searched by the similar image search unit 1 and the feature points of the input drawing image. Attached, the feature points of the drawing image are detected as corresponding points.
- the feature point extraction unit 21 extracts the feature points of the object and / or the constituent elements constituting the object shown in the drawing image G1 by using a local feature amount such as HOG.
- the feature point extraction unit 21 extracts an object and / or a vertex constituting a component constituting the object as a feature point. Further, the feature point extraction unit 21 may extract the target object and / or the branch point and / or the end point constituting the component constituting the target object as the feature point.
- the feature point extraction unit 21 outputs information indicating the extracted feature points to the corresponding point matching unit 22.
- FIG. 7 is a diagram showing feature points of the drawing image.
- FIG. 7 (1) shows G14A to G14D, G12D, and G11D as feature points of the drawing image G1.
- G24A to G24D are shown as feature points of the component G24 of the drawing image G2.
- G22D is shown as a feature point of the component G22 of the drawing image G2
- G21D is shown as a feature point of the component G21.
- the corresponding point matching unit 22 calculates the distance between the feature points of the drawing image G2 searched by the similar image search unit 1 and the feature points of the drawing image G1 extracted by the feature point extraction unit 21, and the drawing image having a short distance.
- the feature point of is detected as a corresponding point. Further, the corresponding point matching unit 22 outputs information indicating the detected corresponding point to the attribute output unit 3.
- the corresponding point matching unit 22 calculates the distance between the feature points based on, for example, the equation (2).
- Equation (2): d
- 2 d is the distance between the feature points, f is the feature amount of the feature point of the drawing image G1, and g is the feature amount of the feature point of the drawing image G2.
- the feature amount of the feature point the feature amount extracted by the feature amount extraction unit 11 may be used. Further, the feature amount of the feature point of the drawing image G2 may be stored in advance in the knowledge database 4.
- the corresponding point matching unit 22 uses G24A as the corresponding point of the feature point G14A, G24B as the corresponding point of the feature point G14B, G24C as the corresponding point of the feature point G14C, and G24D as the corresponding point of the feature point G14D.
- Information indicating the detected and detected corresponding points G24A, G24B, G24C, and G24D is output to the attribute output unit 3.
- the attribute output unit 3 extracts the attribute information corresponding to the corresponding point detected by the image correspondence detection unit 2 from the knowledge database 4 and outputs it as the attribute information of the drawing image G1.
- the attribute output unit 3 is an object and / or a component constituting the object corresponding to the corresponding points G24A, G24B, G24C, and G24D output from the corresponding point matching unit 22 of the image correspondence detection unit 2.
- a component G24 of the drawing image G2 is specified.
- the attribute output unit 3 extracts the attribute information (control device) of the component G24 of the drawing image G2 from the knowledge database 4. Then, the attribute output unit 3 outputs the "control device" extracted from the knowledge database 4 as the attribute information of the component G14 of the drawing image G1 shown by the feature points G14A, G14B, G14C, and G14D.
- the image analysis device 100 may include a knowledge data update unit (not shown).
- the knowledge data update unit may store the knowledge data in which the attribute information output by the attribute output unit 3 is associated with the object of the drawing image and / or the constituent elements constituting the object in the knowledge database 4.
- FIG. 8 is a flowchart showing the operation of the similar image search unit 1.
- the feature amount extraction unit 11 of the similar image search unit 1 extracts the feature amount of the input drawing image G1 (ST11).
- the search unit 12 of the similar image search unit 1 extracts the feature amount of the drawing image stored in the knowledge database 4 (ST12). Further, the search unit 12 calculates the similarity based on the feature amount of the drawing image G1 extracted in ST11 and the feature amount of the drawing image extracted in ST12 (ST13).
- the search unit 12 repeats ST12 and ST13 until the drawing image for which the similarity has not been calculated no longer exists in the knowledge database 4 (ST14).
- the search unit 12 sorts the drawing images stored in the knowledge database 4 by the degree of similarity, and outputs the drawing images having the highest degree of similarity to the image correspondence detection unit 2 in order (ST15).
- FIG. 9 is a flowchart showing the operation of the image correspondence detection unit 2.
- the feature point extraction unit 21 of the image correspondence detection unit 2 extracts feature points from the input drawing image G1 (ST21).
- the feature point extraction unit 21 of the image correspondence detection unit 2 extracts the feature points of the drawing image G2 searched by the similar image search unit 1 (ST22).
- the corresponding point matching unit 22 of the image correspondence detection unit 2 calculates the distance between the feature point of the drawing image G2 searched by the similar image search unit 1 and the feature point of the input drawing image G1 (ST23). Further, the corresponding point matching unit 22 detects a feature point of the drawing image G2 having a short distance as a corresponding point, and outputs information indicating the detected corresponding point to the attribute output unit 3 (ST24).
- FIG. 10 is a diagram showing an example of hardware constituting the image analysis apparatus 100.
- the image input unit 101 is an interface for inputting image data to the image analysis device 100 from the outside.
- the image input unit 101 is, for example, a scanner or a camera, reads a printed image of a printed matter, and inputs digital image data to the image analysis device 100.
- the processor 102 realizes each function of the similar image search unit 1, the image correspondence detection unit 2, and the attribute output unit 3 shown in FIG. 1 by executing the program stored in the memory 103.
- the memory 103 is, for example, a non-volatile memory and stores various programs executed by the processor 102. Further, the processor 102 and the memory 103 may be realized by hardware such as a processing circuit.
- the storage unit 104 stores various data (knowledge data, programs, etc.) processed by the processor 102.
- the display unit 105 is, for example, a liquid crystal display and displays attribute information and the like output from the processor 102.
- the storage unit 104 and / or the display unit 105 may be included in the image analysis device 100 or may be externally attached.
- the image analysis device 100 refers to the newly input input image based on the drawing image stored in the knowledge database 4 and the attribute information indicating the attributes of the drawing image. It is possible to output the attribute information to be given. Therefore, this makes it possible to support the addition of attribute information to the object and / or the component constituting the object shown in the input image.
- Embodiment 2 the image analysis device 100 suppresses the processing load of the image analysis device 100, and outputs only the drawing image from the outside in order to output the attribute information of the components shown in the drawing image with high accuracy. Instead, the text related to the attribute information of the drawing image may be input.
- FIG. 11 is a block diagram showing the configuration of the image analysis apparatus according to the second embodiment.
- the image analysis device 100a includes a similar image search unit 1a, an image correspondence detection unit 2, an attribute output unit 3, and a knowledge database 4. Since the image correspondence detection unit 2 and the attribute output unit 3 are the same as those in the first embodiment, the description thereof will be omitted.
- FIG. 12 is a block diagram showing a configuration of a similar image search unit according to the second embodiment.
- the similar image search unit 1a includes a feature amount extraction unit 11, a search unit 12a, and a text reception unit 13.
- the text reception unit 13 receives input of text from the outside and outputs the received text to the search unit 12a.
- the text reception unit 13 does not output the received text as it is to the search unit 12a, but uses a well-known morphological analysis technique to decompose the received text into the minimum meaningful units and decompose it.
- the unit may be output to the search unit 12a.
- the text reception unit 13 may extract words having similar meanings from the thesaurus (synonym dictionary) and output them to the search unit 12a without outputting the received text to the search unit 12a as it is.
- the search unit 12a can search not only the input text itself but also various variations of the notation, and can search the drawing image associated with more appropriate attribute information. For example, when the text input from the outside describes "elevator door”, the text reception unit 13 decomposes it into “elevator” and “door”, and “elevator” as a synonym for "elevator” and “elevator”. , "Door” and "door” as a synonym for "door” are output to the search unit 12a.
- the search unit 12a When “elevator” is described in the text input from the outside, the search unit 12a knows the drawing image of the attribute information related to the text "elevator” and is similar to the input drawing image G1. Search from database 4. In this way, the similar image search unit 1a narrows down the drawing images searched from the knowledge database 4 by using the input text, thereby suppressing the processing load of the image analysis device 100 and displaying the drawing images in the drawing image. It is possible to output the attribute information of the created components with high accuracy.
- the search unit 12 of the similar image search unit 1 sorts the drawing images stored in the knowledge database 4 by the degree of similarity, and outputs the drawing images having the highest degree of similarity to the image correspondence detection unit 2 in order. ..
- the search unit 12 is not limited to this, and may output a drawing image having a similarity equal to or higher than a threshold value to the image correspondence detection unit 2.
- the range of similarity is represented by a numerical value from 0 to 100, and the larger the numerical value, the more similar it is. Then, a threshold value for determining similarity is set in advance.
- the search unit 12 sorts the drawing images stored in the knowledge database 4 in descending order of similarity, searches for drawing images having a similarity equal to or higher than a preset threshold value, and sends the searched drawing images to the image correspondence detection unit 2. You may output it. Then, the image correspondence detection unit 2 calculates the distance between the feature points of all the drawing images output from the similar image search unit 1 and the feature points of the input image, and the feature points of the drawing image having a short distance are the corresponding points. May be detected as.
- the search unit 12 may be set so that the threshold value becomes lower as the number of components shown in the drawing image G1 increases. By doing so, it is possible to easily search the knowledge database 4 for a drawing image including a component corresponding to the component shown in the drawing image G1. Further, the search unit 12 may output a preset number of drawing images in descending order of similarity to the image correspondence detection unit 2 instead of the threshold value. Further, also in this case, the search unit 12 may be set so that the number of drawing images output to the image correspondence detection unit 2 increases as the number of components shown in the drawing image G1 increases. good. By doing so, it is possible to easily search the knowledge database 4 for a drawing image including a component corresponding to the component shown in the drawing image G1.
- the attribute information of the component G14 is output based on the attribute information of the component G24 having the same or similar shape as the component G14 of the drawing image G1.
- the image correspondence detection unit 2 is not limited to the components having the same or similar shape, and may output the attribute information of the components having the feature points whose feature amounts are close to each other.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Library & Information Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
図1は、実施形態1に係る画像解析装置100の構成を示すブロック図である。
画像解析装置100は、図面データに応じた図面画像(入力画像の一例)が入力され、入力された入力画像の属性情報を出力する装置である。具体的には、画像解析装置100は、図面画像に示された対象物の属性情報及び/又は対象物を構成する構成要素の属性情報を出力する装置である。以降、対象物の属性情報及び対象物を構成する構成要素の属性情報をそれぞれ区別して説明する必要のない場合は、単に、図面画像の属性情報と称する。出力される図面画像の属性情報は、対象物及び/又は対象物を構成する構成要素の名称の他、型番、図面画像の作成日、図面画像の更新日、図面画像を作成した担当者の名前、図面画像を作成した案件の名称等を示すテキストを含んでいてもよい。また、出力される図面画像の属性情報は、対象物及び/又は対象物を構成する構成要素の寸法、材質及び/又は重量などを含んでいてもよい。
G1は、外部から入力された図面画像(入力画像の一例)である。図面画像は、機械や配置物などの対象物及び/又は対象物を構成する構成要素の形状、構造、配置などを示す。また、図面画像は、プラント、工場又は店舗を対象物とし、当該対象物における設備、機械などの配置を示してもよい。本実施の形態においては、エレベータの設計図面を図面画像として説明する。また、本実施の形態においては、エレベータを対象物とし、エレベータを構成する昇降路、かご、ガイドレール、制御盤及び/又は梯子装置を構成要素として説明する。また、本実施の形態においては、図面画像に示された対象物及び/又は対象物を構成する全ての構成要素に対して属性情報が対応づけられていないものとするが、対象物及び/又は対象物を構成する一部の構成要素に対して属性情報が対応づけられていてもよい。ここでいう「対応づけ」とは、少なくとも、図面画像に示された対象物及び/又は対象物を構成する構成要素と、属性情報との対応が一意に示されていればよい。具体的には、図面画像に示された対象物及び/又は対象物を構成する構成要素から引き出し線を用いて属性情報を表示する態様であってもよい。
G2、G3は、知識データベース4に記憶された図面画像である。G21~G24は、図面画像G2に示された構成要素である。また、G31~G34は、図面画像G3に示された構成要素である。ここで、構成要素G21、G31には、属性情報として「昇降路」が対応づけられているものとする。構成要素G22、G32には、属性情報として「かご」が対応づけられているものとする。構成要素G23、G33には、属性情報として「ガイドレール」が対応づけられているものとする。構成要素G24には、属性情報として「制御盤」が対応づけられているものとするである。構成要素G34には、属性情報として「梯子装置」が対応づけられているものとする。本実施の形態では、図面画像中の全ての構成要素に対して属性情報が対応づけられているものとするが、一部の構成要素に対して属性情報が対応づけられていてもよい。なお、知識データベース4には、対象物及び/又は対象物を構成する構成要素を示す画像と、属性情報とが対応付けて記憶されていてもよい。
類似画像検索部1は、特徴量抽出部11、及び、検索部12を備え、入力された図面画像に類似する図面画像を知識データベース4から検索する。特徴量抽出部11は、HOG(Histograms of Oriented Gradients)や、グラフカーネルアルゴリズムを用いて入力された図面画像G1の特徴量を抽出し、抽出した特徴量を検索部12に出力する。
特徴量抽出部11は、図面画像に示された対象物及び/又は対象物を構成する構成要素の各頂点に識別子を付与する。特徴量抽出部11は、例えば、図5に示すように、各頂点の角度、各頂点を結ぶ線分の長さ、太さなどの特徴を示す識別子としての番号を付与してもよい。図5では、各頂点にのみ識別子を付与しているが、線分全ての画素、あるいは間引いた画素に対してノードを生成し、当該ノードに識別子を付与してもよい。なお、多くの画素に識別子を付与した方が、構成要素の特徴を特定する精度を向上させることができるが、後続の処理の負荷が増加するため、ハードウェアのリソースなどに応じて決定されればよい。
式(1):S=(f×g)/(|f||g|)
Sは類似度、fは図面画像G1から得られた特徴量、gは知識データに含まれた図面画像の特徴量を示す。
画像対応検出部2は、特徴点抽出部21と、対応点マッチング部22とを備え、類似画像検索部1で検索された図面画像の特徴点と、入力された図面画像の特徴点とを対応付け、図面画像の特徴点を対応点として検出する。
図7(1)には、図面画像G1の特徴点として、G14A~G14D、G12D、G11Dが示されている。また、図7(2)には、図面画像G2の構成要素G24の特徴点としてG24A~G24Dが示されている。また、図面画像G2の構成要素G22の特徴点としてG22D、構成要素G21の特徴点としてG21Dが示されている。
式(2):d=||f-g||2
dは特徴点間の距離、fは図面画像G1の特徴点の特徴量、gは図面画像G2の特徴点の特徴量を示す。なお、特徴点の特徴量は、特徴量抽出部11で抽出した特徴量を用いてもよい。また、図面画像G2の特徴点の特徴量は、知識データベース4に予め記憶されていてもよい。
類似画像検索部1の特徴量抽出部11は、入力された図面画像G1の特徴量を抽出する(ST11)。
画像対応検出部2の特徴点抽出部21は、入力された図面画像G1から特徴点を抽出する(ST21)。
画像入力部101は、外部から画像データを画像解析装置100に入力するインターフェースである。画像入力部101は、例えば、スキャナ、カメラであり、印刷物の印刷画像を読み取り、デジタルの画像データを画像解析装置100に入力する。プロセッサ102は、メモリ103に格納されたプログラムを実行することにより、図1に示す、類似画像検索部1、画像対応検出部2及び属性出力部3という各機能を実現する。メモリ103は、例えば、不揮発性のメモリであり、プロセッサ102に実行される各種プログラムを記憶する。また、プロセッサ102とメモリ103は、例えば、処理回路といった、ハードウェアで実現されてもよい。記憶部104は、プロセッサ102で処理される各種データ(知識データやプログラムなど)を記憶する。表示部105は、例えば、液晶ディスプレイであり、プロセッサ102から出力された属性情報などを表示する。なお、記憶部104及び/又は表示部105は、画像解析装置100に含まれていても良いし、外付けであってもよい。
実施の形態2では、画像解析装置100は、画像解析装置100の処理の負荷を抑制しつつ、図面画像に示された構成要素の属性情報を高精度で出力するために、外部から図面画像だけでなく、図面画像の属性情報に関連したテキストが入力されてもよい。
画像解析装置100aは、類似画像検索部1aと、画像対応検出部2と、属性出力部3と、知識データベース4とを備える。画像対応検出部2及び属性出力部3は、実施の形態1と同一であるため説明は省略する。
類似画像検索部1aは、特徴量抽出部11、検索部12a、及び、テキスト受付部13を備える。テキスト受付部13は、外部からテキストの入力を受け付け、受け付けたテキストを検索部12aに出力する。ここで、テキスト受付部13は、受け付けたテキストをそのまま検索部12aに出力せずに、周知の形態素解析技術を用いることで、受け付けたテキストが意味を持つ最小限の単位に分解し、分解された単位で、検索部12aに出力してもよい。また、テキスト受付部13は、受け付けたテキストをそのまま検索部12aに出力せずに、シソーラス(類語辞典)から類似の意味を有する単語を抽出して、検索部12aに出力してもよい。これにより、検索部12aは、入力されたテキストそのものの表記だけでなく、様々なバリエーションの表記で検索可能となり、より適切な属性情報に対応づけられた図面画像を検索可能となる。例えば、テキスト受付部13は、外部から入力されたテキストに「昇降機ドア」と記載されている場合、「昇降機」と「ドア」に分解し、「昇降機」、「昇降機」の類義語としての「エレベータ」、「ドア」及び「ドア」の類義語としての「扉」を検索部12aに出力する。
上述した実施例は本開示の実施の一例に過ぎず、以下のような構成を追加/変更した応用例が考えられる。
Claims (7)
- 入力された入力画像の属性情報を出力する画像解析装置であって、
図面画像、当該図面画像の属性を示す属性情報を含む知識データベースから、入力された入力画像に類似する図面画像を検索する類似画像検索部と、
前記類似画像検索部で検索された図面画像の特徴点と前記入力画像の特徴点とを対応付け、当該図面画像の特徴点を対応点として検出する画像対応検出部と、
前記画像対応検出部により検出された対応点に対応する属性情報を前記知識データベースから抽出し、前記入力画像の属性情報として出力する属性出力部と
を備えることを特徴とする画像解析装置。 - 前記画像対応検出部は、
前記類似画像検索部で検索された図面画像の特徴点と、前記入力画像の特徴点との距離を算出し、当該距離が近い図面画像の特徴点を対応点として検出する
ことを特徴とする請求項1に記載された画像解析装置。 - 前記画像対応検出部は、
前記入力画像に示された対象物及び/又は当該対象物を構成する構成要素の頂点及び/又は分岐点を特徴点とする
ことを特徴とする請求項1又は2に記載された画像解析装置。 - 前記類似画像検索部は、
前記入力画像を構成する頂点の角度及び/又は頂点と頂点とを結ぶ線分の長さに基づいて抽出した特徴量に類似する図面画像を検索する
ことを特徴とする請求項1~3のいずれかに記載された画像解析装置。 - 前記入力画像は当該入力画像の属性情報に関連したテキストと共に入力され、
前記類似画像検索部は、
前記テキストに関連した属性情報の図面画像であって、前記入力画像に類似した図面画像を前記知識データベースから検索する
ことを特徴とする請求項1~4のいずれかに記載された画像解析装置。 - コンピュータを、
図面画像、当該図面画像の属性を示す属性情報を含む知識データベースから、入力された入力画像に類似する図面画像を検索する類似画像検索部と、
前記類似画像検索部で検索された図面画像の特徴点と、前記入力画像の特徴点の距離を算出し、当該距離が近い図面画像の特徴点を対応点として検出する画像対応検出部と、
前記画像対応検出部により検出された対応点に対応する属性情報を前記知識データベースから抽出し、前記入力画像の属性情報として出力する属性出力部と
して機能させるための画像解析プログラム。 - 入力された入力画像の属性情報を出力する画像解析方法であって、
類似画像検索部が、図面画像、当該図面画像の属性を示す属性情報を含む知識データベースから、入力された入力画像に類似する図面画像を検索し、
画像対応検出部が、前記類似画像検索部で検索された図面画像の特徴点と、前記入力画像の特徴点とを対応付け、当該図面画像の特徴点を対応点として検出し、
属性出力部が、前記画像対応検出部により検出された対応点に対応する属性情報を前記知識データベースから抽出し、前記入力画像の属性情報として出力する
ことを特徴とする画像解析方法。
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE112020005885.9T DE112020005885T5 (de) | 2020-02-06 | 2020-02-06 | Bildanalyseeinrichtung, Bildanalyseverfahren und Bildanalyseprogramm |
CN202080094621.4A CN115004221A (zh) | 2020-02-06 | 2020-02-06 | 图像解析装置、图像解析方法和图像解析程序 |
KR1020227025734A KR20220112853A (ko) | 2020-02-06 | 2020-02-06 | 화상 해석 장치, 화상 해석 방법 및 화상 해석 프로그램을 기억한 기억 매체 |
JP2021559415A JP7004125B2 (ja) | 2020-02-06 | 2020-02-06 | 画像解析装置、画像解析方法及び画像解析プログラム |
PCT/JP2020/004551 WO2021157009A1 (ja) | 2020-02-06 | 2020-02-06 | 画像解析装置、画像解析方法及び画像解析プログラム |
TW109115180A TWI794605B (zh) | 2020-02-06 | 2020-05-07 | 影像解析裝置、影像解析方法以及記錄影像解析程式的媒體 |
US17/857,802 US20220335741A1 (en) | 2020-02-06 | 2022-07-05 | Image analysis device, image analysis method and storage medium storing image analysis program |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2020/004551 WO2021157009A1 (ja) | 2020-02-06 | 2020-02-06 | 画像解析装置、画像解析方法及び画像解析プログラム |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/857,802 Continuation US20220335741A1 (en) | 2020-02-06 | 2022-07-05 | Image analysis device, image analysis method and storage medium storing image analysis program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021157009A1 true WO2021157009A1 (ja) | 2021-08-12 |
Family
ID=77199490
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2020/004551 WO2021157009A1 (ja) | 2020-02-06 | 2020-02-06 | 画像解析装置、画像解析方法及び画像解析プログラム |
Country Status (7)
Country | Link |
---|---|
US (1) | US20220335741A1 (ja) |
JP (1) | JP7004125B2 (ja) |
KR (1) | KR20220112853A (ja) |
CN (1) | CN115004221A (ja) |
DE (1) | DE112020005885T5 (ja) |
TW (1) | TWI794605B (ja) |
WO (1) | WO2021157009A1 (ja) |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH04163671A (ja) * | 1990-10-26 | 1992-06-09 | Mitsubishi Electric Corp | 図面検索方式 |
JPH07262208A (ja) * | 1994-03-17 | 1995-10-13 | Hitachi Ltd | イメージ情報を用いた検索方法 |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2906826B2 (ja) | 1992-05-13 | 1999-06-21 | 三菱電機株式会社 | 設備管理システム |
JP4267475B2 (ja) * | 2004-02-13 | 2009-05-27 | 富士通株式会社 | 図面照合装置 |
JP2008046823A (ja) * | 2006-08-14 | 2008-02-28 | Oki Electric Ind Co Ltd | 画像解釈装置、画像解釈方法、及びプログラム |
JP2016021101A (ja) | 2014-07-14 | 2016-02-04 | 株式会社日立製作所 | 設計変更管理支援装置、及び方法 |
CN104408161A (zh) * | 2014-12-08 | 2015-03-11 | 周理 | 基于相似度查询的模具cad图纸查询与管理方法 |
CN110399509A (zh) * | 2019-06-10 | 2019-11-01 | 万翼科技有限公司 | 一种智能识图系统及方法 |
-
2020
- 2020-02-06 DE DE112020005885.9T patent/DE112020005885T5/de active Pending
- 2020-02-06 KR KR1020227025734A patent/KR20220112853A/ko not_active Application Discontinuation
- 2020-02-06 JP JP2021559415A patent/JP7004125B2/ja active Active
- 2020-02-06 CN CN202080094621.4A patent/CN115004221A/zh active Pending
- 2020-02-06 WO PCT/JP2020/004551 patent/WO2021157009A1/ja active Application Filing
- 2020-05-07 TW TW109115180A patent/TWI794605B/zh active
-
2022
- 2022-07-05 US US17/857,802 patent/US20220335741A1/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH04163671A (ja) * | 1990-10-26 | 1992-06-09 | Mitsubishi Electric Corp | 図面検索方式 |
JPH07262208A (ja) * | 1994-03-17 | 1995-10-13 | Hitachi Ltd | イメージ情報を用いた検索方法 |
Also Published As
Publication number | Publication date |
---|---|
DE112020005885T5 (de) | 2022-09-29 |
JP7004125B2 (ja) | 2022-01-21 |
JPWO2021157009A1 (ja) | 2021-08-12 |
TW202131223A (zh) | 2021-08-16 |
US20220335741A1 (en) | 2022-10-20 |
TWI794605B (zh) | 2023-03-01 |
CN115004221A (zh) | 2022-09-02 |
KR20220112853A (ko) | 2022-08-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111986178B (zh) | 产品缺陷检测方法、装置、电子设备和存储介质 | |
CN111522994B (zh) | 用于生成信息的方法和装置 | |
JP2993673B2 (ja) | 電子ファイル装置 | |
CN111125435B (zh) | 视频标签的确定方法、装置和计算机设备 | |
CN113159091B (zh) | 数据处理方法、装置、电子设备及存储介质 | |
CN111782977B (zh) | 兴趣点处理方法、装置、设备及计算机可读存储介质 | |
EP3852008A2 (en) | Image detection method and apparatus, device, storage medium and computer program product | |
CN111079638A (zh) | 基于卷积神经网络的目标检测模型训练方法、设备和介质 | |
JP2004515826A (ja) | ソフトウェアステッププログラムを構成するためのオブジェクト指向方法 | |
CN111611990B (zh) | 用于识别图像中表格的方法和装置 | |
CN113780098B (zh) | 文字识别方法、装置、电子设备以及存储介质 | |
JP4997892B2 (ja) | 検索システム、検索方法及び検索プログラム | |
CN111652103B (zh) | 室内定位方法、装置、设备以及存储介质 | |
CN114511661A (zh) | 图像渲染方法、装置、电子设备及存储介质 | |
CN113396424A (zh) | 用于自动化材料提取的系统和方法 | |
CN113643260A (zh) | 用于检测图像质量的方法、装置、设备、介质和产品 | |
CN114495147B (zh) | 识别方法、装置、设备以及存储介质 | |
CN116205889A (zh) | 偏移检测方法、装置、电子设备及存储介质 | |
CN116844177A (zh) | 一种表格识别方法、装置、设备及存储介质 | |
CN114297535A (zh) | 查询条件的页面展示方法、装置、设备及存储介质 | |
WO2021157009A1 (ja) | 画像解析装置、画像解析方法及び画像解析プログラム | |
KR20120075626A (ko) | 웹 기반 서비스에서의 전자해도 처리 장치 및 방법 | |
CN114972910A (zh) | 图文识别模型的训练方法、装置、电子设备及存储介质 | |
CN111507265B (zh) | 表格关键点检测模型训练方法、装置、设备以及存储介质 | |
CN114225420A (zh) | 动作数据获取方法、系统、装置、设备和存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20917694 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2021559415 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20227025734 Country of ref document: KR Kind code of ref document: A |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20917694 Country of ref document: EP Kind code of ref document: A1 |