JP2011048791A - Device, method and program for providing information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To attain effective use of knowledge information by allowing a user to execute search of documents related to the part of a component to be referred to. <P>SOLUTION: An information providing device 1 is configured to divide a 3D model of a component stored in a 3D model storage part 2 into parts, to extract, from a document stored in a document information storage part 3, image of a component added to the document, and to extract, from the extracted image, a symbol specifying a part of the image. The information providing device 1 is further configured to select, when the symbol is extracted, a part of the 3D model corresponding to the part of the image specified by the symbol and to store the selected part of the 3D model in a partial document information storage part 4 in association with the document related with the image of the part. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an information providing apparatus, an information providing method, and an information providing program.

  In recent years, as a variety of products has been reduced in volume, companies have increasingly developed projects for similar products in parallel. Under such a development system, it is very important to effectively use the know-how acquired in the development of a certain product in the development of other products in order to improve the development efficiency.

  As a mechanism for accumulating and sharing know-how, an information providing system called a knowledge management system is known. The knowledge management system stores knowledge information describing various know-how in a database. Specifically, as shown in FIG. 47, the knowledge management system associates a part expressed in a 3D (3 dimensions) model with document information in which failure information and the like are stored in a database.

  In addition, the knowledge management system can search knowledge information about the parts designated by the user from the database. For example, in the knowledge management system, when a part is designed with a 3D model, a part similar to the part under design is searched from the 3D model in the database, and a document stored in association with the searched 3D model is stored. Output.

JP 2008-262540 A

  However, since the conventional knowledge management system technique described above stores parts and documents in association with each other, it is possible to select a part of a part represented by a 3D model and refer to document information relating to the selected part. There was a problem that it was not possible.

  For example, as shown in FIG. 48, the user may want to refer to a document such as failure information described for a past part portion similar to the part portion being designed. However, in the conventional knowledge management system technique, a part similar to the whole part under design is searched from a 3D model in a database, and a document stored in association with the searched 3D model is searched.

  For this reason, a document that is not related to a document other than the document related to the portion that the user wants to refer to is searched. As a result, there has been a problem that a document relating to a part portion that the user wants to reference cannot be searched, and knowledge information cannot be effectively used.

  The present invention has been made to solve the above-described problems of the prior art, and stores a 3D model part and a document in association with each other so that a user can search for a document related to a part part that the user wants to refer to. The objective is to effectively use knowledge information.

  In the present invention, the document associated with the 3D model includes an image of the 3D model (such as a photograph of a part), and an ellipse or an arrow indicating a notable range such as a failure location is added to the image. In order to solve the above-described problems and achieve the purpose, the 3D model of the part is first divided into parts. At the same time, for the part image added to the document stored in association with the 3D model, a symbol (such as an ellipse surrounding the part or an arrow pointing to the part) that identifies the part in the image is extracted. To do. When the symbol is extracted, the part of the 3D model corresponding to the part of the part shown in the image specified by the symbol is selected, and the selected 3D model part and the part image are associated with each other. The document is associated with and stored in the storage unit.

  According to one aspect of the information providing apparatus disclosed in the present application, since the 3D model part and the document are stored in association with each other, it is possible to perform a search of the document related to the part part that the user wants to refer to, so Effective use can be realized.

FIG. 1 is a block diagram illustrating the configuration of the information providing apparatus according to the first embodiment. FIG. 2 is a block diagram illustrating the configuration of the information providing apparatus according to the second embodiment. FIG. 3 is a diagram illustrating an example of a 3D model. FIG. 4 is a diagram illustrating an example of document information. FIG. 5 is a diagram illustrating an example of a component table. FIG. 6 is a diagram illustrating an example of a component / document association table. FIG. 7 is a diagram illustrating an example of the point information table. FIG. 8 is a diagram illustrating an example of the surface information table. FIG. 9 is a diagram illustrating an example of the segment information table. FIG. 10 is a diagram illustrating an example of the text information table. FIG. 11 is a diagram illustrating an example of the image information table. FIG. 12 is a diagram illustrating an example of the graphic information table. FIG. 13 is a diagram illustrating an example of the pixel information table. FIG. 14 is a diagram illustrating an example of the image feature amount table. FIG. 15 is a diagram illustrating an example of the viewpoint position information table. FIG. 16 is a diagram illustrating an example of a similarity calculation table. FIG. 17 is a diagram illustrating an example of a document reference information table. FIG. 18 is a diagram illustrating an example of the query point information table. FIG. 19 is a diagram illustrating an example of a query / surface information table. FIG. 20 is a diagram illustrating an example of a query segment information table. FIG. 21 is a diagram showing an example of a query / segment similarity calculation table. FIG. 22 is a diagram illustrating an example of a query segment evaluation value calculation table. FIG. 23 is a diagram illustrating an example of a partial / document association table. FIG. 24 is a diagram for explaining an example of the segmentation technique. FIG. 25 is a diagram illustrating an example of a document in the HTML file format. FIG. 26 is a diagram illustrating an example of an instruction mark. FIG. 27 is a diagram illustrating an example of instruction mark information. FIG. 28 is a diagram illustrating an example of image information with an instruction mark. FIG. 29 is a diagram illustrating an arrangement format of pixels of a color image. FIG. 30 is a diagram illustrating a pixel arrangement format of a monochrome image. FIG. 31 is a diagram illustrating processing for aligning an image and 3D model in document information. FIG. 32 is a diagram for explaining the edge extraction result. FIG. 33 is a diagram illustrating an example of the surface number information. FIG. 34 is a diagram illustrating processing for selecting a portion of the 3D model corresponding to the position of the instruction mark. FIG. 35 is a diagram illustrating an example of instruction mark range pixel information. FIG. 36 is a diagram illustrating an example of pixel histogram information. FIG. 37 is a diagram illustrating processing for matching an image in a document with a 3D model portion. FIG. 38 is a diagram for explaining search processing. FIG. 39 is a diagram illustrating a screen display example of search results. FIG. 40 is a diagram for describing processing for increasing reference information. FIG. 41 is a flowchart illustrating the processing operation of the information providing apparatus according to the second embodiment. FIG. 42 is a flowchart illustrating the processing operation of the information providing apparatus according to the second embodiment. FIG. 43 is a flowchart illustrating the processing operation of the information providing apparatus according to the second embodiment. FIG. 44 is a flowchart illustrating the processing operation of the information providing apparatus according to the second embodiment. FIG. 45 is a flowchart illustrating the processing operation of the information providing apparatus according to the second embodiment. FIG. 46 is a diagram illustrating a computer that executes an information providing program. FIG. 47 is a diagram for explaining the prior art. FIG. 48 is a diagram for explaining the prior art.

  Exemplary embodiments of an information providing apparatus, an information providing method, and an information providing program according to the present invention will be described below in detail with reference to the accompanying drawings.

  In the following embodiment, the configuration and processing of the information providing apparatus according to the first embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating the configuration of the information providing apparatus according to the first embodiment. As shown in FIG. 1, the information providing apparatus 1 according to the first embodiment includes a 3D model storage unit 2, a document information storage unit 3, a partial document information storage unit 4, a division unit 5, an image extraction unit 6, and a specific symbol extraction unit. 7, a partial selection unit 8, and a partial document storage unit 9.

  The 3D model storage unit 2 stores a 3D model of a part. The document information storage unit 3 stores a document related to a component to which the component image is added. The partial document information storage unit 4 stores the 3D model part and the document associated with the part image in association with each other.

  The dividing unit 5 divides the 3D model of the component stored in the 3D model storage unit 2 into parts. The image extraction unit 6 extracts an image of a part added to the document from the document stored in the document information storage unit 3. The specific symbol extraction unit 7 extracts a symbol (such as an ellipse surrounding the part or an arrow pointing to the part) that identifies the part of the part shown in the image extracted by the image extraction unit 6.

  When the symbol is extracted by the specific symbol extraction unit 7, the portion selection unit 8 selects a part of the 3D model corresponding to the part of the image specified by the symbol. The partial document storage unit 9 stores the part of the 3D model selected by the partial selection unit 8 and the document associated with the part image in the partial document information storage unit 4 in association with each other.

  As described above, the information providing apparatus 1 divides the 3D model of the component stored in the 3D model storage unit 2 into parts, and extracts the image of the component added to the document from the document stored in the document information storage unit 3. Then, a symbol specifying a part of the image is extracted from the extracted image. When the symbol is extracted, the information providing apparatus 1 selects a 3D model portion corresponding to the image portion specified by the symbol, and associates the selected 3D model portion with the image of the part. The document is stored in the partial document information storage unit 4 in association with the document.

  For this reason, the information providing apparatus 1 stores the 3D model part and the document in association with each other, and enables the user to search for a document related to the part part that the user wants to refer to, thereby effectively realizing the knowledge information. It is possible.

  In the following embodiments, the configuration and processing flow of the information providing apparatus according to the second embodiment will be described in order, and finally the effects of the second embodiment will be described.

[Configuration of Information Providing Device]
First, the configuration of the information providing apparatus 10 will be described with reference to FIG. FIG. 2 is a block diagram illustrating the configuration of the information providing apparatus 10 according to the second embodiment. As illustrated in FIG. 2, the information providing apparatus 10 includes an input unit 11, an output unit 12, a control unit 13, and a storage unit 14. The processing of each of these units will be described below.

  The input unit 11 inputs an operation instruction for designing a 3D model, an instruction for a user to select and display necessary information during design, and includes a keyboard, a mouse, a microphone, and the like. The output unit 12 displays a 3D model of the part under design, document information that is knowledge information about the part, information necessary for the user during design, a monitor (or display, touch panel), a speaker, and the like. It is.

  The storage unit 14 stores data and programs necessary for various processes performed by the control unit 13, and particularly includes a 3D model information storage area 14a, a document information storage area 14b, a table storage area 14c, and a partial document information storage area 14d. .

  The 3D model information storage area 14a stores a 3D model of a part. Specifically, as illustrated in FIG. 3, the 3D model information storage area 14a stores a part number, a designer, and a design date together with the 3D model. FIG. 3 is a diagram illustrating an example of a 3D model.

  The document information storage area 14b stores a document related to the component to which the component image is added. For example, as shown in FIG. 4, the document information storage area 14b stores, as information about the designed part, information that a failure is likely to occur in a part of the part together with an image of the part. FIG. 4 is a diagram illustrating an example of document information.

  The table storage area 14c stores various tables necessary for various processes by the control unit 13. Specifically, the table storage area 14c includes a component table, a component / document association table, a point information table, a surface information table, a segment information table, a text information table, an image information table, a graphic information table, a pixel information table, and an image feature. Has a quantity table.

  The table storage area 14c includes a viewpoint position information table, a similarity calculation table, a document reference information table, a query point information table, a query / surface information table, and a query / segment information table.

  The table storage area 14c includes a query / segment similarity calculation table and a query / segment evaluation value calculation table. In addition, although each table is demonstrated below, what kind of process each table is used for is described in description of the control part 13 mentioned later. Further, for each table, it is not necessary to take a table format, and a database other than the table configuration may be used.

  The component table stores component types of components represented by a 3D model. Specifically, as illustrated in FIG. 5, the component table stores a component number that uniquely identifies a component and a component type that indicates the component type in association with each other. FIG. 5 is a diagram illustrating an example of a component table.

  The part / document association table stores associations between parts of the 3D model and document information. As shown in FIG. 6, the component / document association table associates a document number that uniquely identifies document information, a document title that indicates the title of the document information, a document URL that indicates the URL of the document information, and a component number. Add and remember. FIG. 6 is a diagram illustrating an example of a component / document association table.

  The point information table stores coordinate information indicating the coordinate position of a point of polygon data of the 3D model. Specifically, as shown in FIG. 7, the point information table indicates a part number, a point number for uniquely identifying a point of polygon data, an X coordinate indicating the X coordinate of the point, and a Y coordinate of the point. The Y coordinate and the Z coordinate indicating the Z coordinate of the point are stored in association with each other. FIG. 7 is a diagram illustrating an example of the point information table.

  The surface information table stores information regarding the surface of the polygon data of the 3D model. Specifically, as shown in FIG. 8, the surface information table includes a part number, a surface number that uniquely identifies the surface of the polygon data, and point numbers 1 to 3 that indicate the point numbers of the surfaces. Store in association with each other. FIG. 8 is a diagram illustrating an example of the surface information table.

  The segment information table stores information on segments (parts). Specifically, as shown in FIG. 9, the segment information table includes a segment number that has been segmented, a segment number that uniquely identifies the segmented component, a surface number after segmentation, and a pre-segmentation And the feature quantity of the segment (part) are stored in association with each other. FIG. 9 is a diagram illustrating an example of the segment information table.

  The text information table stores information related to text information in document information. Specifically, as shown in FIG. 10, the text information table associates the document number, the X and Y coordinates of the character string in the document information, the width and height of the character string, and the contents of the character string. And remember. FIG. 10 is a diagram illustrating an example of the text information table.

  The image information table stores information related to images in document information. Specifically, as shown in FIG. 11, the image information table associates the document number, the X coordinate and the Y coordinate that are the position of the image in the document information, the image width and height, and the image file name. And remember. FIG. 11 is a diagram illustrating an example of the image information table.

  The graphic information table stores information related to a graphic that specifies a part of an image. Specifically, as shown in FIG. 12, the graphic information table includes a document number, an X coordinate and a Y coordinate of a graphic name that specifies a part of an image, a width and a height of a graphic that specifies a part of the image, The figure shape is associated and stored. FIG. 12 is a diagram illustrating an example of the graphic information table.

  The pixel information table stores information related to image pixels in the document information. Specifically, as shown in FIG. 13, the pixel information table stores image numbers and pixel information in association with each other. FIG. 13 is a diagram illustrating an example of the pixel information table. The image feature amount table stores image feature amounts in document information. Specifically, as shown in FIG. 14, the image feature amount table stores an image number and a feature amount in association with each other. FIG. 14 is a diagram illustrating an example of the image feature amount table.

  The viewpoint position information table stores information for setting a viewpoint position for rendering the 3D model. Specifically, as shown in FIG. 15, the viewpoint position information table includes a part number, a segment number, a rotation angle θ and a rotation angle γ that are rotation angles of the 3D model, image information, and surface number information. The feature amount information is stored in association with each other. FIG. 15 is a diagram illustrating an example of the viewpoint position information table.

  The similarity calculation table stores the similarity between the 3D model and the document information image at each viewpoint position. Specifically, as shown in FIG. 16, the similarity calculation table includes the image number, the part number, the segment number, the rotation angle θ and the rotation angle γ, and the similarity between the 3D model and the document information image. Are stored in association with each other. FIG. 16 is a diagram illustrating an example of a similarity calculation table.

  The document reference information table stores the number of times each document information is referenced. Specifically, as shown in FIG. 17, the document reference information table stores a document number and a reference count indicating the number of times the document information is referred to in association with each other. FIG. 17 is a diagram illustrating an example of a document reference information table.

  The query point information table stores the point information of the 3D model of the part that is a search key. Specifically, as shown in FIG. 18, the query point information table corresponds to the point number that uniquely identifies the point of the 3D model, the X coordinate of the point, the Y coordinate of the point, and the Z coordinate of the point. Add and remember. FIG. 18 is a diagram illustrating an example of the query point information table.

  The query / surface information table stores the surface information of the 3D model of a part that is a key for search. Specifically, as shown in FIG. 19, the query / surface information table associates surface numbers that uniquely identify the surfaces of the 3D model with point numbers 1 to 3 that indicate the point numbers of the surfaces. And remember. FIG. 19 is a diagram illustrating an example of a query / surface information table.

  The query segment information table stores information related to a segment of a 3D model of a part that is a key for search. Specifically, as shown in FIG. 20, the query segment information table includes a query segment number of each part that is a search key, a face number after segmentation, a face number before segmentation, and a segment (part ) Are stored in association with each other. FIG. 20 is a diagram illustrating an example of a query segment information table.

  The query segment similarity calculation table stores the similarity between each part of the 3D model of a part that is a key for search and each part that is a search target. Specifically, as shown in FIG. 21, the query segment similarity calculation table includes a query segment number, a part number of each part to be searched, and a segment number of each segment to be searched. The similarity is stored in association with each other. FIG. 21 is a diagram showing an example of a query / segment similarity calculation table.

  The query / segment evaluation value calculation table stores evaluation values for ranking by weighting the degree of similarity between each part of the 3D model of a part serving as a search key and each part to be searched. Specifically, as shown in FIG. 22, the query segment evaluation value calculation table includes a query segment number, a part number of each part to be searched, and a segment number of each segment to be searched. The evaluation values are stored in association with each other. FIG. 22 is a diagram illustrating an example of a query segment evaluation value calculation table.

  The partial document information storage area 14d stores a part / document association table in which the part of the 3D model and the document information are associated with each other. Specifically, as shown in FIG. 23, the partial document information storage area 14d stores a partial / document association table in which document numbers, component numbers, and segment numbers are associated with each other. FIG. 23 is a diagram illustrating an example of a partial / document association table.

  Returning to FIG. 2, the control unit 13 includes an internal memory for storing a program that defines various processing procedures and the necessary data, and executes various processes using these programs. The control unit 13 includes a 3D model dividing unit 13a, a 3D model feature extracting unit 13b, a document image extracting unit 13c, and an instruction mark extracting unit 13d. The control unit 13 includes an edge extraction unit 13e, a rendering unit 13f, a shape feature extraction unit 13g, an image correspondence extraction unit 13h, an instruction mark part correspondence extraction unit 13i, a partial document storage unit 13j, a query acquisition unit 13k, and a search unit 13l. Have

  The 3D model dividing unit 13a divides the 3D model of the part into parts. Specifically, the 3D model dividing unit 13a divides polygon data representing a 3D model of a part for each part using an existing segmentation method. Polygon data is represented by a set of point information and surface information. The point information and the surface information of the polygon data are stored in the point information table and the surface information table (see FIGS. 7 and 8).

  Here, the segmentation method will be specifically described with reference to FIG. FIG. 24 is a diagram for explaining an example of the segmentation technique. As illustrated in FIG. 24, the 3D model dividing unit 13a detects a planar portion of the 3D model and extracts the outline of the 3D model. Then, if there is a plane that can divide the 3D model, the 3D model dividing unit 13a divides the 3D model with the plane, and polygonizes each part.

  Thereafter, the 3D model dividing unit 13a assigns a segment number to each divided part, and stores the segment number after segmentation and the segment number before segmentation in the segment information table.

  The 3D model feature extraction unit 13b extracts a feature amount related to a three-dimensional shape from polygon data of each part. For example, the 3D model feature extraction unit 13b extracts vector information that is a feature amount of a three-dimensional shape as a feature amount related to the three-dimensional shape. Thereafter, the 3D model feature extraction unit 13b stores the extracted feature amount in the segment information table.

  The document image extraction unit 13c extracts an image of a part associated with the document. Specifically, the document image extraction unit 13c analyzes the document information and extracts an image from the document information associated with the part. For example, referring to the example of FIG. 25, the instruction mark extraction unit 13d extracts “image” indicating image information from the document information. FIG. 25 is a diagram illustrating an example of a document in the HTML file format.

  When the document image extraction unit 13c extracts “image” indicating the image information, the X coordinate “0” and the Y coordinate “40” where the image information is located, the image size (the width of the image and the height of the image), The image file name is acquired and stored in the image information table. The image size can be acquired from the header of the image file. In the above-described example, the HTML file format has been described. However, the same processing is performed for documents in other formats (PDF, PowerPoint, Word, etc.).

  The instruction mark extraction unit 13d extracts an instruction mark that specifies a part of the image from the extracted images. Specifically, the instruction mark extraction unit 13d analyzes the document information and extracts an instruction mark from the document information associated with the part. For example, referring to the example of FIG. 25, the instruction mark extraction unit 13d extracts “oval” and “line” indicating a graphic.

  When the instruction mark extracting unit 13d extracts “oval” and “line” indicating the graphic, the X coordinate and the Y coordinate where the graphic is located, the graphic size (the graphic width and the graphic height), and the graphic shape are obtained. Acquired and stored in the graphic information table.

  Subsequently, the instruction mark extraction unit 13d refers to the graphic information table, and if graphic information is included in the document itself, all of the graphic information such as an ellipse and an arrow are used as instruction mark candidates. Even if the document itself does not include graphic information, it can be extracted by a technique such as Hough conversion if it is described as a part of the graphic in the graphic.

  Here, the instruction mark extraction unit 13d determines whether the instruction mark candidate is a correct instruction mark based on the shape and position of the instruction mark. Specifically, when the graphic shape of the instruction mark candidate is an ellipse, the instruction mark extraction unit 13d determines whether the position of the graphic of the instruction mark is at a position overlapping the image position as illustrated in FIG. . FIG. 26 is a diagram illustrating an example of an instruction mark.

  This determination can be easily made from the coordinate positions of the graphic information table and the image information table. In the following explanation, the position, width, and height of the instruction mark candidate are represented by “shape_x, shape_y, shape_width, shape_height”, and the position, width, and height of the image are represented by “image_x, image_y, image_width, image_height”. When the following expressions (1) to (4) are true, the instruction mark is determined.

  Here, an offset value (offset_x, offset_y) is specified on the assumption that the figure slightly protrudes from the image position. This value may be fixed or a relative value with respect to the image size. When the graphic shape of the instruction mark candidate is an arrow, it is determined that the tip of the arrow overlaps the image as shown in FIG. When the following formulas (5) to (8) are true, the indication mark is used. Here, the tip (end) of the arrow is “shape_x, shape_y”. (When obtaining from the graphic information table, “shape_x” = [x coordinate] + [width], “shape_y” = [y coordinate] + [height]. Here, the same offset value (offset_x as in the case of an ellipse) , Offset_y).

  Thereafter, as shown in FIG. 27, the instruction mark extraction unit 13d stores information on the extracted instruction mark. FIG. 27 is a diagram illustrating an example of instruction mark information. Then, the instruction mark extraction unit 13d stores the instruction mark and the image in association with each other as illustrated in FIG. FIG. 28 is a diagram illustrating an example of image information with an instruction mark. An instruction mark number that is a number that does not overlap is assigned to the instruction mark. Also, image numbers that are non-overlapping numbers are assigned to image information with instruction marks.

  Returning to the description of FIG. 2, the edge extraction unit 13 e, the rendering unit 13 f, the shape feature extraction unit 13 g, and the image correspondence extraction unit 13 h perform various processes in order to align the image and the 3D model in the document information. The image here is stored in image information with an instruction mark, and the correspondence between the image and the 3D model of the component can be acquired from the component table and the component / document association table.

  The image format is a two-dimensional array of Width × Height. Each element of the array normally has three elements R, G, and B as shown in FIG. 29. However, in the case of an edge or the like, since it is often targeted for monochrome, as shown in FIG. The brightness value (I) may be one. These pieces of pixel information are stored in association with image numbers in the pixel information table. FIG. 29 is a diagram illustrating an arrangement format of pixels of a color image. FIG. 30 is a diagram illustrating a pixel arrangement format of a monochrome image.

  In addition, the edge extraction unit 13e extracts an object part from an image in advance using an existing method such as background removal in order to improve the accuracy by aligning the size of the rendered image and the image. Then, the rendering unit 13f may obtain a size of the object part, perform trimming in accordance with the size, and perform a method of rendering the 3D model in accordance with the trimming.

  In the background removal of an image, when the image is a light background or the like, the color of the surrounding pixels is checked from the pixels in contact with the edge of the image, and the same color portion (or a similar color portion with some error allowed) ) Only when it is determined as a background and the same color portion around it is further determined from the range determined as the background.

  As a result, the same color portion around the object shown in the image can be extracted, so by extracting the smallest rectangle surrounding the other range, the circumscribed rectangle of the object shown in the image can be extracted. Image information for only the range is generated. Also for a rendering image of a 3D model to be described later, a rectangle that circumscribes the object is similarly extracted after rendering. The accuracy can be improved by extracting the feature amount between the two images and calculating the similarity.

  As described above, the edge extraction unit 13e, the rendering unit 13f, the shape feature extraction unit 13g, and the image correspondence extraction unit 13h perform various processes in order to align the image and the 3D model in the document information. Here, with reference to FIG. 31, the overall flow of the alignment processing between the image in the document information and the 3D model for the edge extraction unit 13e, the rendering unit 13f, the shape feature extraction unit 13g, and the image correspondence extraction unit 13h will be described. FIG. 31 is a diagram illustrating processing for aligning an image and 3D model in document information.

  As shown in FIG. 31, the edge extraction unit 13e extracts edges from an image in a document, and extracts a shape feature amount of the image in the document. In addition, the rendering unit 13f sets the viewpoint position of the 3D model and generates an edge rendering image viewed from each set viewpoint position. Then, the shape feature extraction unit 13g extracts the shape feature amount of each edge rendering image.

  Then, the image correspondence extracting unit 13h calculates the similarity between the shape feature amount of the image in the document and the shape feature amount of each edge rendering image, and selects the viewpoint position with the highest similarity. Hereinafter, each process of the edge extraction unit 13e, the rendering unit 13f, the shape feature extraction unit 13g, and the image correspondence extraction unit 13h will be described in detail.

  The edge extraction unit 13e extracts an edge from an image in the document and extracts a shape feature amount of the image in the document. Specifically, the edge extraction unit 13e uses an edge extraction operator as an edge feature extraction method, directly using a shape feature amount extraction method for edges, or directly using a shape feature amount, such as wavelet transform. Use the extraction method. In a method using wavelet transform, an image is subjected to wavelet transform, and a frequency variation at each position of the image is extracted and used as a feature amount.

  In a method using an edge extraction operator, a Laplacian matrix is used as an edge extraction method. In this technique, a 3 × 3 matrix called a Laplacian matrix is convolved with each pixel to extract an edge amount at each pixel. The edge amount can be expressed as a two-dimensional array of integers or real numbers having the same vertical and horizontal sizes as the image size.

  When the state in which the pixel columns are arranged is viewed as a two-dimensional matrix, the value p (x, y) is present, and when the convolution matrix is q (x, y), the transformed matrix p ′ (x, y y) is expressed by the following equation (9). Further, as the Laplacian matrix, the following matrix (10) is used. The edge extraction result is a two-dimensional array having the same size as the image as shown in FIG. FIG. 32 is a diagram for explaining the edge extraction result.

  The edge extraction unit 13e extracts an edge feature amount by using a method such as Grid Fourier as a feature extraction method for the edge extraction result. In this method, a histogram of edge amounts is extracted for each grid obtained by dividing an image into a grid, and the result of performing the two-dimensional Fourier transform is used as a feature amount. This makes it possible to obtain a feature quantity that is invariant to two-dimensional rotation. The result of the extracted feature value is a one-dimensional fixed-length vector value.

  The edge extraction unit 13e extracts a one-dimensional fixed length vector value as a result of the feature amount. Then, the edge extraction unit 13e extracts the result of the extracted feature amount for each image stored in the image information with the instruction mark, and stores it in the image feature amount table.

  The rendering unit 13f sets the viewpoint position for rendering the 3D model. For example, the edge extraction unit 13e sets the position and orientation of the camera for viewing the 3D model from around as the viewpoint position.

  Here, the determination of the camera position and orientation can be performed by the following method. Here, when the center position of the 3D model is (0, 0, 0) and the camera rotates around it, the camera position c, orientation f, and upper vector u are rotation angles θ and γ, distance Is represented by the following formulas (11) and (12).

  Here, the rendering unit 13f sets the rotation angles θ and γ at appropriate intervals, designates them in a double loop with equal intervals, and sets them in the viewpoint position information table. In the example of FIG. 15, it designates at intervals of 30 degrees. At each angle, image information and feature amount information are stored in the viewpoint position information table.

  Then, the rendering unit 13f generates a rendering image viewed from the set viewpoint position. Here, an example of a rendering image generation processing method will be described. When each point on the 3D model is projected on the screen, when the camera position is c, the camera orientation is f, and the camera upper vector is u, the point p on the 3D model is Projected to the point p2 on the screen represented by the equations (13) and (14).

  The rendering unit 13f draws each surface on the polygon at a position obtained by linearly interpolating the polygon points. Thereby, an image in which each polygon is mapped on a two-dimensional plane can be generated. In addition, when the positions of p2 overlap, the rendering unit 13f preferentially draws the one having a small z value (the one in front). Further, in order to render only the edge, the rendering unit 13f may render only an acute angle or a contour edge.

  Here, the angle formed by the two surfaces can be obtained by calculating the inner product of the normal vectors. The (a · b = | a || b | cos θ) contour is obtained because one surface faces the camera direction, and one surface does not face the camera direction. When the normal vector of the surface facing the camera direction is a and the normal vector not facing is b, the line that satisfies the conditions of “f · a <0” and “f · a> 0” is outlined. It corresponds to.

  The rendering unit 13f holds information indicating which position in the 3D model each pixel corresponds to. For example, the rendering unit 13f prepares a two-dimensional array corresponding to the number of pixels of the image, and uses the surface number of the polygon corresponding to the pixel calculated in the above calculation (surface number before segmentation) as the surface number information, Save it in a table.

  In the example of FIG. 33, the pixel corresponding to surface number 0, the pixel corresponding to surface number 1, and the pixel not corresponding to any surface (-1 pixel) are set. FIG. 33 is a diagram illustrating an example of the surface number information. This array is obtained by initializing each element as a pixel (-1) that does not correspond to any surface first, and by replacing the coordinate position when the surface is converted into dimensional coordinates with the corresponding surface number in the calculation at the time of drawing. create. Although the surface number is set here, the segment number may be set using a segment information table.

  The shape feature extraction unit 13g extracts the shape feature amount of each edge rendering image. Specifically, the shape feature extraction unit 13g can extract an edge feature amount by using a method such as Grid Fourier as a feature extraction method for the edge extraction result, similarly to the above-described processing. When the image is generated as a normal rendering image, a wavelet transform feature amount or the like can be used. When only the edge is rendered, an edge feature amount such as Grid Fourier can be used. Since the extraction result is a vector, the result is stored in the viewpoint position information table.

  The image correspondence extracting unit 13h calculates the similarity between the shape feature amount of the image in the document and the shape feature amount of each edge rendering image, and selects the viewpoint position with the highest similarity. Specifically, the image correspondence extracting unit 13h calculates the Euclidean distance between the shape feature amount vector of the image in the document and the shape feature amount vector of each edge rendering image. That is, the shape feature amount can be expressed as a vector, and the distance between the vectors having the same dimensionality can be calculated by the Euclidean distance. Assuming that feature vectors f (i) and g (i) are m, the Euclidean distance can be calculated by the following equation (15).

  That is, the image correspondence extraction unit 13h determines that the similarity is higher as the distance between the features is smaller. Based on the information in the viewpoint position information table, the similarity between the image feature amount corresponding to each rotation angle extracted by the shape feature extraction unit 13g and the image feature amount extracted by the edge extraction unit 13e is expressed by (15). Calculate with the formula and extract one field of the viewpoint position information table with the smallest value. Thereby, the image correspondence extraction unit 13h can acquire corresponding image information and face number information. For example, in the example of the similarity calculation table illustrated in FIG. 16, the image correspondence extraction unit 13 h selects a field with the rotation angle θ = 0 and the rotation angle γ = 60 having the smallest similarity value.

  The instruction mark part correspondence extracting unit 13i selects a part of the 3D model corresponding to the part of the image specified by the symbol. Specifically, as shown in FIG. 34, the instruction mark part correspondence extraction unit 13i extracts the area within the instruction mark range in each part of the segmented 3D model, and calculates the area within the part area. Calculate the percentage. Then, the instruction mark part correspondence extracting unit 13i selects the part with the highest ratio. FIG. 34 is a diagram illustrating processing for selecting a portion of the 3D model corresponding to the position of the instruction mark.

  For example, in the example of FIG. 34, the instruction mark part correspondence extracting unit 13i includes the entire area “9” within the instruction mark range among the places 1 to 5 indicating each part, and is within the range in the component area. The portion of the place “1” having the highest area ratio of “100%” is selected.

  Here, the process of selecting the part of the 3D model corresponding to the part of the image specified by the symbol will be described in detail. The instruction mark part correspondence extraction unit 13i acquires the type and position coordinates of the instruction mark from the instruction mark information stored by the instruction mark extraction unit 13d, and converts the instruction mark part correspondence extraction unit 13i into the image of the viewpoint (rotation angle) selected by the image correspondence extraction unit 13h. On the other hand, it is determined whether it is within the range of the instruction mark.

  Here, when the type of the instruction mark is an ellipse, it can be determined that the image range corresponding to the instruction mark is within the range of the ellipse. For example, when the center of the ellipse is (cx, cy), the radius in the x-axis direction, and the radius in the y-axis direction are a and b, it is determined whether the (x, y) coordinates are in the range (16 ).

  When the type of the instruction mark is an arrow, the space ahead of the arrow can be set as an image range corresponding to the instruction mark. The space may be circular, and its radius may be fixed, or may vary depending on the length of the arrow. For example, if the tip of the arrow is (x1, y1) and the root is (x2, y2), the center coordinates (cx, cy) of the circle and the radius (r) of the circle are the following (17) to (19) It can be done with a formula. In this case, whether or not the (x, y) coordinates are in the range can be determined by the following equation (20).

  As a result of the determination described above, the instruction mark part correspondence extraction unit 13i stores instruction mark range pixel information that is information regarding pixels within the range of the instruction mark, as illustrated in FIG. As shown in FIG. 35, the instruction mark part correspondence extracting unit 13i includes, as instruction mark range pixel information, an instruction mark number, an X coordinate and a Y coordinate of a pixel included in the instruction mark, a surface number, a part number, and a segment number, respectively. Remember. FIG. 35 is a diagram illustrating an example of instruction mark range pixel information.

  Here, since the instruction mark portion correspondence extracting unit 13i has obtained an array of surface numbers corresponding to each pixel (surface number information illustrated in FIG. 33) by the rendering unit 13f, refer to the surface number information. The surface number corresponding to the pixel can be acquired. In the surface number information, since -1 is set when there is no corresponding surface number, the value is checked. If the surface number is other than -1, the surface number is set in the segment information table (see FIG. 9). Get the corresponding segment number. When the rendering unit 13f sets a segment number corresponding to each pixel, the instruction mark part correspondence extracting unit 13i can directly acquire the segment number.

  Subsequently, the instruction mark part correspondence extracting unit 13i adds the areas of the parts corresponding to the pixels. The area referred to here is the number of pixels corresponding to the image range corresponding to the indication mark in each segment number portion. In addition, since the instruction mark part correspondence extracting unit 13i knows the segment number corresponding to each pixel, it can acquire the number of pixels corresponding to the instruction mark corresponding to each segment number. Further, the instruction mark portion corresponding extraction unit 13i obtains the total number of pixels by referring to pixel information (see FIGS. 29 and 30) and converting the number of pixels related to each segment.

  Then, as shown in FIG. 36, the instruction mark part correspondence extracting unit 13i, for each part (segment) included in the instruction mark, indicates the number of pixels included in the instruction mark, the total number of pixels of the part, and the instruction mark for the total number of pixels. The ratio of the number of pixels included in is stored. FIG. 36 is a diagram illustrating an example of pixel histogram information. Specifically, the instruction mark part correspondence extracting unit 13i instructs the instruction mark number, the part number, the segment number of the part included in the instruction mark, the number of pixels included in the instruction mark, the total number of pixels in the part, and the total number of pixels. The ratio of the number of pixels included in the mark is stored in association with each other.

  Then, the instruction mark part corresponding extraction unit 13i selects a part of the segment number having the highest ratio of the number of pixels included in the instruction mark to the total number of pixels. Note that the segment number having the highest number of pixels may be selected.

  The partial document storage unit 13j associates the selected 3D model portion with the document associated with the part image, and stores them in the partial document information storage area 14d. Specifically, the partial document storage unit 13j associates the document number corresponding to the instruction mark number, the partial number corresponding to the document number, and the part of the 3D model selected by the instruction mark part correspondence extracting unit 13i. And stored in the part / document association table (see FIG. 23).

  Thus, as shown in FIG. 37, the information providing apparatus 10 matches the specified portion with the 3D model when a portion is specified by a red circle or the like in the image in the document information such as the failure information. To estimate the orientation and position. Then, the information providing apparatus 10 associates only the corresponding part with the document information and stores it in the table. For this reason, the user can automatically search for document information related to the part portion without explicitly registering it in the database. FIG. 37 is a diagram illustrating processing for matching an image in a document with a 3D model portion.

  Returning to the description of FIG. 2, the query acquisition unit 13k extracts a part from the part under design and calculates a feature amount of the part. Specifically, the query acquisition unit 13k acquires a 3D model of a component that is a search key during the design of the component using CAD. The query acquisition unit 13k acquires the 3D model of the part when only one part is edited with CAD, and the currently selected part when editing a plurality of parts. The 3D model is acquired.

  The query acquisition unit 13k acquires point information and surface information for the component from the query point information table (see FIG. 18) and the query / surface information table (see FIG. 19). And the query acquisition part 13k extracts a part from the components in design, and extracts the partial feature-value. Here, in order to extract a part from a part under design, the above-described segmentation technique can be used. Similarly, the feature extraction method described above can be used to extract the partial feature amount. Here, a query segment number that is a non-overlapping number is added to each part, and feature quantities are extracted from all segments of the query and stored in the query segment information table (see FIG. 20).

  The search unit 13l acquires the feature amount of the part of the 3D model, calculates the similarity with the 3D model under design, calculates the evaluation value of the ranking, and documents related to the part of the 3D model under design Search and display information.

  Specifically, the search unit 13l performs similarity calculation between the search target segment and all segments of the 3D model under design, and stores the similarity in the query / segment similarity calculation table. As described above, the search unit 13l may be all segments as described above, and may be limited to only a portion intended by the user. For the calculation of the similarity, the above-described method for calculating the distance between vectors can be used. It is assumed that a smaller similarity (distance) value is more similar.

  Then, the search unit 13l searches for a part having a feature amount having a similarity (distance equal to or less than a threshold) equal to or greater than a predetermined threshold with respect to the result of similarity calculation. For example, the search unit 13l searches for a portion having a feature amount with a similarity of 0.2 or less. For example, as shown in FIG. 38, the search unit 131 searches for a 3D model and related documents similar to each part of the 3D model under design, and acquires the number of links and references of the searched 3D model and related documents. To do. FIG. 38 is a diagram for explaining search processing.

  The search unit 13l can perform processing on all segments, but may be limited to only the portion intended by the user. For example, it is conceivable to perform a search only on a part that is currently visible from the viewpoint position, or perform a search only on a part under the mouse cursor.

  Then, the search unit 13l calculates an evaluation value for ranking the portion of the search result. As shown in the equation (21), the evaluation value is calculated using the weighted evaluation value and stored in the query segment evaluation value calculation table (see FIG. 22). Here, as the maximum distance, the above-described distance threshold may be used. The number of documents is the number of documents linked to the same document, and is the number of documents hit in the above search. The reference count is the number of times the document is displayed by clicking the document title, and can be acquired from the document information table. Note that the distance may be used as it is as an evaluation value for ranking.

  Thereafter, the search unit 13l can perform ranking by sorting the query segment evaluation value calculation table in descending order by evaluation value. As shown in FIG. 39, the search unit 13l displays document titles on a task bar or the like in order from documents ranked higher in the ranking. FIG. 39 is a diagram illustrating a screen display example of search results.

  Then, if the user is interested in the content and wants to know more, you can click on the title part of the document. As shown in FIG. 40, the information providing apparatus 10 monitors the user's operation. When the user clicks, the original document is displayed, and at the same time, the reference count of the document reference information table is increased by one. When the user does not click, the next data of the above ranking is displayed. FIG. 40 is a diagram for describing processing for increasing reference information.

[Processing by information provider]
Next, processing performed by the information providing apparatus 10 according to the second embodiment will be described with reference to FIGS. FIGS. 41 to 45 are flowcharts illustrating the processing operation of the information providing apparatus 10 according to the second embodiment.

  As shown in the figure, the information providing apparatus 10 divides a part into parts (step S101), and extracts a feature amount related to a three-dimensional shape from the part (step S102). Then, the information providing apparatus 10 performs processing for associating an image with an instruction mark (detailed later using FIG. 42) (step S103), and aligning the image and the 3D model (detailed later using FIG. 43). (Step S104).

  Then, the information providing apparatus 10 performs processing for detecting a portion corresponding to the instruction mark (detailed later with reference to FIG. 44) (step S105), and stores the document and the portion in association with each other (step S106).

  Here, the process of associating an image with an instruction mark will be described with reference to FIG. As shown in FIG. 42, the information providing apparatus 10 extracts an image from a document associated with a part (step S201), detects an indication mark candidate graphic (step S202), and determines the position of the image and the indication mark candidate. A determination is made (step S203). If there is an instruction mark corresponding to the image (Yes at Step S204), the information providing apparatus 10 stores the instruction mark in association with the image (Step S205).

  If there is no instruction mark corresponding to the image (No at Step S204), the information providing apparatus 10 proceeds to Step S206 without storing the instruction mark in association with the image. The information providing apparatus 10 determines whether all instruction mark candidates have been processed (step S206). If all instruction mark candidates have been processed (Yes in step S206), the information providing apparatus 10 determines whether all images have been processed. (Step S207). As a result, when all of the images have been processed (Yes at Step S207), the information providing apparatus 10 ends the process.

  Further, the information providing apparatus 10 returns to S201 and performs the process when not processing all the instruction mark candidates (No at Step S206), and when not processing all the images (No at Step S207). repeat.

  Here, the alignment processing of the image and the 3D model will be described with reference to FIG. As shown in FIG. 43, as shown in FIG. 43, the information processing apparatus 10 extracts edges from the image (step S301), and extracts edge feature amounts (step S302). Then, the information processing apparatus 10 sets the viewpoint position of the 3D model (step S303), edge-renders the 3D model (step S304), and associates and saves the 3D model plane, segmentation, and pixels (step S305).

  The information processing apparatus 10 extracts edge feature amounts (step S306), calculates and stores the similarity between the two edge extraction results (step S307), and determines whether rendering has been performed from all viewpoints (step S308). As a result, when rendering is performed with all viewpoints (Yes at Step S308), the information processing apparatus 10 extracts the viewpoint with the highest similarity (Step S309). The process ends. If the information processing apparatus 10 has not rendered all the viewpoints (No at Step S308), the information processing apparatus 10 returns to Step S303 and repeats the processes at Steps S303 to S308.

  Here, a process of detecting a portion corresponding to the instruction mark will be described with reference to FIG. As shown in FIG. 44, the information processing apparatus 10 extracts an image range corresponding to the instruction mark (step S401), and acquires a segmentation number corresponding to the pixel corresponding to the instruction mark (step S402).

  Then, the information processing apparatus 10 determines whether there is a corresponding surface (step S403). If there is a corresponding surface (Yes in step S403), the area of the portion corresponding to the pixel is added (step S404). Subsequently, the information processing apparatus 10 determines whether all the pixels have been checked (step S405), and when all the pixels have been checked (step S405 affirmative), selects the portion with the largest ratio (step S406). ), The process is terminated. In addition, when all the pixels are not checked (No at Step S405), the information processing apparatus 10 returns to Step S402 and repeats the process.

  Next, the search process of the information providing apparatus will be described with reference to FIG. As shown in FIG. 45, the information providing apparatus 10 extracts a part under design (step S501), extracts a part from the part under design, and extracts a partial feature amount (step S502). Then, the information providing apparatus 10 determines whether all parts have been extracted (step S503). If all parts have not been extracted (No in step S503), the information providing apparatus 10 returns to step S501.

  In addition, when all the parts are extracted (Yes at Step S503), the information providing apparatus 10 acquires the partial feature amount of the part from the database and calculates the similarity (Step S504). Then, the information providing apparatus 10 determines whether the similarity is equal to or higher than the threshold (step S505). If the similarity is equal to or higher than the threshold (Yes in step S505), the information providing apparatus 10 calculates a ranking evaluation value (step S506). .

  Then, the information providing apparatus 10 determines whether all the components have been checked (step S507). If all the components have not been checked (No in step S507), the information providing apparatus 10 returns to step S504. Further, when all the components are checked (Yes at Step S507), the information providing apparatus 10 performs ranking and extracts a portion having a higher rank (Step S508).

  Then, the information providing apparatus 10 presents the title of the document (step S509), performs a user input check (step S510), and determines whether the user has clicked on the title of the document (step S511). As a result, when the information providing apparatus 10 determines that the user has clicked on the title of the document (Yes in step S511), the information providing apparatus 10 presents the document corresponding to the clicked title (step S512), and increments the reference count by one. Increase (step S513).

[Effect of Example 2]
As described above, the information providing apparatus 10 divides the 3D model of the part stored in the 3D model information storage area 14a into parts, and the part of the part added to the document from the document stored in the document information storage area 14b. An image is extracted, and a symbol specifying a part of the image is extracted from the extracted image. When the symbol is extracted, the information providing apparatus 10 selects the 3D model portion corresponding to the image portion specified by the symbol, and associates the selected 3D model portion with the image of the part. The document is associated and stored in the partial document information storage area 14d. For this reason, the information providing apparatus 10 stores the 3D model portion and the document in association with each other, and enables the user to search for the document related to the part portion that the user wants to refer to, thereby effectively realizing the knowledge information. It is possible.

  Further, according to the second embodiment, when the extracted instruction mark is extracted, the similarity between the image portion specified by the instruction mark and the model portion is calculated, and the instruction mark is determined according to the similarity. The part of the 3D model corresponding to the part of the image specified by is selected. For this reason, it is possible to appropriately select a portion of the 3D model corresponding to the portion of the image specified by the instruction mark.

  Further, according to the second embodiment, a part similar to a predetermined part is acquired from the 3D model storage unit 14a, and a document associated with the acquired part is searched from the partial document information storage unit 14d. It is possible to appropriately search for documents related to.

  Further, according to the second embodiment, when there are a plurality of documents retrieved from the partial document information storage unit 14d, evaluation values for determining the display order of each document are calculated, and the documents are sorted in descending order of evaluation values. indicate. For this reason, it is possible to display documents related to the part portion in order from the most important one.

  Although the embodiments of the present invention have been described so far, the present invention may be implemented in various different forms other than the embodiments described above. Therefore, another embodiment included in the present invention will be described below as a third embodiment.

(1) System Configuration, etc. Further, each component of each illustrated apparatus is functionally conceptual and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the 3D model dividing unit 13a and the 3D model feature extracting unit 13b may be integrated.

(2) Program The various processes described in the first and second embodiments can be realized by executing a program prepared in advance on a computer system such as a personal computer or a workstation. Therefore, in the following, an example of a computer that executes an information providing program having the same function as in the first and second embodiments will be described with reference to FIG. FIG. 46 is a diagram illustrating a computer that executes an information providing program.

  As shown in FIG. 46, a computer 600 as an information providing apparatus is configured by connecting a RAM 610, a CPU 620, an HDD 630, and an input / output interface 640 with a bus or the like.

  The HDD 630 stores information necessary for the CPU 620 to execute various processes. The RAM 610 temporarily stores various information. The CPU 620 executes various arithmetic processes.

  As shown in FIG. 46, the HDD 630 stores in advance an information providing program 611 that exhibits the same function as each processing unit of the interrupt controller shown in the first and second embodiments. The information providing program 611 may be appropriately distributed and stored in a storage unit of another computer that is communicably connected via a network.

  Then, the CPU 620 reads this information provision program 611 from the HDD 630 and expands it in the RAM 610. The information providing program 611 reads various data from the HDD 630, expands the data in an area allocated to itself in the RAM 610, and executes various processes based on the expanded data and the like.

  The information providing program 611 is not necessarily stored in the HDD 630 from the beginning. For example, a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, and an IC inserted into the computer 600 are not necessary. Each program is stored in a “portable physical medium” such as a card, and “another computer (or server)” connected to the computer 600 via a public line, the Internet, a LAN, a WAN, or the like. The computer 600 may read out and execute each program from these.

  The following appendices are further disclosed with respect to the embodiments including the first to third embodiments.

(Supplementary note 1) a 3D model storage unit that stores a 3D model of a part;
A document information storage unit for storing a document related to the part to which the image of the part is added;
A dividing unit that divides the 3D model of the part stored in the 3D model storage unit into parts;
An image extraction unit for extracting an image of a part added to the document from the document stored in the document information storage unit;
A specific symbol extraction unit for extracting a symbol for specifying a part of the image from the image extracted by the image extraction unit;
When a symbol is extracted by the specific symbol extraction unit, a partial selection unit that selects a part of the 3D model corresponding to the part of the image specified by the symbol;
A partial document storage unit that associates a portion of the 3D model selected by the partial selection unit with a document associated with the image of the component and stores the document in a partial document information storage unit;
An information providing apparatus comprising:

(Supplementary Note 2) When a symbol is extracted by the specific symbol extraction unit, the partial selection unit calculates a similarity between an image portion specified by the symbol and a 3D model portion, and the similarity is calculated. The information providing apparatus according to appendix 1, wherein a part of the 3D model corresponding to the part of the image specified by the symbol is selected.

(Additional remark 3) It further has a search part which acquires the part similar to a predetermined part from the said 3D model memory | storage part, and searches the document linked | related with the acquired part from the said partial document information storage part, It is characterized by the above-mentioned. The information provision apparatus according to Supplementary Note 1 or 2.

(Supplementary Note 4) When there are a plurality of documents searched from the partial document information storage unit, the search unit calculates an evaluation value for determining the display order of each document, and the evaluation value is in descending order. The information providing apparatus according to any one of appendices 1 to 3, wherein the document is displayed.

(Supplementary Note 5) A dividing step of dividing the part 3D model stored in the 3D model storage unit that stores the part 3D model into parts;
An image extracting step of extracting an image of the component added to the document from a document stored in a document information storage unit that stores a document related to the component to which the image of the component is added;
A specific symbol extraction step for extracting a symbol for specifying a part of the image from the image extracted by the image extraction step;
A partial selection step of selecting a portion of the 3D model corresponding to a portion of the image specified by the symbol when the symbol is extracted by the specific symbol extraction step;
A partial document storage step of storing the portion of the 3D model selected by the partial selection step and the document associated with the image of the part in association with each other and storing the partial document information in the partial document information storage unit;
An information providing method characterized by including

(Supplementary Note 6) When the symbol is extracted by the specific symbol extraction step, the partial selection step calculates the similarity between the image portion specified by the symbol and the portion of the 3D model. The information providing method according to appendix 5, wherein a part of the 3D model corresponding to the part of the image specified by the symbol is selected.

(Supplementary note 7) The method further includes a retrieval step of acquiring a part similar to a predetermined part from the 3D model storage unit and searching the partial document information storage unit for a document associated with the acquired part. The information provision method according to appendix 5 or 6.

(Supplementary note 8) When there are a plurality of documents searched from the partial document information storage step, the search step calculates an evaluation value for determining the display order of each document, and the evaluation value is in descending order. The information providing method according to any one of appendices 5 to 7, wherein the document is displayed.

(Supplementary Note 9) A division procedure for dividing the part 3D model stored in the 3D model storage unit that stores the part 3D model into parts,
An image extraction procedure for extracting an image of a component added to the document from a document stored in a document information storage unit that stores a document related to the component to which the image of the component is added;
A specific symbol extraction procedure for extracting a symbol for specifying a part of the image from the images extracted by the image extraction procedure;
When a symbol is extracted by the specific symbol extraction procedure, a partial selection procedure for selecting a portion of the 3D model corresponding to a portion of the image specified by the symbol;
A partial document storage procedure for storing a portion of the 3D model selected by the partial selection procedure and a document associated with the image of the part in the partial document information storage unit;
An information providing program for causing a computer to execute.

(Additional remark 10) When the symbol is extracted by the specific symbol extraction procedure, the partial selection procedure calculates the similarity between the part of the image specified by the symbol and the part of the 3D model. The information providing program according to appendix 9, wherein a part of the 3D model corresponding to the part of the image specified by the symbol is selected accordingly.

(Supplementary Note 11) Acquiring a part similar to a predetermined part from the 3D model storage procedure, and causing the computer to further execute a search procedure for searching a document associated with the acquired part from the partial document information storage unit. The information providing program according to appendix 9 or 10, characterized by:

(Supplementary Note 12) When there are a plurality of documents searched from the partial document information storage procedure, the search procedure calculates an evaluation value for determining the display order of each document, and the evaluation value is in descending order. The information providing program according to any one of appendices 9 to 11, wherein the document is displayed.

DESCRIPTION OF SYMBOLS 1, 10 Information provision apparatus 11 Input part 12 Output part 13 Control part 13a 3D model division part 13b 3D model feature extraction part 13c Document image extraction part 13d Instruction mark extraction part 13e Edge extraction part 13f Rendering part 13g Shape feature extraction part 13h Image Correspondence extraction unit 13i Instruction mark part correspondence extraction unit 13j Partial document storage unit 13k Query acquisition unit 13l Search unit 14 Storage unit 14a 3D model information storage region 14b Document information storage region 14c Table storage region 14d Partial document information storage region

Claims (6)

A 3D model storage unit for storing a 3D model of the component;
A document information storage unit for storing a document related to the part to which the image of the part is added;
A dividing unit that divides the 3D model of the part stored in the 3D model storage unit into parts;
An image extraction unit for extracting an image of a part added to the document from the document stored in the document information storage unit;
A specific symbol extraction unit for extracting a symbol for specifying a part of the image from the image extracted by the image extraction unit;
When a symbol is extracted by the specific symbol extraction unit, a partial selection unit that selects a part of the 3D model corresponding to the part of the image specified by the symbol;
A partial document storage unit that associates a portion of the 3D model selected by the partial selection unit with a document associated with the image of the component and stores the document in a partial document information storage unit;
An information providing apparatus comprising:   When the symbol is extracted by the specific symbol extraction unit, the partial selection unit calculates the similarity between the image portion specified by the symbol and the 3D model portion, and according to the similarity, The information providing apparatus according to claim 1, wherein a part of the 3D model corresponding to the part of the image specified by the symbol is selected.   2. The apparatus according to claim 1, further comprising a search unit that acquires a part similar to a predetermined part from the 3D model storage unit and searches the partial document information storage unit for a document associated with the acquired part. Or the information provision apparatus of 2.   When there are a plurality of documents retrieved from the partial document information storage unit, the retrieval unit calculates an evaluation value for determining the display order of each document, and displays the documents in descending order of the evaluation value. The information providing apparatus according to claim 1, wherein the information providing apparatus is an information providing apparatus. A dividing step of dividing the 3D model of the part stored in the 3D model storage unit that stores the 3D model of the part into parts;
An image extracting step of extracting an image of the component added to the document from a document stored in a document information storage unit that stores a document related to the component to which the image of the component is added;
A specific symbol extraction step for extracting a symbol for specifying a part of the image from the image extracted by the image extraction step;
A partial selection step of selecting a portion of the 3D model corresponding to a portion of the image specified by the symbol when the symbol is extracted by the specific symbol extraction step;
A partial document storage step of storing the portion of the 3D model selected by the partial selection step and the document associated with the image of the part in association with each other and storing the partial document information in the partial document information storage unit;
An information providing method characterized by including A dividing procedure for dividing the 3D model of the part stored in the 3D model storage unit that stores the 3D model of the part into parts;
An image extraction procedure for extracting an image of a component added to the document from a document stored in a document information storage unit that stores a document related to the component to which the image of the component is added;
A specific symbol extraction procedure for extracting a symbol for specifying a part of the image from the images extracted by the image extraction procedure;
When a symbol is extracted by the specific symbol extraction procedure, a partial selection procedure for selecting a portion of the 3D model corresponding to a portion of the image specified by the symbol;
A partial document storage procedure for storing a portion of the 3D model selected by the partial selection procedure and a document associated with the image of the part in the partial document information storage unit;
An information providing program for causing a computer to execute.

Images