JP7015130B2 - Data processing equipment - Google Patents

Description

The present invention relates to a technique for providing information for managing a structure to a manager of the structure.

For the manager of a structure, information such as the start date and expiration date of the structure to be managed (hereinafter referred to as "management information") is used to judge the necessity of inspection and the necessity of replacement of parts. Helps me. Therefore, in order to facilitate the management work of the structure by the manager, a technique for providing management information to the manager has been proposed.

For example, Patent Document 1 describes an actual image of a device to be maintained and managed by a camera, a display image of CAD data of the device, a display image of CAD data of the contents contained in the device, and the relevant image. A facility maintenance management system that superimposes and displays a display image of maintenance management data of the contents is described. According to the equipment maintenance management system described in Patent Document 1, when the administrator captures an image of the equipment to be maintained and managed with a camera, the contents of the maintenance management data of the contents stored in the equipment are displayed to the administrator. To.

Japanese Unexamined Patent Publication No. 2013-149042

For those with attributes included in the management information of the structure, the change over time is important. For example, when the rate of aging of strength differs depending on the part of the structure, if the strength (an example of the attribute) at each of the past multiple periods can be known for each of the plurality of parts constituting the structure, the administrator. Can correctly judge the necessity of repair of each part. However, if the attributes for each of the plurality of parts in the past are available for each of the plurality of parts, the amount of information thereof becomes enormous, and it is not easy to find the attributes for the parts required by the administrator.

In view of the above circumstances, the present invention provides a technique that enables a manager of a structure to easily know the attributes that change with time with respect to each of a plurality of parts constituting the structure.

In order to achieve the above object, the present invention acquires data representing an image obtained by capturing an image of at least a part of a structure from an image pickup device, and with respect to each of a plurality of parts constituting the structure, the three-dimensional parts of the structure are obtained. A predetermined relationship is established between the data representing the shape and the positional relationship of the feature point group included in the image from the storage device that stores the data representing the attributes of the relevant part in each of the past multiple periods in association with each other. Acquisition means for acquiring data representing a three-dimensional shape having a group of feature points having a positional relationship to be satisfied and data representing attributes stored in association with the data, and features included in the image with respect to the image. An image of the three-dimensional shape positioned so that the point group and the feature point group of the three-dimensional shape represented by the data acquired by the acquisition means match, and an image of the attribute represented by the data acquired by the acquisition means. As the first aspect, a data processing apparatus including a generation means for generating data representing the superimposed superimposed image and an output means for outputting the data representing the superimposed image to the display device is proposed.

According to the data processing device according to the first aspect, when the administrator images a structure to be managed by an image pickup device and specifies, for example, a part of the structure whose attributes are desired to be known in the captured image, the part thereof is related to the image. The attributes that change over time are superimposed and displayed on the captured image together with the three-dimensional shape that indicates the site. As a result, the manager of the structure can easily know the time-varying attributes of each of the plurality of parts constituting the structure.

Further, in the data processing device according to the first aspect, the acquisition means acquires data designating any part of the plurality of parts, and the generation means stores the data in the storage device. The present invention has a configuration in which data representing a superimposed image in which images of attributes of a plurality of times represented by data corresponding to a part designated by the data acquired by the acquisition means are superimposed among data representing a plurality of attributes is generated. It may be adopted as the second aspect.

According to the data processing apparatus according to the second aspect described above, the manager can easily know the change over time in the attributes of a specific part among the plurality of parts constituting the structure.

Further, in the data processing device according to the first aspect, the acquisition means acquires data representing any of the past plurality of periods, and the generation means stores the data in the storage device. The third aspect of the present invention is to generate data representing a superimposed image in which images of a plurality of attributes represented by data according to the time represented by the data acquired by the acquisition means are superimposed among the data representing a plurality of attributes. It may be adopted as an embodiment.

According to the data processing apparatus according to the third aspect described above, the manager can list the attributes of each of the plurality of parts constituting the structure at a specific time.

Further, in the data processing device according to the second or third aspect, the storage device stores data representing a three-dimensional shape in each of the past plurality of periods with respect to each of the plurality of parts, and the generation means. Is to generate data representing a superimposed image in which an image of a three-dimensional shape represented by data according to the time represented by the data acquired by the acquisition means is superimposed among the data representing a plurality of shapes stored in the storage device. The configuration may be adopted as the fourth aspect of the present invention.

According to the data processing apparatus according to the fourth aspect described above, the manager can easily know the change over time in the shape of the structure.

Further, in the present invention, there is a process of acquiring data representing an image of a structure from an image pickup device on a computer, and data representing a three-dimensional shape of the part with respect to each of a plurality of parts constituting the structure. And from the storage device that stores the data representing the attributes of the relevant part in each of the past multiple periods in association with each other, the positional relationship satisfying a predetermined relationship with the positional relationship of the feature point group included in the image. A process of acquiring data representing a three-dimensional shape having a feature point group and data representing an attribute stored in association with the data, and the feature point group included in the image and the storage for the image. A superimposed image in which the image of the three-dimensional shape positioned so as to match the feature point group of the three-dimensional shape represented by the data acquired from the device and the image of the attribute represented by the data acquired from the storage device are superimposed. As the fifth aspect, we propose a program for executing the process of generating the data to be represented and the process of outputting the data representing the superimposed image to the display device.

According to the program according to the fifth aspect, the computer realizes the data processing device according to the first aspect.

Further, the present invention includes a storage device, an image pickup device, a data processing device, and a display device.
The storage device stores, for each of the plurality of parts constituting the structure, the data representing the three-dimensional shape of the part and the data representing the attributes of the part in each of the past plurality of periods in association with each other. The data processing device acquires data representing an image obtained by capturing at least a part of the structure from the image pickup device, and is predetermined from the storage device with respect to the positional relationship of the feature point group included in the image. An acquisition means for acquiring data representing a three-dimensional shape having a group of feature points having a positional relationship satisfying the relationship of the above, and data representing attributes stored in association with the data, and the image with respect to the image. The image of the three-dimensional shape positioned so that the included feature point group and the feature point group of the three-dimensional shape represented by the data acquired by the acquisition means match, and the attributes represented by the data acquired by the acquisition means. As a sixth aspect, a data processing system including a generation means for generating data representing a superimposed image in which an image is superimposed and an output means for outputting the data representing the superimposed image to the display device is proposed.

According to the data processing system according to the sixth aspect, when the administrator images a structure to be managed by an image pickup device and specifies, for example, a part of the structure whose attributes are desired to be known in the captured image, the part is related to the image. The attributes that change over time are superimposed and displayed on the captured image together with the three-dimensional shape that indicates the site. As a result, the manager of the structure can easily know the time-varying attributes of each of the plurality of parts constituting the structure.

According to the present invention, the manager of the structure can easily know the time-varying attributes of each of the plurality of parts constituting the structure.

The figure which illustrates the situation where the data processing system which concerns on one Embodiment is used. The figure which showed a plurality of parts constituting a structure to be managed. The figure which showed the hardware configuration of the data processing system which concerns on one Embodiment. The figure which showed the structure of the data processing apparatus which concerns on one Embodiment. The figure which illustrated the data structure of the management database which concerns on one Embodiment. The figure which showed the flow of the processing performed by the data processing apparatus which concerns on one Embodiment. The figure which showed the feature point group extracted from the image of a structure by the extraction means which concerns on one Embodiment. The figure which illustrates the screen displayed by the data processing system which concerns on one Embodiment. The figure which illustrates the screen displayed by the data processing system which concerns on one Embodiment. The figure which illustrates the screen displayed by the data processing system which concerns on one Embodiment. The figure which illustrated the structure to be managed with a marker. The figure for demonstrating the role of a marker attached to a structure.

[Embodiment]
The data processing system 1 according to the embodiment of the present invention will be described below. In the data processing system 1, when the manager of the structure takes an image of the structure to be managed and selects a part whose attributes are desired to be known from a plurality of parts constituting the structure, the data processing system 1 responds to the operation. It is a system that displays the attributes of the selected part.

FIG. 1 is a diagram illustrating a situation in which a data processing system 1 is used. The example of FIG. 1 shows a situation in which the data processing system 1 displays the attributes of the structure 9 to be managed. The illustrated structure 9 in FIG. 1 is a part of a pier.

FIG. 2 is a diagram showing a plurality of parts constituting the structure 9. The structure 9 is between the steel pipe piles 901 to 904, the blocks 911 to 914 fixed so as to cover the heads of the steel pipe piles 901 to 904, and the two adjacent blocks 911 to 914. It is composed of the arranged blocks 921 to 924 and the superstructure 931 arranged on the blocks 911 to 914 and the blocks 921 to 924.

In the present embodiment, the hardware used to realize the data processing system 1 is a tablet-type computer including a touch display and a camera arranged on the back surface when the display surface of the touch display is the front surface. .. FIG. 3 is a diagram showing a hardware configuration of the data processing system 1.

The data processing system 1 includes a computer 10, a display 12, a touch panel 13, a camera 14, and an SSD (Solid State Drive) 15. The display 12 is a display device, and the touch panel 13 is an operation device that accepts a user's touch operation. The display 12 and the touch panel 13 are stacked and arranged to form a touch display. The camera 14 is an image pickup device, and the SSD 15 is an example of a storage device. The data processing system 1 may include a storage device of another type such as an HDD (Hard Disk Drive) instead of the SSD 15.

The computer 10 is an input / output interface 103 that inputs / outputs data between a memory 101 that stores a program and various data, a processor 102 that performs various data processing according to the program stored in the memory 101, and an external device. To prepare for. The display 12, the touch panel 13, the camera 14, and the SSD 15 are connected to the input / output interface 103.

The computer 10 performs a process according to the program according to the present embodiment, so that the image of the structure 9 captured by the camera 14 is an image showing the three-dimensional shape of the structure 9 and the attributes of the portion designated by the user. It functions as a data processing device 11 for displaying a superposed image (hereinafter referred to as “superimposed image”) on the display 12. FIG. 4 is a diagram showing the configuration of the data processing device 11. That is, the computer 10 realizes the data processing device 11 having the configuration shown in FIG. 4 by performing the processing according to the program according to the present embodiment.

The data processing device 11 is an acquisition unit 111 that acquires various data from an external device, and a generation unit that generates data representing a superimposed image (hereinafter, referred to as “superimposed image data”) using the data acquired by the acquisition unit 111. The 112 is provided with an output means 113 for outputting the superimposed image data generated by the generation means 112 to the display 12.

The acquisition means 111 acquires data (hereinafter, referred to as “image data”) representing an image captured by the camera 14 from the camera 14. Further, the acquisition means 111 acquires data indicating the management information of the structure 9 (hereinafter, referred to as “management data”) from the SSD 15. The SSD 15 stores a management database, which is a database for managing management data. FIG. 5 is a diagram illustrating the data structure of the management database.

The management database includes tables corresponding to each of the plurality of structures (hereinafter referred to as "management tables"). Data indicating the identification number, name, and position of the corresponding structure is associated with each of the management tables.

In addition, each of the management tables contains data records corresponding to each of the plurality of parts constituting the structure. The data records included in the management table include a data field [number] that stores data representing the part number, a data field [name] that stores data representing the part name, and data indicating multiple past periods (hereinafter,). A data field [time] that stores (referred to as "time data"), and a data field [hereinafter referred to as "shape data") that represents a three-dimensional shape corresponding to each of a plurality of times represented by the time data of a part. Shape], a data field [attribute 1] that stores data (hereinafter referred to as "attribute data") representing attributes 1 to n (where n is an arbitrary natural number) corresponding to each of a plurality of periods represented by the timing data of the part. ] To [attribute n].

The three-dimensional shape represented by the shape data stored in the data field [shape] means a shape including the position and posture of the part. Therefore, the three-dimensional shape represented by the shape data corresponding to each of the plurality of past periods represents not only the deformation of the part but also the change in the position and posture of the part.

The types of attributes represented by the attribute data stored in the data fields [attribute 1] to [attribute n] differ variously depending on the type of the part. The attributes represented by the attribute data include material, size (diameter, length, etc.), strength, inspection results (degree of corrosion, etc.), repair details, captured images, drawings such as blueprints, and replacement of parts. Existence and the like can be mentioned as an example, but the present invention is not limited to these.

The data field [shape] and the data fields [attribute 1] to [attribute n] can store data corresponding to each of the plurality of times represented by the time data, but the available data should be stored. It suffices, and it is not necessary to store data for all times.

With reference to FIG. 4, the description of the configuration of the data processing device 11 will be continued. The acquisition means 111 acquires data indicating a user's operation (hereinafter referred to as “operation data”) from the touch panel 13. The acquisition means 111 interprets the user's operation represented by the operation data acquired from the touch panel 13, and the user interprets the time data representing the time selected by the user from among the plurality of times, and the user from among a plurality of parts constituting the structure. Generates data that specifies the selected part (hereinafter referred to as "part specification data"), data that specifies the attribute specified by the user from multiple attributes of the structure (hereinafter referred to as "attribute specification data"), and the like. ..

The generation means 112 includes an extraction means 1121 that extracts feature points from an image represented by image data, and feature points of an arrangement pattern similar to a plurality of feature points (hereinafter referred to as “feature point cloud”) extracted by the extraction means 1121. The search means 1122 for searching the shape data representing the three-dimensional shape including the group from the management database, the feature point cloud of the three-dimensional shape of the part represented by the shape data searched by the search means 1122, and the feature point of the part represented by the image data. The structure consists of a transforming means 1123 that transforms the three-dimensional shape so that the group matches, an image of the three-dimensional shape deformed by the transforming means 1123, and an image of the attribute represented by the attribute data corresponding to the shape data searched by the search means 1122. A superimposing means 1124 superimposing on an image of an object is provided.

In the present application, the arrangement pattern of the feature point cloud included in the three-dimensional shape is similar to the arrangement pattern of the feature point cloud included in the image of the structure, that is, the position of the feature point cloud included in the three-dimensional shape. It means that the positional relationship between the relationship and the feature point cloud included in the image of the structure satisfies a predetermined relationship. As an example of the predetermined relationship, the positional relationship of the feature point group included in the three-dimensional shape and the positional relationship of the feature point group included in the image of the structure when the three-dimensional shape is viewed from any viewpoint. There is a relationship in which the degree of similarity with is equal to or less than a predetermined threshold, but the relationship is not limited to this. Further, as an example of the similarity, when two feature point groups are arranged on the same plane and feature points that are close to each other from each of the feature point groups are associated with each other to form a pair of feature points. Examples include, but are not limited to, the average value of the distances between the two feature points included in each pair.

FIG. 6 is a diagram showing a flow of processing performed by the data processing device 11. The data processing device 11 reads, for example, the management table from the management database (see FIG. 5) at startup (step S01). That is, the acquisition means 111 acquires the management table from the SSD 15.

After that, when the user performs an operation instructing the data processing system 1 to start imaging, the data processing device 11 acquires image data from the camera 14 (step S02). That is, the image data representing the image captured by the camera 14 is handed over from the camera 14 to the acquisition means 111.

When the data processing device 11 acquires image data from the camera 14, it extracts a group of feature points included in the image represented by the acquired image data (step S03). The extraction of this feature point group is performed by the extraction means 1121. FIG. 7 is a diagram showing a group of feature points extracted by the extraction means 1121 from the image of the structure 9. As shown in FIG. 7, end points, intersections, and the like of lines included in the image of the structure 9 are extracted as feature points.

With reference to FIG. 6, the description of the flow of processing performed by the data processing device 11 will be continued. Subsequently, the data processing device 11 searches for shape data representing a three-dimensional shape including a feature point group having an arrangement pattern similar to the feature point group extracted in step S03 from the management table read out in step S01 (step S04). ). The search for the shape data is performed by the search means 1122. In this case, the search means 1122 includes an arrangement pattern of feature point groups (line end points, intersections, etc.) of the three-dimensional shape of the structure 9 represented by a plurality of shape data included in the management table corresponding to the structure 9, and a step. In S03, it is determined that the arrangement patterns of the feature point groups extracted by the extraction means 1121 are similar, and a plurality of shape data included in the management table corresponding to the structure 9 is obtained as a search result.

Subsequently, the data processing device 11 transforms the three-dimensional shape represented by the shape data searched in step S04 so that the feature point group matches the feature point group extracted from the image represented by the image data (step). S05). This deformation of the three-dimensional shape is performed by the deformation means 1123. The deformation of the three-dimensional shape performed by the transforming means 1123 is a process of changing the inclination of the coordinate axes of the three-dimensional shape represented by the shape data and the scale of the coordinates so that the feature point group of the three-dimensional shape matches the feature point group of the image. Is.

Subsequently, the data processing device 11 selects a part, a time, and an attribute to be displayed (step S06). When the part, time and attribute are not specified by the user as in the case of starting the data processing device 11, for example, the part with the smallest number, the design time, and all the attributes are selected as initial values. The process of selecting the part, time, and attribute to be displayed is performed by the superimposing means 1124.

Subsequently, the data processing device 11 superimposes the three-dimensional image of the portion selected in step S06 and the image of the attribute selected in step S06 on the image represented by the image data at the time selected in step S06 (step). S07). This superposition process is performed by the superimposition means 1124. The superimposing means 1124 generates superimposing image data representing the superimposing image.

Subsequently, the data processing device 11 outputs the superimposed image data generated in step S07 to the display 12 (step S08). The output of the superimposed image data is performed by the output means 113. The above is the processing flow of the data processing device 11.

FIG. 8 is a diagram illustrating a screen displayed on the display 12 (hereinafter, referred to as “management information display screen”) when the user takes an image of the structure 9 with the camera 14 of the data processing system 1. The name of the imaged structure is displayed in the area A01 of the management information display screen. In the area A02 of the management information display screen, the names of the parts constituting the structure are displayed in a list, and the currently selected parts are displayed in a mode different from the others (thick frame in the example of FIG. 8).

In the area A03 of the management information display screen, a plurality of past times are displayed in a list, and the currently selected time is displayed in a mode different from the others (thick frame in the example of FIG. 8). In the area A04 of the management information display screen, a plurality of attributes are displayed in a list, and the currently selected attribute is displayed in a mode different from the others (thick frame in the example of FIG. 8).

In the area A05 of the management information display screen, an image represented by the image data, that is, an image captured by the camera 14 is displayed. The image of the region A05 is superposed with an image G01 having a three-dimensional shape of the currently selected portion and an image G02 of the currently selected attribute of the currently selected time and region.

The user can change the part to display the attribute by touching any of the parts included in the list displayed in the area A02. Further, the user can change the time of the attribute to be displayed by touching any of the times included in the list displayed in the area A03. Further, the user can change the displayed attribute by touching any of the attributes displayed in the list of the area A04.

When the user performs an operation on the list of areas A02 to A04 and any of the display target parts, times, and attributes is changed, the acquisition means 111 is selected by the user based on the operation data acquired from the touch panel 13. When time data representing the time, part specification data that specifies the part selected by the user, or attribute specification data that specifies the attribute selected by the user are generated, and the time, part, and attribute displayed by those generated data are displayed. , Site and attribute (FIG. 6, step S06). After that, the data processing device 11 performs the processing of steps S07 and S08 of FIG. As a result, the content of the management information display screen displayed on the display 12 is updated.

In FIG. 9, the user selects “first steel pipe pile” from the list of area A02, selects “all time” from the list of area A03, and selects “attribute 4 (corrosion degree)” from the list of area A04. It is a figure exemplifying the management information display screen displayed on the display 12 in this case. On the management information display screen illustrated in FIG. 9, the change with time of the degree of corrosion of the first steel pipe pile selected by the user is displayed.

In FIG. 10, the user selects "all parts" from the list of the area A02, selects "at the time of the first inspection (July 15, 2015)" from the list of the area A03, and "at the time of the first inspection (July 15, 2015)", and " It is a figure exemplifying the management information display screen displayed on the display 12 when "attribute 4 (corrosion degree)" is selected. On the management information display screen illustrated in FIG. 10, the degree of corrosion of each of the plurality of parts of the structure 9 at the time of the first inspection selected by the user is displayed in a list. When the user selects any part from the list of the image G02 on this management information display screen, the image displayed in the area A05 is updated with the superimposed image of the three-dimensional shape image of the part selected by the user. Will be done.

In the management information display screen illustrated in FIG. 10, the display mode (for example, color) of the image showing the three-dimensional shape of each part is changed according to the content of the designated attribute (in this case, the degree of corrosion). You may let me. For example, on the management information display screen of FIG. 10, if the three-dimensional shape of each part is displayed in purple, which is closer to blue as the degree of corrosion is lower, and purple, which is closer to red as the degree of corrosion is higher, the user has a degree of corrosion. You can intuitively know the spatial distribution of.

Further, when a specific time (other than "all times") is selected in the list of the area A03 by the user, the area A05 of the management information display screen is a three-dimensional shape represented by the shape data of the time specified by the user. The superimposed image on which the image of is superimposed is displayed. Therefore, by sequentially selecting different times in the list of the area A03, the user can change the shape of the selected part in the list of the area A02 with time (the appearance of the steel pipe pile tilting with the passage of time, and the wave-dissipating block). It is possible to easily know how it moves or disappears with the passage of time.

According to the data processing system 1 described above, the manager of the structure can easily know the attributes that change with time for each of the plurality of parts constituting the structure.

[Modification example]
The above-described embodiment is a specific example of the present invention, and can be variously modified within the scope of the technical idea of the present invention. An example of these variations is shown below.

(1) The storage device (SSD15), the image pickup device (camera 14), the data processing device 11 and the display device (display 12) included in the above-mentioned data processing system 1 are housed in one housing. These devices constituting the data processing system 1 do not need to be housed in one housing, and may be configured as individual devices connected to each other by a communication cable, wireless communication, or the like.

For example, the data processing system 1 may be configured by a tablet computer having a built-in camera and a server device having a communication function. In this case, the tablet computer functions as an image pickup device and a display device, and the server device functions as a storage device and a data processing device. The tablet-type computer transmits image data representing an image of the structure captured in response to the user's operation to the server device. The server device generates superimposed image data representing a superimposed image in which an image of a three-dimensional shape and an attribute is superimposed on an image represented by the image data received from the tablet computer, and transmits the superimposed image data to the tablet computer. The tablet computer displays the superimposed image represented by the superimposed image data received from the server device.

(2) A configuration in which the functions of the data processing device 11 described above are shared by a plurality of devices may be adopted. For example, the data processing system 1 is composed of a tablet computer having a built-in camera having a communication function and a server device, and the tablet computer is a storage device (SSD15) for storing a management database and a search means 1122 in the configuration of the data processing system 1. A storage device (SSD15) for storing the management database and an extraction means 1121 may be provided.

In this case, the tablet computer transmits data representing the feature point group extracted from the image of the structure to the server device. The server device searches the management database for shape data similar to the arrangement pattern of the feature point group represented by the data received from the tablet computer, and transmits the management table including the searched shape data to the tablet computer. The tablet computer uses the management table received from the server device to generate and display the superimposed image.

(3) In order to facilitate the processing of the search means 1122 and the transforming means 1123, a marker, which is an image showing the direction of the coordinate axes of the structure, may be attached in advance to the structure to be managed. FIG. 11 is a diagram illustrating the structure 9 to which the marker M is attached, and FIG. 12 is a diagram for explaining the role of the marker M.

The marker M includes six feature points, that is, feature points P1 to P6. The direction from the feature point P3 to the feature point P4 indicates the X-axis direction of the structure. The direction from the feature point P3 to the feature point P6 indicates the Y-axis direction of the structure. The direction from the feature point P3 to the feature point P2 indicates the Z-axis direction of the structure.

When the marker M is included in the image represented by the image data, the search means 1122 can specify the direction of the coordinate axes of the structure based on the feature point group of the marker M. As a result, the search means 1122 determines the similarity between the arrangement pattern of the feature point group extracted from the image represented by the image data and the arrangement pattern of the feature point group included in the three-dimensional shape represented by the shape data included in the management table. It can be easily identified.

Further, when the image represented by the image data includes the marker M, the transforming means 1123 facilitates the deformation of the three-dimensional shape based on the direction of the coordinate axis indicated by the marker M and, for example, the distance between the feature point P1 and the feature point P2. Can be done.

(4) The program according to the present invention may be provided in a state of being continuously stored in a computer-readable recording medium such as an optical recording medium or a semiconductor memory, or may be provided via a communication network such as the Internet. May be provided. When the program according to the present invention is provided stored in a recording medium, a computer reads and uses the program from the recording medium. Further, when the program according to the present invention is provided via a communication network, the computer receives the program from the device of the distribution source and uses it.

1 ... Data processing system, 9 ... Structure, 10 ... Computer, 11 ... Data processing device, 12 ... Display, 13 ... Touch panel, 14 ... Camera, 15 ... SSD, 101 ... Memory, 102 ... Processor, 103 ... Input / output interface , 111 ... acquisition means, 112 ... generation means, 113 ... output means, 901 to 904 ... steel pipe piles, 911 to 914 ... blocks, 921 to 924 ... blocks, 931 ... superstructure, 1121 ... extraction means, 1122 ... search means, 1123 ... Transforming means, 1124 ... Superimposing means.

Claims (6)

Data representing an image of at least a part of the structure is acquired from the image pickup device, and for each of the plurality of parts constituting the structure, data representing the three-dimensional shape of the part and each of the past multiple periods. A three-dimensional shape having a feature point cloud having a positional relationship that satisfies a predetermined relationship with the positional relationship of the feature point group included in the image from a storage device that stores the data representing the attribute of the relevant portion in association with each other. And the acquisition means for acquiring the data representing the data representing the data and the data representing the attributes stored in association with the data.
An image of the three-dimensional shape positioned so that the feature point cloud included in the image and the feature point cloud of the three-dimensional shape represented by the data acquired by the acquisition means match the image, and the acquisition means. A generation means for generating data representing a superimposed image in which an image of an attribute represented by the acquired data is superimposed.
A data processing device including an output means for outputting data representing the superimposed image to a display device. The acquisition means acquires data that specifies any part of the plurality of parts, and obtains data.
The generation means represents a superimposed image in which images of attributes of a plurality of periods represented by data corresponding to a part designated by the data acquired by the acquisition means among the data representing a plurality of attributes stored in the storage device are superimposed. The data processing apparatus according to claim 1, which generates data. The acquisition means acquires data representing any of the past plurality of periods, and obtains data.
The generation means generates data representing a superimposed image in which images of a plurality of attributes represented by data according to the time represented by the data acquired by the acquisition means among the data representing a plurality of attributes stored in the storage device are superimposed. The data processing apparatus according to claim 1. The storage device stores data representing a three-dimensional shape at each of the past plurality of periods for each of the plurality of parts.
The generation means generates data representing a superimposed image in which an image of a three-dimensional shape represented by data according to the time represented by the data acquired by the acquisition means is superimposed among the data representing a plurality of shapes stored in the storage device. The data processing apparatus according to claim 2 or 3. On the computer
The process of acquiring data representing an image of a structure from an image pickup device, and
From the storage device that stores the data representing the three-dimensional shape of the part and the data representing the attributes of the part in each of the past plurality of periods in association with each other for each of the plurality of parts constituting the structure. Data representing a three-dimensional shape having a positional relationship feature point cloud satisfying a predetermined relationship with the positional relationship of the feature point cloud included in the image, and data representing attributes stored in association with the data. And the process of getting
With respect to the image, the image of the three-dimensional shape positioned so that the feature point group included in the image and the feature point group of the three-dimensional shape represented by the data acquired from the storage device coincide with each other, and the storage device . Processing to generate data representing a superimposed image that is superimposed on the image of the attribute represented by the data obtained from
A program for executing a process of outputting data representing the superimposed image to a display device. A storage device, an image pickup device, a data processing device, and a display device are provided.
The storage device stores, for each of the plurality of parts constituting the structure, the data representing the three-dimensional shape of the part and the data representing the attributes of the part in each of the past plurality of periods in association with each other.
The data processing device is
Data representing an image obtained by capturing at least a part of the structure is acquired from the image pickup device, and a positional relationship satisfying a predetermined relationship with the positional relationship of the feature point cloud included in the image from the storage device. An acquisition means for acquiring data representing a three-dimensional shape having a group of feature points and data representing attributes stored in association with the data.
An image of the three-dimensional shape positioned so that the feature point cloud included in the image and the feature point cloud of the three-dimensional shape represented by the data acquired by the acquisition means match the image, and the acquisition means. A generation means for generating data representing a superimposed image in which an image of an attribute represented by the acquired data is superimposed.
A data processing system including an output means for outputting data representing the superimposed image to the display device.

Images