JP2024039070A - Feature management system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system capable of efficiently managing features.

SOLUTION: A captured image of a feature imaged by a camera 244 of a mobile terminal device 240 is transmitted to a server device 210 by image transmitting means 242. At this time, if the camera 244 has an imaging position addition function, an imaging position (position by absolute coordinates) is also transmitted to the server device 210. Receiving means 216 of the server device 210 receives the captured image and the imaging position. Based on the imaging position, two-dimensional projection image generating means 218 generates two-dimensional projection images for a plurality of angles when viewing surrounding three-dimensional point cloud data from the imaging position. Comparison means 220 selects a two-dimensional projection image that best matches the received captured image from among the plurality of generated two-dimensional projection images. As a result, the captured image of the feature is recorded in association with the feature of the selected two-dimensional projection image.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to a feature management system using point cloud data obtained by measuring features.

BACKGROUND ART Laser surveying devices are mounted on unmanned aerial vehicles (UAVs), automobiles, and the like to measure the shape of the ground surface and terrestrial features. For example, in an MMS (mobile mapping system) using a car, a GPS receiver, a video camera, an IMU, and a laser scanner are mounted on the ceiling of the car, and measurements are taken while the car is driving on a road. The video camera may be either analog or digital.

Through such measurements, it is possible to obtain three-dimensional point cloud data indicating the ground surface and the contours of features. Based on the position obtained by GPS and the scanning direction and distance of the laser scanner, each three-dimensional point group data is assigned a position in absolute coordinates (plane rectangular coordinates, latitude, longitude, altitude, etc.). Further, the ground surface and terrestrial features are imaged by a video camera to generate video data (still image data). This moving image data is recorded in association with a constantly changing imaging position.

Obtaining 3D point cloud data allows us to know the current state of the ground surface and features, which can be useful for maintenance, map creation, etc.

The inventors have proposed that attribute data, including data indicating the area in absolute coordinates, a name, etc., be assigned to features in three-dimensional point cloud data.

However, although the above attribute data has been standardized and data has become more abundant, it is desirable to have a system that can efficiently manage features when using the attribute data for maintenance, etc. It is rare. For example, it is desired to appropriately manage photographs taken of the appearance of each feature.

An object of the present invention is to provide a system that can solve the above-mentioned problems and efficiently manage terrestrial features.

The independently applicable features of the invention are listed below.

(1)(2)(3) A feature management system according to the present invention is a feature management system comprising a server device and a terminal device capable of communicating with the server device, wherein the server device is configured to a recording unit that records three-dimensional point cloud data including defined features; a receiving unit that receives a captured image including the features and an imaging position transmitted from the terminal device; The three-dimensional point cloud data is projected onto a plane from different angles to generate two-dimensional projected images from a plurality of angles, and the captured image including the received feature and each of the generated two-dimensional projected images are generated. The terminal device includes a captured image/point cloud data correspondence means for associating a captured image of the feature with three-dimensional point cloud data of the feature based on a comparison with the feature, the terminal device including the feature imaged by the camera. The present invention is characterized by comprising an image transmitting means for transmitting the captured image together with the captured position to the feature management server device.

Therefore, a photograph of the feature can be recorded in association with the three-dimensional point cloud data of the feature.

(4) In the feature management system according to the present invention, the captured image/point cloud data correspondence means includes a captured image outer shape generating means that indicates the outer shape of the feature in the captured image, and a captured image outer shape generating means that indicates the outer shape of the feature in the captured image; A two-dimensional projection image outer shape generation means that shows the outer shape of the feature in the captured image and the outer shape of the feature in each of the two-dimensional projection images are compared to determine the outer shape. The present invention is characterized by comprising means for selecting a matching two-dimensional projection image and associating the captured image of the feature with the three-dimensional point cloud data of the feature.

Therefore, by using an external shape that clearly changes depending on the angle, matching processing can be easily performed.

(5) In the feature management system according to the present invention, the recording unit of the server device records the outer shape area surrounding the outer shape of the feature and the feature name, and the captured image/point cloud data correspondence means: a feature estimation means for extracting a feature part from the captured image and estimating a feature name; The three-dimensional point cloud data is projected onto a plane from different angles to generate two-dimensional projected images from a plurality of angles, and the received captured image of the feature portion is compared with each of the generated two-dimensional projected images. The feature is that a captured image of a feature is associated with three-dimensional point cloud data of the feature based on the following.

Therefore, matching can be performed only on predetermined terrestrial features, and the accuracy can be improved.

(6) In the feature management system according to the present invention, the recording unit of the server device records the outer shape area surrounding the outer shape of the feature and the feature name, and the image transmitting means of the terminal device In response to the input, the boundary box and feature name of the feature in the captured image are added to the server device, and the captured image/point cloud data correspondence means transmits the data centering on the received imaging position. The three-dimensional point cloud data of a feature having the same feature name as the received feature name is projected onto a plane from different angles to generate two-dimensional projected images from a plurality of angles, and the received feature portion is The method is characterized in that the captured image of the feature is associated with the three-dimensional point cloud data of the feature based on a comparison between the captured image and each of the generated two-dimensional projection images.

Therefore, matching can be performed only on predetermined terrestrial features, and the accuracy can be improved.

(7) In the feature management system according to the present invention, the captured image/point cloud data correspondence means generates a plurality of virtual imaging positions in the vicinity of the imaging position, and uses each virtual imaging position as a reference to generate three-dimensional point cloud data. It is characterized by generating a two-dimensional projection image that is a two-dimensional projection image.

Therefore, the imaging position can be specified more accurately.

(8) The feature management system according to the present invention is characterized in that the photographed image/point cloud data correspondence means also specifies the imaging position and imaging direction with respect to the feature.

Therefore, not only the imaging position but also the imaging direction can be specified and this information can be used.

(9) In the feature management system according to the present invention, an imaging direction is also attached to a photographed image of a feature transmitted from a terminal device, and the photographed image/point cloud data correspondence means The present invention is characterized in that the angle at which the two-dimensional projection image is generated is narrowed down based on the above-described method.

Therefore, matching processing can be speeded up.

(10) The feature management system according to the present invention is characterized in that the three-dimensional point cloud data and the external shape area of the attribute data are indicated in absolute coordinates.

Therefore, attribute data and three-dimensional point group data can be recorded independently, and attribute data generated from three-dimensional point group data can be used for other three-dimensional point group data.

(11) In the feature management system according to the present invention, the feature extraction means extracts external shape area data of the feature or three-dimensional point cloud data of the feature based on the past imaging position and imaging direction of the feature. , generates a past two-dimensional projected image, and transmits it to a second terminal device, and when the second terminal device takes an image with the camera, the second terminal device displays the two-dimensional projected image of the position on the finder image of the camera. It is characterized by displaying images in an overlapping manner.

Therefore, it becomes easy to image the terrestrial feature from the same position and angle as in the past image capturing.

(12) In the feature management system according to the present invention, the server device includes a point cloud data server device that records the three-dimensional point cloud data, and an attribute data server device that records attribute data including the external shape area. It is characterized by

Therefore, it is possible to provide and utilize an attribute data server device that records attribute data independently.

(13) The feature management system according to the present invention projects the three-dimensional point cloud data onto a plane from different angles, centering on the imaging position at which the captured image including the feature is captured, and from a plurality of angles. A two-dimensional projection image of the feature is generated, and the captured image of the feature is associated with three-dimensional point cloud data of the feature based on a comparison between the captured image and each of the generated two-dimensional projection images. There is.

Therefore, a photograph of the feature can be recorded in association with the three-dimensional point cloud data of the feature.

(14)(15)(16) In the feature management system according to the present invention, the server device has a recording unit that records three-dimensional point cloud data in which a first area and a second area in which positions are defined are distinguished. a receiving means for receiving a captured image including a first area and a second area and an image capturing position transmitted from the terminal device; a region estimating means for distinguishing and extracting, and a plane projection of the differentiated three-dimensional point group data of the first region and the second region centering on the received imaging position from different angles, and A two-dimensional projection image is generated, and the captured image is associated with three-dimensional point cloud data based on a comparison between the differentiated captured images of the first region and the second region and each of the generated two-dimensional projection images. The terminal device includes a captured image/point cloud data correspondence means, and the terminal device includes an image transmitting means for transmitting the captured image including the first area and the second area captured by the camera to the feature management server device together with the captured position. It is characterized by the fact that it is equipped with

Therefore, even if only a partial area of a part of the terrestrial feature is imaged instead of the entire terrestrial feature, the imaging position and direction can be specified.

(17)(18)(19) In the feature management system according to the present invention, the server device includes a recording unit that records three-dimensional point cloud data including features, past imaging positions, and imaging directions; a past two-dimensional projected image generating means for generating a past two-dimensional projected image by projecting three-dimensional point cloud data of a feature or its three-dimensional model onto a plane based on an imaging position and an imaging direction, and transmitting the past two-dimensional projected image to a terminal device; characterized in that the terminal device is equipped with display means for displaying a past two-dimensional projected image received from the server device as a transparent image on a display unit superimposed on a finder image of a camera for imaging a terrestrial feature. It is said that

Therefore, by adjusting the imaging position while referring to past images, it is possible to perform imaging at the same position and direction as in the past imaging.

(20) The feature management method according to the present invention records three-dimensional point cloud data including features, past imaging positions, and imaging directions, and based on the past imaging positions and imaging directions, A past two-dimensional projected image is generated by projecting three-dimensional point cloud data or a three-dimensional model thereof onto a plane, and the past two-dimensional projected image is displayed superimposed on the camera's finder image.

Therefore, by adjusting the imaging position while referring to past images, it is possible to perform imaging at the same position and direction as in the past imaging.

(21)(22) The feature management system according to the present invention is a feature management system comprising a server device and a mobile terminal device capable of communicating with the server device, wherein the server device includes a feature including a feature. The terminal device includes a recording unit that records three-dimensional point cloud data, past imaging positions, and imaging directions, and transmitting means that transmits the imaging positions and imaging directions to a terminal device, and the terminal device transmits the received imaging positions and imaging directions. The present invention is characterized in that the display unit includes display means for displaying on a map.

Therefore, past imaging positions and imaging directions can be easily known.

(23)(24)(25) A feature management system according to the present invention is a feature management system comprising a server device and a terminal device capable of communicating with the server device, wherein the server device includes a road display Three-dimensional point cloud data with reflection intensity including features and external shape area data surrounding the external shape of the road display feature are recorded, and the terminal device receives the external shape area of the road display feature from the server device. a contour area data acquisition means for acquiring data; a three-dimensional point cloud data acquisition means for acquiring three-dimensional point cloud data corresponding to the contour area data of the road display feature from the server device; The present invention is characterized by comprising display means for displaying an object's external shape area and three-dimensional point cloud data in a superimposed manner on a display section.

Therefore, the degree of deterioration of the road display feature can be easily determined.

(26) The feature management method according to the present invention records three-dimensional point cloud data with reflection intensity including road display features and external shape area data surrounding the outer shape of the road display features, and Obtain the outer shape area data of the displayed feature, obtain 3D point cloud data corresponding to the outer shape region data of the road display feature, and overlap the obtained outer shape region of the road display feature with the 3D point cloud data. The feature is that the information is displayed on the display section.

Therefore, the degree of deterioration of the road display feature can be easily determined.

In the embodiment, the "receiving means" corresponds to the process performed by the attribute data server device 60 in response to step S602.

In the embodiment, the "captured image/point cloud data correspondence means" corresponds to steps S701 to S707.

In the embodiment, the "image transmitting means" corresponds to step S602.

In the embodiment, the "outline area data acquisition means" corresponds to step S651.

In the embodiment, the "three-dimensional point group data acquisition means" corresponds to step S652.

In the embodiment, the "display means" corresponds to step S653.

The term "program" is a concept that includes not only programs that can be directly executed by a CPU, but also programs in source format, compressed programs, encrypted programs, and the like.

FIG. 1 is a functional block diagram of a feature management system according to an embodiment of the present invention. This is the system configuration of the feature management system. This is the hardware configuration of the mobile terminal device 21. This is the hardware configuration of the attribute server device 60. 3 is a flowchart of a terminal program. This is an example of three-dimensional point cloud data. It is a figure which shows the process for detecting a ground point. This is an example of design data for features. It is a diagram showing a state in which three-dimensional point cloud data is superimposed on a plan view of a feature. This is an example of three-dimensional point cloud data of a feature. This is an example of an item of attribute data. FIG. 3 is a diagram for explaining a process of estimating a feature name based on three-dimensional point cloud data of the feature. It is a flowchart of feature management processing. It is a flowchart of feature management processing. It is a flowchart of feature management processing. This is a captured image of a feature taken by a user. FIG. 3 is a diagram showing a three-dimensional model of a feature generated based on three-dimensional point cloud data. This is an example of a two-dimensional projected image generated by projecting at different angles. This is a contour image obtained by extracting the contour of a feature from a captured image. This is a captured image of a feature cut out based on the contour of the feature in a two-dimensional projection image. It is a diagram showing an imaging position P and virtual positions P1, P2...P23, P24. FIG. 3 is a diagram showing a captured image in which features are marked by a user. This is an example of displaying the previous image superimposed on the camera's finder screen. FIG. 2 is a functional block diagram of a feature management system according to a second embodiment. It is a flowchart of feature management processing. FIG. 7 is a diagram showing a state in which three-dimensional point cloud data with brightness of a center line and an outline area are displayed in an overlapping manner. FIG. 3 is a functional block diagram of a point cloud data management system according to a third embodiment of the present invention. This is the system configuration of a point cloud data management system. This is the hardware configuration of the terminal device. It is a flowchart of acquiring three-dimensional point cloud data of a feature and generating a three-dimensional model. It is a flowchart of acquiring three-dimensional point cloud data of a feature and generating a three-dimensional model. This is an example of displaying the external shape of a feature. FIG. 33A is first three-dimensional point group data of a feature viewed from a first direction, and FIG. 33B is second three-dimensional point group data of a feature viewed from a second direction. This is synthesized three-dimensional point cloud data. FIG. 35A shows the generated three-dimensional model. FIG. 35B is a display example of the measurement direction. 36A shows the first three-dimensional point group data, FIG. 36B shows the second three-dimensional point group data, and FIG. 36C shows the composite three-dimensional point group data. FIG. 3 is a functional block diagram of a point cloud data management system according to a fourth embodiment. It is a flowchart for acquiring a change over time. It is a flowchart for acquiring a change over time. This is an example of displaying changes in features over time. This is an example of a screen that requests selection of three-dimensional point cloud data with different dates and times. It is a functional block diagram of a feature search system according to a fifth embodiment. This is the system configuration of the feature search system. It is a flowchart of feature search. FIG. 3 is a diagram showing the structure of video data. FIG. 3 is a diagram showing an imaging area. It is a flowchart of video extraction. 3 is a diagram showing the relationship between an imaging trajectory 200 and a terrestrial feature 106 at each time. FIG. FIG. 13 is a diagram showing an image in which search target features are marked.

1. First embodiment
1.1 Overall Configuration FIG. 1 shows the overall configuration of a feature management system according to an embodiment of the present invention. The recording unit 214 of the server device 210 records three-dimensional point group data 222 in absolute coordinates including terrestrial objects.

The mobile terminal device 240 includes a camera 244. The captured image of the feature captured by the camera 244 is transmitted to the server device 210 by the image transmitting means 242. At this time, if the camera 244 has an imaging position addition function, the imaging position (position based on absolute coordinates) is also transmitted to the server device 210. If the camera 244 does not have such a function, it is sent to the server device 210 with an image capturing position by GPS or the like attached thereto.

The receiving means 216 of the server device 210 receives the captured image and the captured position. Based on the imaging position, the two-dimensional projection image generation means 218 generates two-dimensional projection images at a plurality of angles when viewing the surrounding three-dimensional point group data from the imaging position. The comparison unit 220 selects a two-dimensional projection image that best matches the received captured image from among the plurality of generated two-dimensional projection images.

As a result, the captured image of the feature is recorded in association with the feature of the selected two-dimensional projection image. In this way, the three-dimensional point cloud data can be recorded in association with a captured image showing the external appearance and the like.

1.2 System Configuration FIG. 2 shows the system configuration of a point cloud data management system according to one embodiment. In this embodiment, the point cloud server devices 40a, 40b, . . . , 40n and the attribute data server device 60 constitute a server device.

Point cloud server devices 40a, 40b, . . . 40n are provided on the Internet. An attribute data server device 60 is also provided. These server devices 40a, 40b...40n, 60 are provided with terminal devices 20a...20n and mobile terminal devices 20a...20n so as to be able to communicate via the Internet.

The hardware configuration of the mobile terminal device 21 is shown in FIG. A memory 82 , a touch display 84 , a nonvolatile memory 86 , a communication circuit 92 , a camera 88 , and a GPS receiver 98 are connected to the CPU (or a GPU or the like; the same applies hereinafter) 80 . The communication circuit 92 is for connecting to the Internet. As the mobile terminal device 21, a smartphone, a tablet, a wearable computer, etc. can be used.

An operating system 94 and a mobile terminal program 96 are recorded in the nonvolatile memory 86. The mobile terminal program 96 cooperates with the operating system 94 to perform its functions.

The hardware configuration of the attribute data server device 60 is shown in FIG. A memory 83, a display 85, a hard disk 87, a DVD-ROM drive 89, a keyboard/mouse 91, and a communication circuit 93 are connected to the CPU 81. The communication circuit 93 is for connecting to the Internet.

An operating system 95 and an attribute server program 97 are recorded on the hard disk 87. The attribute server program 97 cooperates with the operating system 95 to perform its functions. These programs were recorded on the DVD-ROM 99 and installed on the hard disk 87 via the DVD-ROM drive 89.

The hardware configurations of the terminal devices 20a...20n and the point cloud server devices 40a, 40b...40n are also similar. However, in the terminal device 20a, a terminal program is recorded instead of the attribute server program 97. Furthermore, in the point cloud server devices 40a, 40b, . . . , 40n, a point cloud server program is recorded instead of the attribute server program 97.

1.3 Public processing of point cloud data Surveying companies generate 3D point cloud data and videos of roads and features using MMS installed in cars and the like. The generated three-dimensional point group data is recorded, for example, on a recording medium such as a DVD-ROM, and is imported into the hard disk 87 of the terminal device 20.

Here, features include, for example, road design standard information (distance markers, road center lines, measurement points, etc.), road surface features (roadway sections, sidewalk sections, trackbeds, etc.), and those that share an area with the road surface. Features (stop lines, lane markings, crosswalks, road markings, etc.), features other than the road surface (signposts, lighting poles, traffic lights, pedestrian bridges, etc.), features that indicate the road surface (retaining walls, bridges, road markings, etc.) surface, tunnel, shed, etc.). Note that this is not limited to road features.

The surveying company starts the terminal program and generates attribute data based on the recorded three-dimensional point cloud data. FIG. 5 shows a flowchart of attribute data generation processing in the terminal program 96.

The CPU 81 of the terminal device 20a first reads out the recorded three-dimensional point group data (step S1). An example of three-dimensional point cloud data is shown in FIG. In this embodiment, the three-dimensional point group data is measured including the color of the object to be measured and the reflection intensity of the laser. Additionally, information on the measurement method and measuring equipment is recorded in the three-dimensional point cloud data.

The CPU 80 groups points within a predetermined distance from each other as a cluster in this three-dimensional point group data. This results in the formation of clusters of various sizes. Then, the CPU 81 removes clusters having a predetermined volume, a predetermined number of points or less, a predetermined point group density or less, as noise (step S2).

Subsequently, the CPU 81 extracts a ground point (ground surface) using a cloth simulation method or the like (step S2). In ground point extraction, the CPU 80 first inverts the elevation value of the three-dimensional point group data. For example, if there is three-dimensional point group data of a cross section as shown in FIG. 7A (the point group is represented as a line in the figure), inverted three-dimensional point group data as shown in FIG. 7B can be obtained.

Next, the CPU 81 performs a simulation on the inverted three-dimensional point group data as if a cloth was applied from above. In FIG. 7C, the simulated cloth is shown in dashed lines. Subsequently, the CPU 81 extracts three-dimensional point group data in contact with the simulated cloth as ground points. Next, the CPU 81 inverts the elevation value again to obtain a ground point as shown in FIG. 7D.

The ground points extracted in this way are generally accurate, but as shown in FIG. 7D, they may include some features in the vicinity 100 where the features exist. Therefore, the normal direction of the line formed by each extracted ground point is calculated, and portions where the normal line is at a predetermined angle or more (for example, 30 degrees or more) with respect to the vertical direction are excluded from the ground points. Thereby, a ground point as shown in FIG. 7E can be obtained.

Note that in this embodiment, Cloth Simulation is used to extract the ground points, but the ground points may be extracted by other methods such as the lowest point extraction method.

After extracting the ground points as described above, the CPU 81 removes the ground points from the three-dimensional point group data. As a result, three-dimensional point group data of only the features existing on the ground is obtained (step S4).

Next, the CPU 81 reads out the design data recorded on the hard disk 87 indicating the planar arrangement of the features. This design data is a drawing at the time of designing the feature obtained by measuring the three-dimensional point group data. FIG. 8 shows an example of a design drawing. The CPU 80 extracts the planar shape of each feature based on this design data (step S5). In this embodiment, the planar shape is extracted as a rectangle that includes the planar outer shape of the feature. Note that the planar external shape of each feature may be extracted as is as a planar shape.

Subsequently, the CPU 81 superimposes three-dimensional point cloud data of only the features on this planar shape. Overlay is performed by matching the absolute coordinates of the design data that is the basis of the planar shape with the absolute coordinates of the three-dimensional point group data. At this time, the position of the planar shape in the height direction is made to match the height (elevation) of the ground point (ground surface) of the three-dimensional point cloud data. FIG. 9 shows the matched state. In FIG. 9, only the pedestrian bridge portion of the features shown in FIGS. 7 and 8 is shown. The frame line 300 indicates a rectangular planar shape that surrounds the planar outline of the pedestrian bridge (feature), and it is confirmed that this planar shape 300 matches the position of the three-dimensional point cloud data of the pedestrian bridge (feature). Recognize. Note that although the planar external shape is originally expressed as a plane, in FIG. 9 it is shown with a height for ease of understanding.

For each feature, the CPU 81 selects the one with the highest elevation among the three-dimensional point cloud data corresponding to the planar shape of the feature. The area of the feature is determined by giving the height to the planar shape. Thereby, as shown in FIG. 10, a three-dimensional shape (cuboid) surrounding the feature can be specified. The CPU 81 records the three-dimensional shape thus generated as external shape area data. In this embodiment, external area data is recorded based on the planar shape and the position and height based on absolute coordinates of horizontal azimuth height.

Next, the CPU 81 superimposes each feature area on the three-dimensional point cloud data and displays it on the display 84. The operator looks at this screen and selects the region of each feature using the mouse 91. As a result, an input screen for attribute data such as feature names is displayed, and the operator performs this input. Further, based on the coordinates of the vertices of the three-dimensional shape of the external area data, the coordinates of the center position (other positions representative of the feature may be used) are calculated and used as the coordinate position of the feature. In this way, attribute data can be generated and recorded. FIG. 11 shows items of attribute data in this embodiment. The creation date and time is the creation date and time (measurement date and time) of the three-dimensional point cloud data. The feature name is the name of the feature. The feature area is external shape data indicating the external shape of the feature. The feature position is the center position calculated above. In this embodiment, three-dimensional point group data is recorded in XML format.

Note that, in the above description, the operator inputs the name of the feature, but the name of the feature may be estimated based on three-dimensional point cloud data. For example, as shown in FIG. 12, three-dimensional point cloud data 6 of a feature is projected onto a plane at different angles, and the projected image and feature name are used as learning data to run a deep learning program (or other machine learning program). may also be used). Learn about various features and obtain a trained feature estimation program. This feature estimation program may be provided with three-dimensional point cloud data of a target for feature estimation, and the feature name may be automatically estimated and assigned. Alternatively, the estimated feature name may be displayed and the operator may be asked to confirm it.

Furthermore, since there is a possibility that the three-dimensional point cloud data 6 and the external area are misaligned, it may be displayed on the display 85 and the operator may use a mouse or the like to align the two so that their positions match. .

Attribute data can be obtained as described above.

Note that in the above, the external shape area of the feature is determined by giving the height to the planar data. However, if there is no plane data such as design data, the outer shape area may be determined from the three-dimensional point group data itself.

For example, the outline area is determined as follows. The three-dimensional space is divided into small cubic grids, and if there are points in adjacent grids vertically, horizontally, horizontally, and diagonally, they are combined into one. For example, spatial labeling techniques using connected components can be used.

For each region organized as a grid where points exist, regions having a predetermined volume, a predetermined number of points or less, a predetermined point group density or less are removed as noise. Each remaining area becomes an area corresponding to a feature. For each region, set a rectangular parallelepiped that encloses its outer shape. This rectangular parallelepiped is the outer shape area of the feature.

As described above, attribute data corresponding to three-dimensional point cloud data can be generated in the terminal device 20. The surveying company delivers this three-dimensional point cloud data and attribute data to the delivery destination.

Furthermore, only attribute data can be uploaded to the attribute data server device 60 at the discretion of the surveying company. This allows companies that require three-dimensional point cloud data to access the attribute data server device 60 and refer to the attribute data. If the information of the company providing the corresponding 3D point cloud data is recorded in the attribute data, the company that requires the 3D point cloud data can request the surveying company to obtain the 3D point cloud data based on this information. You can make requests.

Further, the three-dimensional point cloud data may be uploaded to the point cloud server device 40. If the requester for the three-dimensional point cloud data is a local government or the like, it may be requested to upload it to a predetermined point cloud server device 40. As a result, the three-dimensional point cloud data and attribute data will be made public on the Internet.

Furthermore, the three-dimensional point group data and the external area data of the attribute data are indicated in absolute coordinates. Therefore, if there are multiple pieces of 3D point cloud data about the same feature uploaded by different businesses to different point cloud server devices, if at least one attribute data has been uploaded, these multiple 3D point cloud data Group data can be obtained. For example, if three-dimensional point cloud data regarding the same feature is available from different directions depending on the operator, more complete three-dimensional point cloud data can be obtained by acquiring both.

1.4 Feature management processing Figure 13 shows a flowchart of feature management processing. For example, a user (such as a person in charge of managing the feature) goes near a damaged feature and captures an image of the feature using the camera 88 of the mobile terminal device 21 (step S601). The CPU 80 of the mobile terminal device 21 (hereinafter sometimes abbreviated as the mobile terminal device 21) transmits the captured image thus generated to the attribute data server device 60 in response to the user's operation (step S602). At this time, the mobile terminal device 21 also transmits the position (imaging position) acquired by the GPS receiver 98 at the time of imaging. FIG. 16 shows an example of a captured image.

Having received this, the CPU 81 of the attribute data server device 60 (hereinafter sometimes abbreviated as the attribute data server device 60) acquires attribute data of a feature near the imaging position (step S701). Since the attribute data records the type of feature (feature name) and the location of the feature, features within a predetermined distance from the imaging position are extracted.

Next, based on the extracted external area of the feature, the attribute data server device 60 sends three-dimensional point cloud data included in the external area (i.e. If the user has recorded three-dimensional point cloud data of the feature, a request is made to send it (step S702).

Among the point cloud server devices 40a, 40b...40n, the point cloud server device 40 that records and holds the corresponding three-dimensional point cloud data converts the three-dimensional point cloud data of the feature into attribute data. A reply is sent to the server device 60 (step S801).

The attribute data server device 60 generates a three-dimensional model (three-dimensional model specified by absolute coordinates) representing the external shape based on the received three-dimensional point group data of the feature (step S703). Note that this three-dimensional model may be generated and recorded in advance. An example of a three-dimensional model generated for each feature is shown in FIG. 17.

Next, for the three-dimensional model generated for each feature, a two-dimensional projection image is generated while changing the angle around the imaging position PH shown in FIG. 17 (step S704). Examples of generated two-dimensional projection images are shown in FIGS. 18A, B, and C. Although FIG. 18 shows only two-dimensional projection images from three angles, two-dimensional projection images from other angles are similarly generated.

Subsequently, the attribute data server device 60 generates a contour image of the feature depicted in the captured image. The process of extracting the outline of a predetermined object (here, a terrestrial feature) depicted in a captured image can be performed using, for example, Mask R-CNN (https://github.com/matterport/Mask_RCNN). FIG. 19 shows a contour image.

The attribute data server device 60 compares the contoured captured image in FIG. 19 with each two-dimensional projection image shown in FIG. 18, and selects the one that best matches the image (step S705). For example, SIFT, SURF, KAZE, and A-KAZE can be used for this matching. For example, assume that the two-dimensional projection image in FIG. 18B is selected. Through this matching, the imaging direction can also be obtained based on the angle at which FIG. 18B was generated.

Next, the attribute data server device 60 extracts the outline of the feature in the selected two-dimensional projection image. Furthermore, the captured image is cut out based on this external shape to obtain a captured image of only the feature (step S706). FIG. 20 shows the feature image obtained by cutting.

The attribute data server device 60 records the feature image, the imaging position, and the imaging direction in association with the attribute data of the feature (step S707).

Therefore, each of the terminal devices 20a...20n and each of the mobile terminal devices 21a...21n can access the attribute data server device 60 to view and obtain captured images of each feature. This allows each feature to be easily managed. Furthermore, the person who takes the image at the site can automatically make the association without having to make the association with the three-dimensional point cloud data of the feature himself.

1.5Other
(1) In the above embodiment, when generating a two-dimensional projection image, the imaging position received from the mobile terminal device 21 is used as is. However, if there is a problem with the positional accuracy, as shown in FIG. An image may also be generated.

The two-dimensional projection images generated by these are matched with the captured image, and the best matching two-dimensional projection image is selected. A virtual position corresponding to the selected two-dimensional projection image is recorded as a corrected imaging position.

Furthermore, the virtual position may be set by changing not only the planar position (latitude and longitude) but also the height (elevation) upward or downward relative to the received height.

(2) In the above embodiment, matching is performed by comparing the contour of the feature in the captured image and the contour of the feature (three-dimensional model) in the two-dimensional projection image. However, the three-dimensional point group data may be two-dimensionally projected and the point group data and the captured image may be compared. In this case, if the captured image is in color, the two-dimensionally projected point cloud data is also colored based on the color data at the time of measurement, and matching including color is performed to improve the accuracy of determining the degree of matching.

(3) In the above embodiment, two-dimensional projection images are generated for all of the terrestrial objects within a predetermined range from the imaging position, and matching is performed with the entire captured image. However, if only one feature is captured in the captured image, the name of the feature in the captured image is specified, and only the 3D point cloud data of the identified feature name is extracted. A two-dimensional projection image may be generated as a target, and matching with a captured image of a feature may be performed.

In this case, for example, as shown in FIG. 22, the user operates the mobile terminal device 21 to mark the part of the feature of interest in the captured image and sends it to the attribute data server device 60. You can. At this time, the user operates the mobile terminal device 21, inputs a feature name (preferably selected from a pull-down menu of predetermined feature names), and also sends the name.

The attribute server device 60 selects the sent feature name from among the features within a predetermined range from the imaging position, and generates a two-dimensional projection image. At this time, two-dimensional projected images from different angles are generated for each feature.

In matching, a captured image of a feature selected by the user is compared with a two-dimensional projected image of each feature. The captured image of the feature is recorded in association with the two-dimensional projected image of the feature that best matches.

Note that matching may be performed using a three-dimensional model or using point cloud data.

Furthermore, in the above example, the user inputs the name of the feature. However, the portable terminal device 21 or the attribute data server device 60 may identify the object in the captured image and determine the name of the feature. Using a learning model such as deep learning that has learned the image and name of each feature as learning data, it is possible to extract the feature in the image and obtain the feature name (for example, YOLO, Mask R- Programs such as CNN can be used). In this case, it is preferable that the user selects a feature from the captured image, and that the mobile terminal device 21 or the attribute data server device 60 automatically determines the feature name of the selected feature.

(4) In the above embodiment, a two-dimensional projection image is generated for a feature within a predetermined range from the imaging position, and matching is performed with the captured image. In the matching, only image information is used.

However, if the user inputs the feature name of each feature in the captured image, or the mobile terminal device 21 or the attribute data server device 60 automatically determines the feature name (using the above-mentioned YOLO, etc.), Matching may also be performed.

In addition, when multiple features are present in a captured image, the names of each feature are identified from the image, while
The names of each feature may be obtained from the attribute data, and matching may be performed including the positional relationships (order on the image) of these feature names.

(5) In the above embodiment, three-dimensional point group data and attribute data are expressed using absolute coordinates. However, as long as the three-dimensional point cloud data and attribute data are associated with each other, the coordinates do not necessarily have to be absolute coordinates.

(6) In the above embodiment, the accuracy of matching improves if there are many terrestrial objects around the imaging position. However, even in a case where only a road and a slope appear in the captured image, the imaging position can be specified.

For example, it can be used when an accident occurs on a mountain road without features such as signs, and the accident site is identified based on a photograph taken at the scene. In this case, the captured image is separated into road surfaces and slope surfaces using a model learned through deep learning such as SegNet, which is one of the image segmentation methods. On the other hand, a two-dimensional plane image is generated by separating the road surface and slope from each position using three-dimensional point cloud data. It can be determined that the point where the two best match is the imaging position (accident site).

In addition, when a single feature has multiple similar structures (for example, a bridge girder), matching can be performed by setting the above-mentioned virtual imaging position and imaging direction for each location and generating a two-dimensional projection image. By doing so, the location where the image was taken can be specified. Thereby, it is possible to specify which location (which bridge girder among a plurality of bridge girders) has been imaged.

(7) In the above embodiment, the captured image and the captured position are transmitted from the mobile terminal device 21 to the attribute data server device 60. However, the mobile terminal device 21 may be provided with a geomagnetic sensor (to obtain orientation), a gyro sensor (to detect inclinations such as pitch and roll), and the imaging direction may also be transmitted. Based on the received imaging direction, the attribute data server device 60 generates two-dimensional plane images only for the front and rear angles. Thereby, matching accuracy can be improved.

(8) In the above embodiment, past history is not used when capturing an image of a feature with a camera. However, when imaging a feature with a camera, the imaging position and imaging direction are sent to the attribute data server device 60, and if the results of previous imaging from a similar direction in the vicinity are stored, this can be transmitted. You may also obtain it. The attribute data server device 60 generates a two-dimensional projection image based on the past imaging position and imaging direction, and transmits this to the mobile terminal device 21 . The mobile terminal device 21 that has received the past two-dimensional projected image displays the past image IM on the finder screen of the camera in an overlapping manner as a transparent image, as shown in FIG. In FIG. 23, only past images IM of some features are displayed, but the entire past image may be displayed.

This makes it easy to match the positions and directions when it is necessary to take an image at the same position as a position where an image was taken in the past. For example, it is effective when observing changes over time of specific features.

In addition, instead of a past two-dimensional projected image, a past captured image may be displayed superimposed as a transparent image.

Furthermore, instead of or in addition to displaying past images in an overlapping manner, the imaging position and imaging direction at the time of past imaging may be shown on the map. At this time, by also indicating the current imaging position and imaging direction, it becomes easy to perform imaging matching the past imaging position and imaging direction.

(9) In the above embodiment, a feature in the captured image is identified and matching is performed based on its outline and the outline of the three-dimensional point cloud data. However, all contours (edges) of terrestrial objects and other than the terrestrial objects may be extracted from the captured image and matched with the contours of the three-dimensional point group data. For edge extraction from a captured image, for example, a Prewitt filter, a Sobel filter, a Laplacian filter, a LoG filter, a DoG filter, a Canny method, a Hough transform, etc. can be used.

(10) In the above embodiment, the first point cloud server device 40a and the second point cloud server device 40b are constructed as separate server devices, but they may be constructed as a single server device.

(11) In the above embodiment, the first point cloud server device 40a, the second point cloud server device 40b, and the attribute data server device 60 are constructed as separate server devices, but they are constructed as a single server device. You can leave it there.

(12) In the above embodiment, steps S702, S703, S704, S705, and S706 are performed by the attribute data server device. However, this may be executed on a mobile terminal device or a terminal device, and the image of the extracted feature may be transmitted to the attribute data server device and recorded.

(13) In the above embodiment, the mobile terminal device captures an image of a feature and transmits the captured image. However, an image captured by a camera or the like may be imported into a terminal device such as a PC and transmitted.

(14) The above embodiments and modified examples can be implemented in combination with other embodiments unless it contradicts the essence thereof.

2. Second embodiment
2.1 Overall Configuration FIG. 24 shows a functional block diagram of the feature management system according to the second embodiment. In this embodiment, an attribute server device 211 and a three-dimensional point cloud server device 213 constitute a server device.

The outer shape area data acquisition means 262 of the terminal device 260 acquires the outer shape area data 224 of the road display object (terrestrial feature) from the attribute server device 211. It is preferable to use the external shape area data at the time of installation of a road marking such as a center line.

The three-dimensional point cloud data acquisition means 264 acquires three-dimensional point cloud data 222 of the road display object included in the contour area based on the contour area data 224. This three-dimensional point group data 222 also includes the brightness of each point.

The display means 266 displays each point of the three-dimensional point group data and the outer shape region based on the acquired outer shape region data with brightness (high brightness means bright, low brightness dark). As a result, the situation at the time the road display object was installed is shown as the outer shape area, and the current situation is shown as the brightness of the three-dimensional point group data. Therefore, it is possible to know the necessity of repairing the road display object, the repair location, etc.

2.2 System Configuration The system configuration is the same as that shown in FIG. 2 in the first embodiment. Furthermore, the hardware configurations of the point cloud server device 40, attribute data server device 60, terminal device 20, and mobile terminal device 21 are also the same as in the first embodiment.

2.3 Feature management processing Figure 25 shows a flowchart of feature management processing. In the figure, the terminal device is shown as a flowchart of the terminal program, the attribute data server device is shown as the flowchart of the attribute data server program, and the point cloud server device is shown as the flowchart of the point cloud server program. be.

First, the CPU 81 of the terminal device 20b (hereinafter sometimes abbreviated as the terminal device 20b) accesses the attribute data server device 60 and specifies the location where a desired road display object exists (step S651). The location is specified by specifying a range using latitude and longitude, for example. You may enter an address and convert it to a latitude and longitude range.

In response to this, the CPU of the attribute server device 60 (hereinafter sometimes abbreviated as the attribute server device) searches the attribute data and extracts the external shape area data of the road display object within the specified location (step S751). Here, the road display object is one of the features, and is an identification mark displayed on the road surface. For example, center lines, crosswalks, etc.

The attribute server device 60 transmits the extracted external shape area data (planar shape data and its position and height in absolute coordinates), feature names (center line, crosswalk, etc.), etc. to the terminal device 20b. If a plurality of pieces of outline area data are extracted, these pieces of outline area data are transmitted.

The terminal device 20b accesses each of the point cloud server devices 40a...40n and requests them to return the three-dimensional point cloud data of the received external shape area data (step S652). In addition, when a plurality of external area data exist for the same road display object, the measurement date and time of the external area data may be displayed on the terminal device 20b, and the user may be allowed to select one. The user selects the oldest or outer shape area data at the time of installation of the road display object. The terminal device 20b requests three-dimensional point group data based on the selected external area data.

In response, among the point cloud server devices 40a...40n, the point cloud server device that records the three-dimensional point cloud data included in the external shape area data records the three-dimensional point cloud data for each external shape. It is associated with the area data and transmitted to the terminal device 20b (step S851).

The terminal device 20b displays the brightness of the three-dimensional point group data superimposed on the received external shape area data (step S653). An example of the display is shown in FIG. Here, a center line road display is shown as an example. The outline area 600 is displayed based on the outline area data. The high brightness area 602 is an area displayed using three-dimensional point cloud data with high brightness. Although the three-dimensional point group data consists of points, in FIG. 26 they are shown as areas because they are gathered.

If the entire outer area 600 is filled with the high brightness area 602, it can be determined that the center line is healthy with no peeling or the like. If there is peeling or scratches on the center line, the laser reflection intensity at that part will be low. Therefore, as shown in FIG. 26, a low brightness area 604 is displayed within the external area 600. This allows you to know the center line and repair areas that need repair.

2.4 Others
(1) In the above embodiment, a rectangular feature (road display object) having a planar area was described. However, similar processing can be performed on a rectangular parallelepiped feature with a three-dimensional area and whose surface has a higher reflection intensity than other parts.

(2) In the above embodiment, a rectangular road display object is targeted. However, if the shape along the road display object (for example, the shape of a character displayed on the road) is recorded as the outer shape data, it can be applied to road display objects with such a shape. can.

(3) In the above embodiment, the brightness of the three-dimensional point group data is compared with the external shape. However, the three-dimensional point group data itself (without brightness values) may be displayed and compared with the external shape.

(4) In the above embodiment, the first point cloud server device 40a and the second point cloud server device 40b are constructed as separate server devices, but they may be constructed as a single server device.

(5) In the above embodiment, the first point cloud server device 40a, the second point cloud server device 40b, and the attribute data server device 60 are constructed as separate server devices, but they are constructed as a single server device. You can leave it there.

(6) The above embodiments and modified examples can be implemented in combination with other embodiments unless it contradicts the essence thereof.

3. Third embodiment
3.1 Overall Configuration FIG. 27 shows the overall configuration of a point cloud data management system according to the third embodiment of the present invention. The first point cloud server device 40a records a first three-dimensional point cloud data file from a first direction using absolute coordinates including terrestrial objects. The second point cloud server device 40b records a second three-dimensional point cloud data file from a second direction using absolute coordinates including terrestrial objects. The attribute data server device 60 records external area data based on absolute coordinates surrounding the external shape of a feature.

The terminal device 20 can communicate with these server devices. The three-dimensional point cloud data acquisition means 22 of the terminal device 20 acquires external area data of a desired feature from the attribute data server device 60. The three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40a based on the acquired external shape area data of the feature, and converts the three-dimensional point cloud data included in the area into a first three-dimensional point cloud. Obtain as data. Further, the three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40b based on the acquired external shape area data of the feature, and transfers the three-dimensional point cloud data included in the area to the second three-dimensional point cloud data. Acquire as point cloud data.

The three-dimensional model construction means 24 constructs a three-dimensional shape of the feature based on the first three-dimensional point group data and second three-dimensional point group data acquired above. In this way, by taking advantage of the fact that the external area data has absolute coordinates, it is possible to collect three-dimensional point cloud data from different directions of the target feature, which is recorded dispersedly on the Internet. be able to. Therefore, a more complete three-dimensional model of the feature can be formed.

3.2 System Configuration FIG. 28 shows the system configuration of the point cloud data management system according to the third embodiment. Point cloud server devices 40a, 40b, . . . 40n are provided on the Internet. An attribute data server device 60 is also provided. These server devices 40a, 40b...40n, 60 are provided with terminal devices 20a, 20b...20n so as to be able to communicate via the Internet.

The hardware configuration of the terminal device 20 is shown in FIG. 29. A memory 82, a display 84, a hard disk 86, a DVD-ROM drive 88, a keyboard/mouse 90, and a communication circuit 92 are connected to the CPU 80. The communication circuit 92 is for connecting to the Internet.

An operating system 94 and a terminal program 96 are recorded on the hard disk 86. The terminal program 96 cooperates with the operating system 94 to perform its functions. These programs were recorded on the DVD-ROM 98 and installed on the hard disk 86 via the DVD-ROM drive 88.

The point cloud server device 40 and the attribute data server device 60 also have similar hardware configurations. However, in the point cloud server device 40, instead of the terminal program 96, a point cloud server program is recorded. Furthermore, in the attribute data server device 60, instead of the terminal program 96, an attribute data server program is recorded.

3.3 Three-dimensional model construction process Next, the process of constructing a three-dimensional model based on the plurality of three-dimensional point group data released as described above will be described.

Below, a case will be described in which a three-dimensional model of a specific feature is constructed.

30 and 31 show flowcharts of three-dimensional model construction. In the figure, the terminal device is shown as a flowchart of the terminal program, the attribute data server device is shown as the flowchart of the attribute data server program, and the point cloud server device is shown as the flowchart of the point cloud server program. be.

First, the CPU 80 of the terminal device 20b (hereinafter sometimes abbreviated as the terminal device 20b) accesses the attribute data server 60 and specifies the location where a desired feature exists (step S11). The location is specified by specifying a range using latitude and longitude, for example. It is also possible to input (or search) an address or place name, and convert this into a range of latitude and longitude. Furthermore, the range clicked or selected on the map may be converted into a range of latitude and longitude.

In response to this, the CPU of the attribute server device 60 (hereinafter sometimes abbreviated as the attribute server device) searches the attribute data and searches for the external shape within the specified location (which may include the vicinity outside the location). Region data is extracted (step S21). The attribute server device 60 transmits the extracted external shape area data (planar shape data and its position and height in absolute coordinates), feature name, etc. to the terminal device 20b.

The terminal device 20b displays the outer shape area and the feature name on the display 84 based on the received outer shape area data (step S12). A display example is shown in FIG. The outline areas 102 to 112 of each feature are shown, and the name of the feature is shown in the vicinity.

The user operates the mouse 90 of the terminal device 20b to select the outer shape area of the desired feature. Here, the explanation will proceed assuming that the external area 110 of the building has been selected. In response to this selection, the terminal device 20b specifies the external area data of the external area 110 of the building and transmits a request for three-dimensional point cloud data to each of the point cloud server devices 40a to 40n (step S13).

Each of the point cloud server devices 40a to 40n that receives this request determines whether or not it holds the three-dimensional point cloud data within the relevant external area 110, and if so, returns the data. Here, the explanation will proceed assuming that two of the point cloud server devices 40a to 40n, point cloud server devices 40a and 40b, hold three-dimensional point cloud data of the external area 110 of the building.

The CPU of the point cloud server device 40a (hereinafter sometimes abbreviated as point cloud server device 40a) returns the three-dimensional point cloud data of the external area 110 of the building to the terminal device 20b (step S31).

Similarly, the CPU of the point cloud server device 40b (hereinafter sometimes abbreviated as point cloud server device 40b) returns the three-dimensional point cloud data of the external area 110 of the building to the terminal device 20b (step S41 ).

Note that the point cloud server devices 40a and 40b transmit three-dimensional point cloud data of an area widened in consideration of measurement errors to the designated external area 110. Measurement errors vary depending on the measurement method and equipment, and therefore vary depending on the measurement method and equipment used to measure the three-dimensional point cloud data. Therefore, the widened area described above differs depending on the measurement method and equipment.

The terminal device 20b receives three-dimensional point cloud data (first three-dimensional point cloud data) from the point cloud server device 40a and three-dimensional point cloud data (second three-dimensional point cloud data) from the point cloud server device 40b. receive. FIG. 33A shows an example of the first three-dimensional point group data, and FIG. 33B shows an example of the second three-dimensional point group data. As shown in FIGS. 33A and 33B, the first three-dimensional point group data is measured from the front side of the rectangular parallelepiped toward the page. The second three-dimensional point group data is measured from the back side of the rectangular parallelepiped toward the page.

The terminal device 20b aligns and synthesizes the first three-dimensional point group data and the second three-dimensional point group data, and generates synthesized three-dimensional point group data as shown in FIG. 34 (step S14). As is clear from this figure, the 3D point cloud data (which has measurement data from both the front side and the back side) has a larger amount of information than the 1st 3D point cloud data and the 2nd 3D point cloud data alone. Original point cloud data can be obtained.

The composite three-dimensional point group data obtained in this way can be used for various purposes. In this embodiment, a three-dimensional model of a feature is generated based on synthetic three-dimensional point cloud data.

The terminal device 20b generates an external surface of the feature based on the composite three-dimensional point group data (step S15). This processing can be performed, for example, by a three-dimensional convex hull algorithm (http:/www.qhull.org/download/). FIG. 35A shows the generated three-dimensional model of the feature. It is possible to generate a three-dimensional model with a higher degree of completeness than a three-dimensional model generated using only the first three-dimensional point group data and a three-dimensional model generated only using the second three-dimensional point group data. The terminal device 20b displays this on the display 84 and records it on the hard disk 86 (step S15).

According to this embodiment, it is possible to easily search for a plurality of three-dimensional point cloud data scattered on the Internet regarding the same feature, and to synthesize these three-dimensional point cloud data to obtain composite three-dimensional point cloud data. Further, if one attribute data is generated based on any one of a plurality of three-dimensional point cloud data regarding the same feature, the above-described search and synthesis can be performed. This is because the attribute data is independent of the three-dimensional point group data, and the external area data of the attribute data is defined using absolute coordinates.

3.4 Others
(1) In the above embodiment, two three-dimensional point group data are acquired to obtain composite three-dimensional point group data. However, three or more three-dimensional point group data may be acquired to generate composite three-dimensional point group data.

(2) In the above embodiment, composite three-dimensional point group data about one feature is obtained. However, composite three-dimensional point cloud data regarding a plurality of terrestrial features may be obtained. Alternatively, composite three-dimensional data for all terrestrial features within a predetermined range may be obtained. Such examples are shown in FIGS. 36A, 36B, and 36C. 36A is the first three-dimensional point group data, FIG. 36B is the second three-dimensional point group data, and FIG. 36C is the composite three-dimensional point group data.

(3) In the above embodiment, three-dimensional point group data with different measurement directions are combined. However, three-dimensional point cloud data with different measured distances to terrestrial features or three-dimensional point cloud data with different measuring instruments (for example, measurements by MMS and UAV) may be combined.

(4) In the above embodiment, a case has been described in which only one piece of attribute data is found for a location in step S21. However, in some cases, different measurement companies create 3D point cloud data for the same location and each uploads attribute data, or even if the same measurement company creates 3D point cloud data for the same location at different times. In some cases, point cloud data is created and attribute data is uploaded for each. In such cases, multiple attribute data will be found for the specified location. In this case, the attribute data server device 60 transmits to the terminal device 20b that there is a plurality of attribute data. The terminal device 20b displays the creation date and time, creation device, etc. for a plurality of attribute data on the display 84, and allows the operator to make a selection. Based on the selected attribute data, the external shape area is transmitted (step S21).

(5) In the above embodiment, the terminal device 20b requests three-dimensional point group data (step S13). However, the attribute data server device 60 may perform this and transmit the result to the terminal device 20b.

(6) In the above embodiment, the terminal device 20b synthesizes three-dimensional point group data and generates a three-dimensional model. However, either one or both may be performed by the attribute data server device 60, and the results may be transmitted to the terminal device 20b.

(7) In the above embodiment, when the terminal device 20b acquires the three-dimensional point group data in step S14, it does not display which direction the data was measured from. However, in order to clarify this, the measurement direction and distance to the feature are displayed using arrows, as shown in Figure 35B, based on the data attached to the acquired three-dimensional point cloud data. You can. The base of the arrow is the measurement position, and the direction of the arrow is the measurement direction. In the case of FIG. 35B, it is clear that there is three-dimensional point group data measured from two locations.

Therefore, it can be seen that when performing the next measurement based on this screen display, it is preferable to measure from a direction different from the arrow if the quality of the composite three-dimensional point cloud data is to be improved. Furthermore, if you want to know the temporal changes of a feature, it is preferable to measure from the same position and direction as the arrow.

(8) In the above embodiment, the first and second three-dimensional point group data are simply combined to generate composite three-dimensional point group data. However, if the first 3D point cloud data and the second 3D point cloud data exist for the same location in the 3D point cloud data, only the 3D point cloud data with high measurement accuracy will be used for that location. Alternatively, synthetic three-dimensional point group data may be generated.

(9) In the above embodiment, the first point cloud server device 40a and the second point cloud server device 40b are constructed as separate server devices, but they may be constructed as a single server device.

(10) In the above embodiment, the first point cloud server device 40a, the second point cloud server device 40b, and the attribute data server device 60 are constructed as separate server devices, but they are constructed as a single server device. You can leave it there.

(11) The above embodiments and modified examples can be implemented in combination with other embodiments unless it contradicts the essence thereof.

4. Fourth embodiment
4.1 Overall Configuration FIG. 37 shows the overall configuration of a point cloud data management system according to the second embodiment of the present invention. The first point cloud server device 40a records a first three-dimensional point cloud data file at a first date and time based on absolute coordinates including terrestrial objects. The second point cloud server device 40b records a second three-dimensional point cloud data file at a second date and time based on absolute coordinates including terrestrial objects. The attribute data server device 60 records external area data based on absolute coordinates surrounding the external shape of a feature.

The terminal device 20 can communicate with these server devices. The three-dimensional point cloud data acquisition means 22 of the terminal device 20 acquires external area data of a desired feature from the attribute data server device 60. The three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40a based on the acquired external shape area data of the feature, and converts the three-dimensional point cloud data included in the area into a first three-dimensional point cloud. Obtain as data. Further, the three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40b based on the acquired external shape area data of the feature, and transfers the three-dimensional point cloud data included in the area to the second three-dimensional point cloud data. Acquire as point cloud data.

The temporal change acquisition means 25 detects changes in the shape of the feature between the first date and time and the second date and time based on the acquired first three-dimensional point cloud data at the first date and time and second three-dimensional point cloud data at the second date and time. Calculate.

4.2 System Configuration The system configuration is the same as the third embodiment.

4.3 Process for calculating changes over time Next, a process for obtaining changes over time of features based on a plurality of published three-dimensional point cloud data will be described.

FIGS. 38 and 39 show flowcharts for acquiring changes in features over time. In the figure, the terminal device is shown as a flowchart of the terminal program, the attribute data server device is shown as the flowchart of the attribute data server program, and the point cloud server device is shown as the flowchart of the point cloud server program. be.

First, the CPU 80 of the terminal device 20b (hereinafter sometimes abbreviated as the terminal device 20b) accesses the attribute data server 60 and specifies the location where a desired feature exists (step S11). The location is specified by specifying a range using latitude and longitude, for example. It is also possible to input (or search) an address or place name, and convert this into a range of latitude and longitude. Furthermore, the range clicked or selected on the map may be converted into a range of latitude and longitude.

In response to this, the CPU of the attribute server device 60 (hereinafter sometimes abbreviated as the attribute server device) searches the attribute data and extracts external shape area data within the specified location (step S21). The attribute server device 60 transmits the extracted external shape area data (planar shape data and its position and height in absolute coordinates), feature name, etc. to the terminal device 20b.

The terminal device 20b displays the outer shape area and the feature name on the display 84 based on the received outer shape area data (step S12). A display example is shown in FIG. The outline areas 102 to 112 of each feature are shown, and the name of the feature is shown in the vicinity.

The user operates the mouse 90 of the terminal device 20b to select the outer shape area of the desired feature. Here, the explanation will proceed assuming that the external area 110 of the building has been selected. In response to this selection, the terminal device 20b specifies the external area data of the external area 110 of the building and transmits a request for three-dimensional point cloud data to each of the point cloud server devices 40a to 40n (step S13).

Each of the point cloud server devices 40a to 40n that receives this request determines whether or not it holds the three-dimensional point cloud data within the relevant external area 110, and if so, returns the data. Here, the explanation will proceed assuming that two of the point cloud server devices 40a to 40n, point cloud server devices 40a and 40b, hold three-dimensional point cloud data of the external area 110 of the building.

The CPU of the point cloud server device 40a (hereinafter sometimes abbreviated as point cloud server device 40a) returns the three-dimensional point cloud data of the external area 110 of the building to the terminal device 20b (step S31).

Similarly, the CPU of the point cloud server device 40b (hereinafter sometimes abbreviated as point cloud server device 40b) returns the three-dimensional point cloud data of the external area 110 of the building to the terminal device 20b (step S41 ).

The terminal device 20b receives three-dimensional point cloud data (first three-dimensional point cloud data) from the point cloud server device 40a and three-dimensional point cloud data (second three-dimensional point cloud data) from the point cloud server device 40b. receive. Here, the first three-dimensional point group data and the second three-dimensional point group data are obtained by measuring the terrestrial feature from substantially the same direction. However, the measurement date and time are different.

The terminal device 20b generates a three-dimensional model of the feature at the first date and time based on the first three-dimensional point group data (step S17). Furthermore, a three-dimensional model of the feature at the second date and time is generated based on the second three-dimensional point group data (step S17).

The terminal device 20b displays the generated three-dimensional model of the first date and time and the three-dimensional model of the second date and time on the display 84 so that they can be compared (step S18). For example, as shown in FIG. 40, a three-dimensional model 120 for a first date and time and a three-dimensional model 122 for a second date and time are displayed. Below that, the feature name and measurement date are shown. Therefore, in the example of FIG. 40, it can be seen by comparison that the outer shape of the building on February 24, 2010 and the outer shape of the building on March 5, 2019 have changed.

Note that, in order to facilitate comparison, both may be displayed in different colors in an overlapping manner.

Also, for example, if there is a situation where a feature does not exist at the first date and time but does exist at the second date and time (for example, a new road sign has been installed), then if there is measurement data for that location at the second date and time, then 3D point cloud data for the feature can be obtained. However, even if there is measurement data for that location at the first date and time, 3D point cloud data for the feature cannot be obtained. Therefore, by displaying a 3D model based on this, it becomes clear that the feature does not exist at the first date and time but does exist at the second date and time.

Note that for the purpose of the above-mentioned comparison over time, it is preferable to use three-dimensional point group data in which the measurement direction with respect to the feature is the same (or similar).

4.4 Others
(1) In the above embodiment, three-dimensional models are generated using three-dimensional point cloud data at two dates and times, and the models are compared. However, it is also possible to acquire three-dimensional point cloud data at three or more dates and times, generate three-dimensional models for each, and compare them.

(2) In the above embodiment, a comparison is made over time for one feature. However, comparison over time may be performed for a plurality of terrestrial features. Alternatively, comparison over time may be performed for all terrestrial features within a predetermined range.

(3) In the above embodiment, the terminal device 20b requests three-dimensional point group data (step S13). However, the attribute data server device 60 may perform this and transmit the result to the terminal device 20b.

(6) In the above embodiment, the three-dimensional model is generated in the terminal device 20b. However, the attribute data server device 60 may perform the process and send the result to the terminal device 20b.

(7) In the above embodiment, when the terminal device 20b acquires the three-dimensional point group data in step S17, it does not display when each three-dimensional point group data was measured. However, in order to clarify this, the measurement date and time of each 3D point cloud data may be displayed based on the data attached to the acquired 3D point cloud data, as shown in FIG. good. When three-dimensional point group data for many dates and times exists, it is preferable to perform such display. The user can view this, select a predetermined number of dates to compare, and display the three-dimensional model.

(8) The above embodiments and modified examples can be implemented in combination with other embodiments unless it contradicts the essence thereof. For example, if there is multiple 3D point cloud data from different directions at the same date and time (even the same month or year), create a 3D model by generating composite 3D point cloud data, and then It may also be compared with three-dimensional models of other dates and times.

(9) In the above embodiment, the first point cloud server device 40a and the second point cloud server device 40b are constructed as separate server devices, but they may be constructed as a single server device.

(10) In the above embodiment, the first point cloud server device 40a, the second point cloud server device 40b, and the attribute data server device 60 are constructed as separate server devices, but they are constructed as a single server device. You can leave it there.

(11) In the above embodiment, the searched video data is played back on the terminal device. At this time, a map or three-dimensional point cloud data of the vicinity where the video was captured may also be displayed. Furthermore, three-dimensional point group data viewed from an imaging position that changes over time may be displayed to match the playback of a moving image.

(12) In the above embodiment, three-dimensional models of features at two points in time are displayed for comparison. However, for both, the normal vector of the outline of the three-dimensional model may be displayed to make it easier to understand the change in surface angle.

Alternatively, it may be divided into minute areas (25 cm cubic area) and the number of points within the areas may be compared and displayed. This allows the degree of roughness of the surface of the feature to be compared.

Furthermore, the reflection intensity of each point or area may be compared. This allows the smoothness of the surfaces to be compared.

Alternatively, the volumes of three-dimensional models may be compared. This allows you to compare the growth of plants, the condition of leaves, etc.

(13) The above embodiments and modified examples can be implemented in combination with other embodiments unless it contradicts the essence thereof.

5. Fifth embodiment
5.1 Overall Configuration FIG. 42 shows a functional block diagram of a feature search system according to the fifth embodiment of the present invention. The recording unit 158 of the feature search server device 150 records attribute data such as feature names, external areas, and positions of three-dimensional point cloud data, and video data.

The attribute data transmitting means 150 of the feature search server device 150 reads the feature name or external shape area corresponding to the three-dimensional point cloud data recorded in the recording unit 158 and transmits it to the terminal device 40 . The feature attribute data display means 182 of the terminal device 40 displays the feature name or outer shape area of the received feature on the display unit 188.

An operator operating the terminal device 40 uses the mouse 90 to look at the feature name or outline area of the displayed feature and selects the feature for which the moving image is to be searched. Selection information transmitting means 184 transmits which feature has been selected to feature search server device 150.

The feature position acquisition means 154 of the feature search server device 150 obtains the position of the selected feature. The video data transmitting means 156 searches the video data in chronological order and extracts a portion attached with imaging position information in the vicinity of the feature position. The extracted video data is transmitted to the terminal device 40. The video playback means 186 of the terminal device 40 plays back the received video data on the display unit 188.

This makes it possible to easily search for and display locations where desired terrestrial features are imaged from a huge amount of video data.

5.2 System Configuration FIG. 43 shows the system configuration of the point cloud data management system according to the third embodiment. Point cloud server devices 40a, 40b, . . . 40n are provided on the Internet. A feature search server device 62 is also provided. The point cloud server devices 40a, 40b, . . . , 40n record three-dimensional point cloud data of measured features and video data (with imaging position information) captured at the time of measurement. Attribute data as shown in FIG. 11 is recorded in the feature search server device 62. These server devices 40a, 40b...40n, 60 are provided with terminal devices 20a, 20b...20n so as to be able to communicate via the Internet.

The hardware configuration of the terminal device 20 is the same as that shown in FIG. 29. The point cloud server device 40 and the feature search server device 62 also have similar hardware configurations. However, in the point cloud server device 40, instead of the terminal program 96, a point cloud server program is recorded. Furthermore, in the feature search server device 62, a feature search server program is recorded in place of the terminal program 96.

5.3 Feature Search Process Figure 44 shows a flowchart of feature search. In the figure, what is shown as a terminal device is a flowchart of a terminal program, and what is shown as a feature search server device is a flowchart of a feature search server program.

First, the CPU 80 of the terminal device 20b (hereinafter sometimes abbreviated as the terminal device 20b) accesses the feature search server device 62 and specifies the location where a desired feature exists (step S11). The location is specified by specifying a range using latitude and longitude, for example. It is also possible to input (or search) an address or place name, and convert this into a range of latitude and longitude. Furthermore, the range clicked or selected on the map may be converted into a range of latitude and longitude.

In response to this, the CPU of the feature search server device 62 (hereinafter sometimes abbreviated as feature search server device 62) searches the attribute data and extracts the external shape area data within the specified location ( Step S51). The feature search server device 62 transmits the extracted external shape area data (planar shape data and its position and height in absolute coordinates), feature name, etc. to the terminal device 20b.

The terminal device 20b displays the outer shape area and the feature name on the display 84 based on the received outer shape area data (step S12). A display example is shown in FIG. The outline areas 102 to 112 of each feature are shown, and the name of the feature is shown in the vicinity.

The user operates the mouse 90 of the terminal device 20b to select the outer shape area of the desired feature. Here, the description will proceed assuming that the outer shape area 106 of the marker has been selected. In response to this selection, the terminal device 20b specifies the outer shape area data of the outer shape region 106 of the sign and sends an instruction to the feature search server device 62 to search for the part of the video in which this sign is imaged. (Step S19).

The feature search server device 62 reads the attribute data of the specified feature and acquires the location of the feature (position based on absolute coordinates) (step S52). Next, the feature search server device 62 extracts the portion where the feature is imaged in the captured video data 162 based on the location of the feature (step S53).

FIG. 45 shows the data structure of the captured moving image data 162. Time stamps TS1, TS2, . . . indicating imaging times are recorded in association with the moving image body data 170. Additionally, the position of the camera that captured the image (or the vehicle equipped with the camera, etc.) is recorded in absolute coordinates. Further, although not shown, the captured moving image data 162 is recorded in association with imaging area information for the entire moving image.

FIG. 46 shows the concept of imaging area information. The imaging path R is indicated by a solid line. It is shown that imaging started from the starting point S and was performed until the ending point E. An imaging area AR is shown by a broken line surrounding this imaging path R. The imaging area AR is specified, for example, by diagonal absolute coordinates (imaging area information).

FIG. 47 shows details of the moving image portion extraction process in step S53. The feature search server device 62 selects the video data 162 in which the measurement position is included in the imaging area based on the position of the feature to be searched (step S531). The video data 162 selected in this way may include a portion where a terrestrial feature is imaged. The feature search server device 62 searches for each of the selected video data 162 to see if the imaged portion exists.

In step S533, the feature search server device 62 reads out the imaging positions recorded in the video data 162 in chronological order (step S533). In the example of FIG. 45, the imaging positions PS1, PS2, . . . PSn are read out.

The feature search server device 62 identifies the imaging position PSf closest to the location of the feature among the read imaging positions PS1, PS2, . . . PSn (step S534). The feature search server device 62 determines whether the distance between the closest imaging position PSf and the feature position is less than or equal to a predetermined value (step S535). If it is less than or equal to the predetermined value, it means that a feature is being imaged at the location. The feature search server device 62 extracts video data for a predetermined period of time before and after the identified imaging position PSf (step S536). If the distance between the imaging position PSf and the feature position exceeds a predetermined value, the moving image portion is not extracted.

The above process is performed for all the videos selected in step S531 (steps S532, S537).

After extracting the video data portion as described above, the feature search server device 62 transmits it to the terminal device 20b (step S54).

In response to this, the terminal device 20b displays the extracted video data portion on the display 84 (step S20). Note that when there is a plurality of moving image data, it is preferable to display a list of the moving image data together with the imaging date and time before playback, and allow the user to select and play back.

By watching this video, the user can understand the situation of the search target feature as an image. For example, when you want to know the condition of features that can be identified on the screen display in Figure 32 (checking looseness detection marks (for example, matching marks) attached to bolts and nuts, checking the state of appearance deterioration, etc.) It can be used for.

Note that the above-mentioned confirmation of the detection mark and confirmation of the state of appearance deterioration may be performed by the terminal device using image processing or AI.

5.4 Others
(1) In the above embodiment, the feature position of attribute data and the imaging position of video data are indicated in absolute coordinates. Therefore, without having to prepare attribute data corresponding to each piece of video data, it is possible to search for a feature from a plurality of video data as long as there is one piece of attribute data.

In addition, if attribute data is provided corresponding to each video data, the terrestrial feature position and the imaging position do not need to be absolute coordinates as long as they are consistent. Additionally, if there is attribute data corresponding to the video data, it is also possible to search based on the date and time when the 3D point cloud data of the feature was created and the date and time of shooting, instead of searching based on the feature location and imaging location. .

(2) In the above embodiment, as shown in FIG. 32, the feature to be searched for is specified by the outer shape area and the feature name. However, it is also possible to display only the name of the feature without displaying the external area and to allow the user to select it.

(3) In the above embodiment, the portion of the video in which the feature is captured is extracted and played back. However, in addition to this, it is also possible to mark the features in the video and play it back.

The feature search server device 62 can realize this by performing the following processing. For example, when there is an external shape area of a feature as shown in FIG. 48, let us assume that the sign 106 is the search target. A trajectory 200 when a moving image is captured is drawn in the space in which this outer shape area is arranged. Since time is attached to this trajectory 200, the imaging time at each point can be known. At each imaging time t1, t2, . . . , it is determined in which direction and at what distance the target feature 106 exists.

Based on this direction and distance, it calculates how the feature 106 is imaged at the corresponding time in the captured video. The appearance of the outer region of the feature 106 generated in this way at each time is superimposed on the extracted video and transmitted to the terminal device 20b.

When this is played back on the terminal device 20b, a frame of the external area will be displayed to surround the target feature in the image, as shown in FIG. Therefore, the user can easily find the target feature in the video.

(4) In the above embodiment, the point cloud server device 40 and the search server device 62 are constructed as separate server devices, but they may be constructed as a single server device.

(5) In the above embodiment, the case where a video is searched has been described. However, the same process can also be used to search for still images with an image capture location and image capture time attached.

(6) The above embodiments and modified examples can be implemented in combination with other embodiments unless it contradicts the essence thereof.

Claims (26)

A feature management system comprising a server device and a terminal device capable of communicating with the server device,
The server device includes:
a recording unit that records three-dimensional point cloud data including features with defined positions;
receiving means for receiving a captured image including a feature and a captured position transmitted from the terminal device;
Centering the received imaging position, the three-dimensional point cloud data is projected onto a plane from different angles to generate two-dimensional projected images from a plurality of angles, and the captured image including the received terrestrial feature and the generated image are generated. and a captured image/point cloud data correspondence means for associating the captured image of the feature with the three-dimensional point cloud data of the feature based on the comparison with each of the two-dimensional projected images,
The terminal device is
1. A feature management system comprising: image transmitting means for transmitting a captured image including a feature captured by a camera to a feature management server device along with an imaging position. a recording unit that records three-dimensional point cloud data including features with defined positions;
a receiving means for receiving a captured image including a feature and a captured position transmitted from a terminal device;
Centering the received imaging position, the three-dimensional point cloud data is projected onto a plane from different angles to generate two-dimensional projected images from a plurality of angles, and the captured image including the received terrestrial feature and the generated image are generated. a captured image/point cloud data correspondence means for associating the captured image of the feature with the three-dimensional point cloud data of the feature based on the comparison with each of the two-dimensional projected images;
A server device equipped with A server program for realizing a server device by a computer, the computer
a receiving means for receiving a captured image including a feature and a captured position transmitted from a terminal device;
Centering on the received imaging position, three-dimensional point cloud data including defined features recorded in the recording unit is projected onto a plane from different angles to generate two-dimensional projected images from a plurality of angles. and a captured image/point cloud that associates the captured image of the feature with the three-dimensional point cloud data of the feature based on a comparison between the received captured image including the feature and each of the generated two-dimensional projection images. A server program that functions as a data handling means. The system, device or program according to any one of claims 1 to 3,
The captured image/point cloud data correspondence means includes:
a captured image outer shape generating means that indicates an outer shape of a feature in the captured image;
a two-dimensional projection image outer shape generation means that indicates the outer shape of the feature in each of the two-dimensional projection images;
Compare the external shape of the feature in the captured image with the external shape of the feature in each of the two-dimensional projection images, select a two-dimensional projection image whose external shape matches, and compare the captured image of the feature with the feature A system, device, or program comprising means for associating three-dimensional point cloud data of In the system, device or program according to any one of claims 1 to 4,
The recording unit of the server device records an outer shape area surrounding the outer shape of the feature and a feature name,
The captured image/point cloud data correspondence means includes:
comprising a feature estimation means for extracting a feature part from the captured image and estimating a feature name;
Centering on the received imaging position, three-dimensional point cloud data of features having the same feature name as the estimated feature name is projected onto a plane from different angles to generate two-dimensional projected images from multiple angles. and, based on a comparison between the received captured image of the feature portion and each of the generated two-dimensional projection images, the captured image of the feature is associated with the three-dimensional point cloud data of the feature. A system, device or program. In the system, device or program according to any one of claims 1 to 4,
The recording unit of the server device records an outer shape area surrounding the outer shape of the feature and a feature name,
The image transmitting means of the terminal device receives an input from an operator, adds a boundary box and a feature name of a feature in the captured image, and transmits the result to the server device;
The captured image/point cloud data correspondence means includes:
Three-dimensional point cloud data of a feature having the same feature name as the transmitted feature name is projected onto a plane from different angles, centering on the received imaging position, to create two-dimensional projected images from multiple angles. and, based on a comparison between the received captured image of the feature portion and each of the generated two-dimensional projection images, the captured image of the feature is associated with the three-dimensional point cloud data of the feature. A system, device, or program that The system, device or program according to any one of claims 1 to 6,
The photographed image/point cloud data correspondence means generates a plurality of virtual imaging positions in the vicinity of the imaging position, and generates a two-dimensional projection image in which three-dimensional point cloud data is projected onto a plane using each virtual imaging position as a reference. A system, device or program characterized by: The system, device or program according to any one of claims 1 to 7,
A system, device, or program characterized in that the photographed image/point cloud data correspondence means also specifies an imaging position and an imaging direction with respect to a terrestrial feature. The system, device or program according to any one of claims 1 to 8,
An imaging direction is also attached to the photographed image of the feature transmitted from the terminal device,
A system, device, or program, wherein the photographed image/point cloud data correspondence means narrows down the angle for generating the two-dimensional projection image based on the photographing direction. The system, device or program according to any one of claims 1 to 9,
A system, device, or program characterized in that the three-dimensional point group data and the outer shape area of the attribute data are indicated in absolute coordinates. The system, device or program according to any one of claims 1 to 10,
The feature extracting means projects the external shape area data of the feature or the three-dimensional point cloud data of the feature onto a plane based on the imaging position and imaging direction of the feature in the past, and generates a past two-dimensional projection image. and send this to the second terminal device,
A system, device, or program characterized in that the second terminal device displays the two-dimensional positional projection image superimposed on a finder image of the camera when capturing an image with the camera. The system, device or program according to any one of claims 1 to 11,
The server device includes:
a point cloud data server device that records the three-dimensional point cloud data;
A system, device, or program comprising: an attribute data server device that records attribute data including the external shape area. Record 3D point cloud data including features with defined positions,
generating two-dimensional projected images from a plurality of angles by projecting the three-dimensional point cloud data onto a plane from different angles, centering on the imaging position at which the captured image including the terrestrial feature was captured;
A feature management method characterized by associating a captured image of the feature with three-dimensional point cloud data of the feature based on a comparison between the captured image and each of the generated two-dimensional projection images. A feature management system comprising a server device and a terminal device capable of communicating with the server device,
The server device includes:
a recording unit that records three-dimensional point group data in which a first area and a second area with defined positions are distinguished;
Receiving means for receiving a captured image including a first area and a second area and an image capturing position transmitted from the terminal device;
region estimating means for distinguishing and extracting a first region and a second region from other parts based on the captured image;
Centering the received imaging position, the differentiated three-dimensional point group data of the first region and the second region are projected onto a plane from different angles to generate two-dimensional projected images from a plurality of angles, and the a captured image/point cloud data correspondence means for associating the captured image with the three-dimensional point cloud data based on a comparison between the differentiated captured images of the first region and the second region and each of the generated two-dimensional projection images; Equipped with
The terminal device is
1. A feature management system comprising: image transmitting means for transmitting a captured image including a first region and a second region captured by a camera to a feature management server device together with an imaging position. a recording unit that records three-dimensional point group data in which a first area and a second area with defined positions are distinguished;
receiving means for receiving a captured image including the first region and the second region and the captured position transmitted from the terminal device;
region estimating means for distinguishing and extracting a first region and a second region from other parts based on the captured image;
Centering the received imaging position, the differentiated three-dimensional point group data of the first region and the second region are projected onto a plane from different angles to generate two-dimensional projected images from a plurality of angles, and the a captured image/point cloud data correspondence means for associating the captured image with the three-dimensional point cloud data based on a comparison between the differentiated captured images of the first region and the second region and each of the generated two-dimensional projection images; ,
A server device equipped with A server program for realizing a server device by a computer, the computer
receiving means for receiving a captured image including the first region and the second region and the captured position transmitted from the terminal device;
region estimating means for distinguishing and extracting a first region and a second region from other parts based on the captured image;
Centering on the received imaging position, the differentiated three-dimensional point group data of the first area and the second area recorded in the recording unit are projected onto a plane from different angles, and two-dimensional projection is performed from a plurality of angles. A captured image that generates an image and associates the captured image with three-dimensional point cloud data based on a comparison between the differentiated captured images of the first region and the second region and each of the generated two-dimensional projection images. A server program that functions as a point cloud data compatible means. A feature management system comprising a server device and a mobile terminal device capable of communicating with the server device,
The server device includes:
a recording unit that records three-dimensional point cloud data including terrestrial objects, past imaging positions, and imaging directions;
A past two-dimensional projected image is generated by projecting the three-dimensional point cloud data of the feature or its three-dimensional model onto a plane based on the past imaging position and imaging direction, and the past two-dimensional projected image is transmitted to the terminal device. and generating means,
The terminal device is
1. A feature management system comprising: display means for superimposing a past two-dimensional projection image received from a server device as a transparent image on a finder image of a camera for capturing an image of the feature on a display unit. a recording unit that records three-dimensional point cloud data including terrestrial objects, past imaging positions, and imaging directions;
Past two-dimensional projection of generating a past two-dimensional projection image by projecting three-dimensional point cloud data of the feature or its three-dimensional model onto a plane based on the past imaging position and imaging direction, and transmitting the generated past two-dimensional projection image to a mobile terminal device. image generating means;
A server device equipped with A server program for realizing a server device using a computer, the computer
Based on the past imaging position and imaging direction recorded in the recording unit, project the 3D point cloud data of the feature or its 3D model onto a plane to generate a past 2D projection image, and send it to the terminal device. A server program for functioning as a past two-dimensional projection image generation means. Record 3D point cloud data including terrestrial objects, past imaging positions, and imaging directions.
generating a past two-dimensional projected image by projecting three-dimensional point cloud data of the feature or its three-dimensional model onto a plane based on the past imaging position and imaging direction;
A feature management method characterized by displaying past two-dimensional projected images superimposed on a camera finder image. A feature management system comprising a server device and a mobile terminal device capable of communicating with the server device,
The server device includes:
a recording unit that records three-dimensional point cloud data including terrestrial objects, past imaging positions, and imaging directions;
and transmitting means for transmitting the imaging position and imaging direction to the terminal device,
The terminal device is
A feature management system comprising: a display unit that displays the received imaging position and imaging direction on a map on a display unit. Record 3D point cloud data including terrestrial objects, past imaging positions, and imaging directions.
A feature management method characterized by displaying the past imaging positions and imaging directions on a map. A feature management system comprising a server device and a terminal device capable of communicating with the server device,
The server device records three-dimensional point cloud data with reflection intensity including road display features and external shape area data surrounding the outer shape of the road display features,
The terminal device is
external shape area data acquisition means for acquiring external shape area data of the road display feature from the server device;
3D point cloud data acquisition means for acquiring 3D point cloud data corresponding to the external shape area data of the road display feature from the server device;
a display means for superimposing and displaying the acquired outer shape area of the road display feature and three-dimensional point cloud data on a display unit;
A feature management system characterized by: External shape area data acquisition means for acquiring external shape area data of the road display feature from a server device;
three-dimensional point cloud data acquisition means for acquiring three-dimensional point cloud data corresponding to the external shape area data of the road display feature from the server device;
a display means for superimposing and displaying the acquired outer shape area of the road display feature and three-dimensional point cloud data on a display unit;
A terminal device equipped with A server program for realizing a terminal device by a computer, which
External shape area data acquisition means for acquiring external shape area data of the road display feature from a server device;
three-dimensional point cloud data acquisition means for acquiring three-dimensional point cloud data corresponding to the external shape area data of the road display feature from the server device;
A terminal program for functioning as a display means for superimposing and displaying the acquired external area of a road display feature and three-dimensional point cloud data on a display unit. Recording three-dimensional point cloud data with reflection intensity including the road display feature and external shape area data surrounding the outer shape of the road display feature,
Obtaining the outer shape area data of the road display feature,
Obtaining three-dimensional point cloud data corresponding to the outer shape area data of the road display feature,
A feature management method characterized by displaying an acquired external shape area of a road display feature and three-dimensional point cloud data on a display unit in an overlapping manner.

Images