JP2018116005A - Three-dimensional data processing apparatus, three-dimensional data measuring machine, and three-dimensional data processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress a computational complexity from increasing without lowering difference extraction precision when extracting a difference between three-dimensional data obtained through three-dimensional measurement at different points of time.SOLUTION: A three-dimensional data processing apparatus 130 comprises an image recognition part 133 and a difference extraction part 135. Reference data and comparison data are three-dimensional data obtained through three-dimensional measurement at mutually different points of time. The image recognition part 133 determines, from images photographed in parallel with the three-dimensional measurement for obtaining the comparison data, whether features present at the point of time when three-dimensional measurement for obtaining the reference data is performed are still present even at the point of time when the three-dimensional measurement for obtaining the comparison data is performed. The difference extraction part 135 adjusts the comparison data within different wide and narrow ranges according to a determination result of the image recognition part 133 to position the reference data and comparison data, and then extracts differences between the reference data and comparison data.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a three-dimensional data processing device, a three-dimensional data measuring instrument, and a three-dimensional data processing method.

  As described in Patent Document 1, in recent years, there has been a demand for three-dimensional measurement of features around roads due to demand for high accuracy of road management ledgers, and MMS has been developed. “MMS” is an abbreviation for Mobile Mapping System. In MMS, a measuring vehicle is equipped with a three-dimensional measuring device, and the three-dimensional point cloud data is acquired by measuring roads and features around the road while the measuring vehicle is running. Map data can be generated.

  As described in Non-Patent Document 1, as an application technology of MMS, development of a dynamic map in which dynamic information such as vehicles, weather, traffic jams, and accidents is linked to static information that is a three-dimensional map is performed. It is expected to be used for automatic driving technology of vehicles.

JP 2009-204615 A

Nikkei PC Editing Department, “Dynamic Map”, [online], July 19, 2016, ITpro, [October 11, 2016 search], Internet <URL: http: // itpro. nikkeibp. co. jp / atcl / keyword / 15/0500900002/062100011 />

  In order to link dynamic information to static information, it is necessary to extract dynamic information from three-dimensional map data. In order to extract dynamic information, the reference map data, which is the reference three-dimensional map data, is compared with the comparison map data, which is the three-dimensional map data obtained at the same location as the reference map data at different times. A technique for extracting a change point over time has been developed.

The change point over time corresponds to the difference between the reference map data and the comparison map data. In the above technique, the difference is extracted by the following procedure.
(1) The point group of the reference map data is compared with the point group of the comparative map data, and a similar part that is a part where the features of the point group are similar is extracted.
(2) After extracting similar parts, the two map data are collated. In the collation, the process of translating or rotating the other map data is repeated for one map data. At this time, the degree of coincidence of the positions of similar parts in both map data is digitized for each translation and rotation.
(3) It is determined whether or not the reference map data and the comparison map data are overlapped from the degree of coincidence of the positions of the similar parts digitized in (2). At this time, point groups with greatly different positions are extracted as differences in the overlapped map data.

  In the above technique, as the number of change points that are differences between the reference map data and the comparison map data increases, the amount of calculation for matching the reference map data and the comparison map data increases, and the matching becomes difficult.

  An object of the present invention is to suppress an increase in the amount of calculation without degrading the accuracy of extracting a difference when extracting a difference between three-dimensional data obtained by three-dimensional measurement at different time points.

A three-dimensional data processing apparatus according to one aspect of the present invention is provided.
A tertiary that extracts a difference between reference data that is three-dimensional data obtained by three-dimensional measurement at a certain time point and comparison data that is three-dimensional data obtained by three-dimensional measurement at a time point different from the reference data. An original data processing device,
A feature existing at the time when the three-dimensional measurement for obtaining the reference data is performed from an image photographed in parallel with the three-dimensional measurement for obtaining the comparison data is a three-dimensional for obtaining the comparison data. An image recognition unit that determines whether or not it exists even at the time of measurement;
The reference data and the comparison data are aligned by adjusting at least one of the reference data and the comparison data within a wide and narrow range depending on the determination result of the image recognition unit, and then the reference data A difference extraction unit for extracting a difference between the data and the comparison data;

  In the present invention, when the difference between the reference data and the comparison data is extracted, by utilizing the image recognition function, it is possible to suppress an increase in the amount of calculation without deteriorating the difference extraction accuracy.

FIG. 3 shows a configuration of a measurement vehicle according to Embodiment 1. FIG. 3 is a diagram showing a configuration of a three-dimensional data measuring device according to the first embodiment. FIG. 3 is a block diagram showing a configuration of a three-dimensional data measuring device according to Embodiment 1. 5 is a flowchart showing an operation of a difference extraction unit of the three-dimensional data processing apparatus according to the first embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals. In the description of the embodiments, the description of the same or corresponding parts will be omitted or simplified as appropriate. The present invention is not limited to the embodiments described below, and various modifications can be made as necessary. For example, the embodiment described below may be partially implemented.

Embodiment 1 FIG.
This embodiment will be described with reference to FIGS.

*** Explanation of configuration ***
With reference to FIG. 1, the structure of the measurement vehicle 100 which concerns on this Embodiment is demonstrated.

  The measurement vehicle 100 includes a three-dimensional data measurement machine 101 and a measurement carriage 102.

  The three-dimensional data measuring machine 101 is an apparatus having both an image recognition function and a three-dimensional measurement function.

  The measurement carriage 102 is a vehicle in the present embodiment, but may be replaced with a moving body other than the vehicle as long as it has a function of moving on the road.

  With reference to FIG. 2 and FIG. 3, the structure of the three-dimensional data measuring device 101 which concerns on this Embodiment is demonstrated.

  The three-dimensional data measuring machine 101 includes a laser scanner 110, a camera 120, and a three-dimensional data processing device 130. The laser scanner 110 and the camera 120 are each electrically connected to the three-dimensional data processing device 130.

  The three-dimensional data measuring device 101 can communicate with the data server 200 via a network such as a mobile communication network (not shown).

  The three-dimensional data processing device 130 is a computer. The three-dimensional data processing device 130 includes a processor 141 and other hardware such as a memory 142 and a communication device (not shown). The processor 141 is connected to other hardware via a signal line, and controls these other hardware.

  The three-dimensional data processing apparatus 130 includes a data recording unit 131, a first data generation unit 132, an image recognition unit 133, a second data generation unit 134, and a difference extraction unit 135 as functional elements. The functions of the data recording unit 131, the first data generation unit 132, the image recognition unit 133, the second data generation unit 134, and the difference extraction unit 135 are realized by software.

  The processor 141 is an IC that performs various processes. “IC” is an abbreviation for Integrated Circuit. The processor 141 is, for example, a CPU. “CPU” is an abbreviation for Central Processing Unit.

  The memory 142 is, for example, a flash memory or a RAM. “RAM” is an abbreviation for Random Access Memory.

  Although not shown, the communication device includes a receiver that receives various data from the laser scanner 110, the camera 120, and the data server 200, and a transmitter that transmits various data to the laser scanner 110, the camera 120, and the data server 200. The communication device is, for example, a communication chip or a NIC. “NIC” is an abbreviation for Network Interface Card.

  The three-dimensional data processing device 130 may include an input device and a display as hardware.

  The input device is, for example, a mouse, a keyboard, or a touch panel.

  The display is, for example, an LCD. “LCD” is an abbreviation for Liquid Crystal Display.

  The memory 142 stores a three-dimensional data processing program that is a program for realizing the functions of the data recording unit 131, the first data generation unit 132, the image recognition unit 133, the second data generation unit 134, and the difference extraction unit 135. Yes. The three-dimensional data processing program is read into the processor 141 and executed by the processor 141. The memory 142 also stores an OS. “OS” is an abbreviation for Operating System. The processor 141 executes a three-dimensional data processing program while executing the OS. A part or all of the three-dimensional data processing program may be incorporated in the OS.

  The three-dimensional data processing program and the OS may be stored in the auxiliary storage device. The auxiliary storage device is, for example, a flash memory or an HDD. “HDD” is an abbreviation for Hard Disk Drive. The three-dimensional data processing program and OS stored in the auxiliary storage device are loaded into the memory 142 and executed by the processor 141.

  The three-dimensional data processing apparatus 130 may include a plurality of processors that replace the processor 141. The plurality of processors share the execution of the three-dimensional data processing program. Each processor is an IC that performs various processes in the same manner as the processor 141.

  Information, data, signal values, and variable values indicating the processing results of the data recording unit 131, the first data generation unit 132, the image recognition unit 133, the second data generation unit 134, and the difference extraction unit 135 are stored in the memory 142 and auxiliary storage. It is stored in a device or a register or cache memory in the processor 141.

  The three-dimensional data processing program may be stored in a portable recording medium such as a magnetic disk and an optical disk.

*** Explanation of operation ***
With reference to FIG. 3, the operation of the three-dimensional data measuring apparatus 101 according to the present embodiment will be described. The operation of the three-dimensional data measuring device 101 corresponds to the three-dimensional data processing method according to the present embodiment.

  The three-dimensional data processing device 130 extracts a difference between the reference data and the comparison data. The reference data is three-dimensional data obtained by three-dimensional measurement at a certain time. Specifically, the three-dimensional data is three-dimensional point cloud data. The comparison data is three-dimensional data obtained by three-dimensional measurement at a different time from the reference data. Here, the time point at which the three-dimensional measurement for obtaining the reference data is performed is referred to as “reference time point”, and the time point at which the three-dimensional measurement for obtaining the comparison data is performed is referred to as “comparison time point”. The comparison time point is a time point after the reference time point in the present embodiment, but may be a time point before the reference time point.

  The laser scanner 110 performs three-dimensional measurement for obtaining comparison data. Specifically, the laser scanner 110 performs three-dimensional measurement around the measurement carriage 102. The laser scanner 110 outputs the result of the three-dimensional measurement to the first data generation unit 132 as scan data.

  The camera 120 captures an image in parallel with the three-dimensional measurement by the laser scanner 110. Specifically, the camera 120 captures an image around the measurement carriage 102. The camera 120 outputs the captured image to the image recognition unit 133 as captured data.

  The data server 200 stores the three-dimensional point cloud data and the like acquired and processed by the three-dimensional data processing device 130 and transmits the data to the data recording unit 131 by wireless communication or the like.

  The data recording unit 131 receives pattern image data and reference data with pattern matching information from the data server 200. The data recording unit 131 outputs the pattern image data to the image recognition unit 133 and the reference data with pattern matching information to the difference extraction unit 135. The data recording unit 131 stores the received pattern image data and reference data with pattern matching information in the memory 142.

  The pattern image data is recorded before the three-dimensional measurement for obtaining the comparison data, and the reference data with pattern matching information is recorded before the difference between the reference data and the comparison data is extracted. 131 need to be stored.

  In the present embodiment, the pattern image data is an image taken by the camera 120 at the reference time, but may be an image obtained by means different from the camera 120. In this embodiment, the reference data is three-dimensional point group data obtained from the result of three-dimensional measurement by the laser scanner 110 at the reference time point. However, the three-dimensional data obtained by means different from the laser scanner 110 is used. It may be.

  In the present embodiment, the pattern image data is an image of a feature that is unlikely to change in shape and position after the date and time when three-dimensional measurement for obtaining reference data is performed. Preferable examples of the pattern image data include a traffic light image and a road sign image.

  The first data generation unit 132 receives input of scan data. The first data generation unit 132 generates three-dimensional point group data from the scan data, and outputs it to the second data generation unit 134.

  The image recognition unit 133 receives input of pattern image data and shooting data. The image recognizing unit 133 extracts a portion that matches the pattern image data stored in the memory 142 in advance by the data recording unit 131 from the photographing data input from the camera 120 by an image recognition method. The image recognition unit 133 outputs the extracted location information as pattern matching information to the second data generation unit 134. As an image recognition method, an arbitrary method such as a template matching method can be used.

  As described above, the image recognition unit 133 determines whether or not the feature existing at the reference time point also exists from the image photographed in parallel with the three-dimensional measurement for obtaining the comparison data. The image recognition unit 133 may detect the feature from the image by an arbitrary method, but in the present embodiment, the feature is detected from the image by pattern matching.

  The second data generation unit 134 receives input of pattern matching information and three-dimensional point cloud data. The second data generation unit 134 associates pattern matching information with the three-dimensional point cloud data, generates comparison data with pattern matching information, and outputs the comparison data to the difference extraction unit 135. The pattern matching information is linked by comparing the position information of the matching portion included in the pattern matching information with the position information held by each point data included in the three-dimensional point cloud data. For each piece of three-dimensional point data, a pattern corresponding flag, which is a parameter for distinguishing whether or not the feature corresponds to the pattern image data, is defined. The pattern corresponding flag corresponding to the point data matching the position information of the matching portion included in the pattern matching information is set valid, and the others are set invalid.

  The difference extraction unit 135 receives input of reference data with pattern matching information and comparison data with pattern matching information. The difference extraction unit 135 collates the position information of the feature with the reference data and the comparison data, and stores the point cloud data whose positions do not match in the memory 142 as the difference 3D point cloud data. The difference 3D point cloud data stored in the memory 142 is appropriately output to the outside.

  A specific operation of the difference extraction unit 135 will be described with reference to FIG.

  In step S1, pattern matching feature number acquisition processing is performed. In step S2, pattern matching feature number determination processing is performed. In step S3 to step S6, a collation range substitution process is performed. In step S7, collation initial setting processing is performed. In step S8, a comparison data movement / rotation process is performed. In step S9, a coincidence degree calculation process is performed. In step S10, an end flag determination process is performed. In step S11, a difference extraction process is performed.

  In step S <b> 1, the difference extraction unit 135 acquires the number N of features detected by pattern matching based on the pattern matching information associated with the comparison data that is the output of the second data generation unit 134.

  In step S <b> 2, the difference extraction unit 135 determines whether the value of N acquired in step S <b> 1 is 0, 1, 2 or 3 or more. When the value of N is 0, the collation range substitution process of step S3 is performed. When the value of N is 1, the collation range substitution process of step S4 is performed. When the value of N is 2, the collation range substitution process of step S5 is performed. When the value of N is 3 or more, the collation range substitution process of step S6 is performed. In addition, conditions, such as the geometric arrangement relationship of the feature detected by pattern matching, may be added to the determination conditions in step S2. Further, the number of processing branches may be three or less or five or more instead of four.

  In step S3 to step S6, the difference extraction unit 135 sets an upper limit value of the shift width of the comparison data movement and rotation performed in step S8. The larger the upper limit value is, the larger the range of movement and rotation is. The larger the range is, the larger the number of iterations is to be performed, resulting in a larger calculation amount. In steps S3 to S6, different upper limit values are set depending on the number of N.

  Here, since the image of the feature with a low possibility of change in shape and position is selected as the pattern image data, the point data in which the pattern corresponding flag is valid is compared with the point data in which the pattern corresponding flag is invalid, The probability of entering differential 3D point cloud data is low. Therefore, by using the position information of the features detected by pattern matching, the number of processes required for subsequent verification can be limited.

  In step S7, the difference extraction unit 135 arranges reference data and comparison data in the same three-dimensional space. If N is determined to be other than 0 in step S2, that is, if there is a feature detected by pattern matching, the reference data and the comparison data are arranged so that the feature overlaps.

  In step S8, the difference extraction unit 135 moves or rotates the comparison data within a range that does not exceed the upper limit set in any of steps S3 to S6. Note that not the comparison data but the reference data may be moved or rotated, or both the reference data and the comparison data may be moved or rotated.

  In step S9, the difference extraction unit 135 calculates the degree of coincidence of the point cloud data between the reference data and the comparison data.

  In step S10, the difference extraction unit 135 determines whether the end flag is valid or invalid. If the end flag is valid, the difference extraction process in step S11 is performed. If the end flag is invalid, the comparison data movement / rotation process in step S8 is performed again. The end flag is set to be valid when the degree of coincidence calculated in step S9 is below a certain value. The end flag may be set based on the number of repetitions of the comparison data movement / rotation process in step S8, the limit value of the degree of coincidence between the reference data and the comparison data, or a composite condition thereof, in addition to the degree of coincidence. .

  In step S11, the difference extraction unit 135 extracts point cloud data that does not overlap as a difference in the reference data and the comparison data overlapped up to step S10.

  As described above, the difference extraction unit 135 adjusts the comparison data within different ranges depending on the determination result of the image recognition unit 133 to align the reference data with the comparison data, and then compares the comparison data with the reference data. Extract differences from data. Specifically, if the image recognition unit 133 determines that the feature existing at the reference time is also present at the comparison time, the difference extraction unit 135 matches the position of the feature in the reference data and the comparison data in step S7. After the adjustment to be performed, the reference data and the comparison data are aligned by adjusting within the above range in step S8. And the difference extraction part 135 extracts the difference of reference | standard data and comparison data after step S9. Note that the difference extraction unit 135 may adjust the reference data instead of adjusting the comparison data when aligning the reference data and the comparison data, or adjust both the reference data and the comparison data. May be. That is, the difference extraction unit 135 may adjust at least one of the reference data and the comparison data.

  In the present embodiment, the difference extraction unit 135 sets the range of adjustment of the comparison data according to the number of objects that are determined by the image recognition unit 133 to exist at the comparison time as the features existing at the reference time. Specifically, the difference extraction unit 135 divides the number of objects determined by the image recognition unit 133 to exist at the comparison time as the features existing at the reference time, and the comparison in step S8 is performed as the number of the features increases. Set the data adjustment range narrow. In this embodiment, four stages are divided, but three stages or less, or five stages or more may be performed.

*** Explanation of the effect of the embodiment ***
In the present embodiment, when the difference between the reference data and the comparison data is extracted, an increase in the amount of calculation can be suppressed without degrading the difference extraction accuracy by utilizing the image recognition function.

  In the present embodiment, the three-dimensional data measuring machine 101 includes a three-dimensional data processing device 130 that is a computer for performing image recognition, three-dimensional data generation, and differential extraction of a plurality of three-dimensional data, and a ground on a road. A laser scanner 110 for acquiring position information of an object and a camera 120 for photographing a feature on a road are provided. The three-dimensional data processing apparatus 130 includes an image recognition unit 133 that extracts an image of a feature from photographed data based on pattern image data input in advance. The three-dimensional data processing device 130 further includes a first data generation unit 132 that associates position information of each point included in the point group of the three-dimensional data with position information of the feature detected by pattern matching. The three-dimensional data processing device 130 is further generated over time by comparing the three-dimensional data with pattern matching information generated in advance with the three-dimensional data with pattern matching information obtained by measurement. A difference extraction unit 135 for extracting the difference is provided. The difference extraction unit 135 suppresses an increase in the amount of calculation by limiting the collation range of the three-dimensional data to be compared depending on the presence or absence of features that match between the three-dimensional data to be compared. By adopting such a configuration, in the present embodiment, it is possible to reduce the amount of collation for three-dimensional data including a large amount of differences and make it difficult to collate three-dimensional data. The feasibility can be secured.

  In the present embodiment, by adopting an image recognition technique for generating a 3D map in MMS, it is possible to detect a feature with little change in position and shape between 3D maps. And, by limiting the number of repetitive processes in the collation process from information such as the presence / absence or number of features that change little over time, processing time can be reduced and process divergence can be prevented. As a result, it is possible to suppress deterioration in extraction accuracy of change points.

*** Other configurations ***
The three-dimensional data processing device 130 may be installed outside the measurement vehicle 100. The three-dimensional data processing apparatus 130 may be composed of a plurality of computers, and each computer may share processing.

  In the present embodiment, the functions of the data recording unit 131, the first data generation unit 132, the image recognition unit 133, the second data generation unit 134, and the difference extraction unit 135 are realized by software. The functions of the unit 131, the first data generation unit 132, the image recognition unit 133, the second data generation unit 134, and the difference extraction unit 135 may be realized by a combination of software and hardware. That is, some of the functions of the data recording unit 131, the first data generation unit 132, the image recognition unit 133, the second data generation unit 134, and the difference extraction unit 135 are realized by a dedicated electronic circuit, and the rest are realized by software. May be. About this modification, the difference with this Embodiment is mainly demonstrated.

  The dedicated electronic circuit is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an FPGA, or an ASIC. “GA” is an abbreviation for Gate Array. “FPGA” is an abbreviation for Field-Programmable Gate Array. “ASIC” is an abbreviation for Application Specific Integrated Circuit.

  The processor 141, the memory 142, and the dedicated electronic circuit are collectively referred to as a “processing circuit”. That is, the functions of the data recording unit 131, the first data generation unit 132, the image recognition unit 133, the second data generation unit 134, and the difference extraction unit 135 are realized by software, or by a combination of software and hardware. Regardless of this, the functions of the data recording unit 131, the first data generation unit 132, the image recognition unit 133, the second data generation unit 134, and the difference extraction unit 135 are realized by a processing circuit.

  The “device” of the three-dimensional data processing device 130 is read as “method”, and the “data” of the data recording unit 131, the first data generation unit 132, the image recognition unit 133, the second data generation unit 134, and the difference extraction unit 135 is replaced. It may be read as “process”. Alternatively, the “device” of the three-dimensional data processing device 130 is replaced with “program”, “program product”, or “computer-readable medium on which the program is recorded”, and the data recording unit 131, the first data generation unit 132, and image recognition The “part” of the part 133, the second data generation part 134, and the difference extraction part 135 may be read as “procedure” or “processing”.

  DESCRIPTION OF SYMBOLS 100 Measuring vehicle, 101 Three-dimensional data measuring machine, 102 Measuring cart, 110 Laser scanner, 120 Camera, 130 Three-dimensional data processing apparatus, 131 Data recording part, 132 1st data generation part, 133 Image recognition part, 134 2nd data Generation unit, 135 difference extraction unit, 141 processor, 142 memory, 200 data server.

Claims (7)

A tertiary that extracts a difference between reference data that is three-dimensional data obtained by three-dimensional measurement at a certain time point and comparison data that is three-dimensional data obtained by three-dimensional measurement at a time point different from the reference data. In the original data processing device,
A feature existing at the time when the three-dimensional measurement for obtaining the reference data is performed from an image photographed in parallel with the three-dimensional measurement for obtaining the comparison data is a three-dimensional for obtaining the comparison data. An image recognition unit that determines whether or not it exists even at the time of measurement;
The reference data and the comparison data are aligned by adjusting at least one of the reference data and the comparison data within a wide and narrow range depending on the determination result of the image recognition unit, and then the reference data A three-dimensional data processing apparatus comprising a difference extraction unit that extracts a difference between data and the comparison data.   The difference extraction unit, when it is determined that the feature is present even at the time when the feature is obtained by the three-dimensional measurement for obtaining the comparison data, the feature in the reference data and the comparison data The three-dimensional data processing apparatus according to claim 1, wherein the reference data and the comparison data are aligned by adjusting the positions so as to coincide with each other and then adjusting within the range.   The difference extraction unit sets the range according to the number of objects determined to exist even when the image recognition unit performs the three-dimensional measurement for obtaining the comparison data as the feature. The three-dimensional data processing device described in 1.   The difference extraction unit divides the number of objects determined to exist even when the image recognition unit performs the three-dimensional measurement for obtaining the comparison data as the features, and the more the number of the features, The three-dimensional data processing apparatus according to any one of claims 1 to 3, wherein the range is set narrow.   The three-dimensional data processing apparatus according to any one of claims 1 to 4, wherein the image recognition unit detects the feature from the image by pattern matching. A three-dimensional data processing device according to any one of claims 1 to 5,
A laser scanner that performs three-dimensional measurement to obtain the comparison data;
A three-dimensional data measuring machine comprising: a camera that captures the image in parallel with the three-dimensional measurement by the laser scanner. A tertiary that extracts a difference between reference data that is three-dimensional data obtained by three-dimensional measurement at a certain time point and comparison data that is three-dimensional data obtained by three-dimensional measurement at a time point different from the reference data. In the original data processing method,
The camera takes an image in parallel with the three-dimensional measurement for obtaining the comparison data,
The computer determines from the image whether a feature existing at the time when the three-dimensional measurement for obtaining the reference data is performed also exists at the time when the three-dimensional measurement for obtaining the comparison data is performed. Then, after adjusting the reference data and the comparison data by adjusting at least one of the reference data and the comparison data within different ranges depending on the determination result, the reference data and the comparison data A three-dimensional data processing method for extracting a difference from comparison data.

Images