JP2023152591A - Point group information generation system, control method of point group information generation system, and control program of point group information generation system - Google Patents

Abstract

To provide a point group information generation system or the like capable of generating point group information on a desired object simply and quickly.SOLUTION: A point group information generation system 500 comprises: a point group information generation device 1 that generates three-dimensional point group information on an object 600; and a XR generation device 100, which a user P can wear. The XR generation device includes: an imaging portion 111a that acquires at least imaging information corresponding to a line of sight of the user; and a display portion 113 that displays the imaging information. The point group information generation device generates three-dimensional point group information on the object identified by execution instruction information on the basis of the execution instruction information on the object displayed on the display portion.SELECTED DRAWING: Figure 1

Description

The present invention relates to a point cloud information generation system that acquires point cloud information of an object, a method for controlling the point cloud information generation system, and a control program for the point cloud information generation system.

Conventionally, when understanding the shape of a building at a civil engineering or construction site, for example, a 3D scanner irradiates a laser or the like to obtain the 3D shape of the object, such as a building, as point cloud data. It has been proposed (for example, Patent Document 1).

Japanese Patent Application Publication No. 2018-66571

However, in an actual site, there is a desire to acquire point cloud data of a single pillar, etc. of a part of a building, rather than the entire building.
In this case, there is a problem in that it takes time and effort to operate the scanner device, irradiate only one specific pillar, etc. with a laser beam, and obtain point cloud data, resulting in an increase in cost.

Therefore, the present invention provides a point cloud information generation system, a control method for the point cloud information generation system, and a control program for the point cloud information generation system, which can easily and quickly generate point cloud information about a desired object. The purpose is to

According to the present invention, the object is to provide a point cloud information generation system that includes a point cloud information generation device that generates three-dimensional point cloud information of an object, and an XR (cross reality) generation device that can be worn by a user. The XR generation device includes at least an imaging unit that acquires imaging information corresponding to the line of sight of the user, and a display unit, and the is achieved by a point cloud information generation system characterized in that the point cloud information generation device generates three-dimensional point cloud information about the object specified by the execution instruction information.

According to the configuration, the user simply wears the XR generation device and inputs execution instruction information about the object, and the point cloud information generation device can automatically generate point cloud information about the object. can.
Therefore, it is possible to easily and quickly generate point cloud information about a desired object.

Preferably, the three-dimensional point cloud information is generated for a portion of the object specified based on the execution instruction information of the point cloud information system.

According to the configuration, by specifying the part of the object based on the execution instruction information, it is possible to easily and quickly generate three-dimensional point cloud information only for the part of the object.

Preferably, the point cloud information generation device includes an optical axis deflection unit that emits distance measurement light for generating point cloud information and deflects an emission direction of the distance measurement light, and the optical axis deflection unit By deflecting the distance measuring light, the distance measuring light is irradiated only onto a portion of the object.

According to the above configuration, the optical axis deflection unit of the point cloud information generation device deflects and irradiates the distance measurement light toward the part of the target object specified based on the execution instruction information, thereby accurately detecting the target object. Three-dimensional point cloud information can be generated only for parts of objects.

Preferably, the XR generation device of the point cloud information system is characterized in that at least a high reflection part having either a unique shape or a unique arrangement is arranged.

According to the above configuration, by providing at least a high reflection portion having either a unique shape or a unique arrangement, not only the position of the XR generation device but also its inclination can be determined with high accuracy, and the XR generation device can be It is possible to improve the accuracy when detecting generated coordinates and the like.

Preferably, the configuration is such that blueprint information regarding the point cloud information system is displayed on the display section.

According to the above configuration, it is possible to specify the area around a structure or the like on a design drawing that does not yet exist, and to obtain three-dimensional point cloud information.
In addition, when inspecting components after they are actually installed at a construction site, etc. by comparing them with blueprint information, you can specify the components to be inspected on the blueprint information and use the three-dimensional By acquiring point cloud information, it becomes possible to intuitively specify the inspection target, etc.
Furthermore, if the position of a component on the blueprint is different from the planned installation location, it is possible to visually check the amount of difference and obtain 3D point cloud information before completing the installation. Work such as changing the installation position after completion can be done easily and quickly.

According to the present invention, the object is to provide a method for controlling a point cloud information generation system that includes a point cloud information generation device that generates three-dimensional point cloud information of an object, and an XR generation device that can be worn by a user. The imaging unit of the XR generation device acquires imaging information corresponding to the line of sight of the user, and based on the execution instruction information regarding the target object displayed on the display unit, the imaging unit of the XR generation device This is achieved by a method for controlling a point cloud information generation system, characterized in that three-dimensional point cloud information is generated for the object specified by the execution instruction information.

According to the present invention, the object is to provide a point cloud information generation system that includes a point cloud information generation device that generates three-dimensional point cloud information of an object, and an XR reality generation device that can be worn by a user. The imaging unit of the generation device acquires imaging information corresponding to the line of sight of the user, and the point cloud information generation device generates the execution instruction information based on the execution instruction information regarding the object displayed on the display unit. This is achieved by a control program for a point cloud information generation system, which is characterized in that it realizes the function of generating three-dimensional point cloud information about the object specified in the above.

The present invention provides a point cloud information generation system, a control method for the point cloud information generation system, and a control program for the point cloud information generation system, which can easily and quickly generate point cloud information about a desired object. It has the advantage of being possible.

1 is a schematic diagram showing the main configuration of a "point cloud information generation system 500" according to the present embodiment. 2 is a schematic diagram showing the main configuration of the laser scanner 3 of FIG. 1. FIG. 3 is a schematic diagram showing the main configuration of an optical axis deflection section 35 in FIG. 2. FIG. 2 is a schematic block diagram showing the main configuration of head mounted display 100 in FIG. 1. FIG. 4 is a schematic block diagram showing the main contents of the "HMD side various information storage section 120" in FIG. 3. FIG. 2 is a schematic block diagram showing the main configuration of a laser scanner 3 in FIG. 1. FIG. 6 is a schematic block diagram showing the main configuration of the "laser scanner side various information storage section 310" of FIG. 5. FIG. 5 is a schematic flowchart showing an example of the operation of the point cloud information generation system 500. 7 is another schematic flowchart showing an example of the operation of the point cloud information generation system 500. FIG. 7 is a schematic explanatory diagram showing an example of a scan pattern when the optical axis deflection unit 35 and the like perform local scanning. FIG. 7 is another schematic explanatory diagram showing an example of a scan pattern when the optical axis deflection unit 35 and the like perform local scanning. It is a schematic diagram showing a modification of point cloud information generation system 500 concerning an embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings and the like.
Since the embodiments described below are preferred specific examples of the present invention, various technically preferable limitations are attached thereto. Unless otherwise specified, the embodiments are not limited to these embodiments.

FIG. 1 is a schematic diagram showing the main configuration of a "point cloud information generation system 500" according to this embodiment.
As shown in FIG. 1, the point cloud information generation system 500 includes a "surveying device 1" which is an example of a point cloud information generation device, and a "head mounted display (HMD) 100" which is an example of an XR generation device. There is.
Here, XR (cross reality) is a concept that includes "VR (Virtual Reality)", "AR (Augmented Reality)" and "MR (MIXED reality)", This embodiment will be described below using MR as an example.
Therefore, the XR generation device of the present invention includes not only the MR of this embodiment but also VR, AR, and the like.
This surveying device 1 includes a laser scanner 3 and the like that generates three-dimensional point cloud information of a target object (for example, a pillar 600, etc.) by performing scanning or the like.
Further, the head-mounted display 100 is configured to be worn on the head by the user of the system 500 (for example, the worker P).

As shown in FIG. 1, the surveying device 1 includes a laser scanner 3, which can obtain three-dimensional point cloud data by emitting and receiving laser light, and can also perform specific measurements. It is also possible to measure distances to points.

That is, the configuration is such that three-dimensional point group data of an object such as the pillar 600 in FIG. 1 and distance data between the laser scanner 3 and the pillar 600 can be acquired.
Further, the laser scanner 3 is configured to be able to obtain distance data from the head-mounted display 100 as well.

(Main configuration of head mounted display 100, etc.)
The configurations of the head mounted display 100, the surveying device 1, etc. will be described in detail below.
The head mounted display 100 of this embodiment is, for example, a device for mixed reality.
Specifically, real world information is transmitted through the transmissive HDM side display 113 (an example of a display unit). It is configured to be able to generate coordinate information, and also to display information on a virtual world created by a computer or the like.
As shown in FIG. 1, the head-mounted display 100 includes a "main body section 110" mounted on the head of the worker P, and an "HMD side" disposed so as to face both eyes of the worker P. display 113'' and, for example, two HMD-side cameras 111a and 111b, which are imaging units arranged along the line of sight of the worker P. It also has an HMD-side illumination section 112 that is a light source.

The two HMD-side cameras 111a and 111b are installed facing the same direction as the line of sight of the worker P, and the image information captured by the two HMD-side cameras 111a and 111b is used for the line of sight and field of view of the worker P. It is the same as.
Since the HMD-side display 113 is of a "transmissive type", the worker P can see the object through the display 113.

Further, the HMD side illumination unit 112 is configured to emit light in the same direction as the line of sight of the worker P, but is arranged at a different angle from the HMD side cameras 111a and 111b.
The HMD side illumination unit 112 is also configured to emit various patterned lights, and is configured to be able to measure "depth" by measuring the distortion of the emitted light.

Therefore, when these two HMD-side cameras 111a and 111b image an object (for example, a pillar 600) placed in the same direction as the line of sight of the worker P, and the HMD-side irradiation unit 112 irradiates light onto this pillar 600, The structure is such that it is possible to generate "HMD side coordinate data (MR coordinate data)" which is three-dimensional coordinate data of the pillar 600 using a "structured illumination" method.
Further, the reference of this "HMD side coordinate data" is the coordinate position of the HMD side cameras 111a and 111b.

Note that in this embodiment, 3D data is generated using a "structured illumination" method, but the present invention is not limited to this, and can also be applied to cases where "3D data" is generated using only "two cameras". This also includes cases in which data is generated using the ToF (Time of Flight) method, which generates three-dimensional data from the time it takes for light emitted from a light source to return to the camera.
Furthermore, unlike this embodiment, the present invention places an optical camera and a depth sensor on a head-mounted display, emits infrared rays from the camera, reflects them off an object, and measures the time it takes for the reflected infrared rays to return. It is also possible to use a configuration in which the depth of an object is calculated using .

(Main configuration of surveying device 1, etc.)
The configuration of the surveying device 1 will be described in detail below.
O shown in FIG. 1 indicates the distance measuring optical axis in a state where the optical axis is not deflected, and the distance measuring optical axis at this time is taken as the reference optical axis.
The surveying device 1 mainly includes a tripod 2 as a support device, a laser scanner 3, an operating device 4, and a rotary table 5.
A rotary table 5 is attached to the upper end of the tripod 2, and a laser scanner 3 is attached to the rotary table 5 so as to be rotatable horizontally and vertically.

Further, the rotary table 5 has a function of detecting the lateral rotation angle (horizontal rotation angle) of the laser scanner 3.
The rotating table 5 is provided with a lever 7 that extends in the horizontal direction. By operating the lever 7, the laser scanner 3 can be rotated vertically (vertically) or laterally (horizontally), and can also be fixed in a desired position.
Further, the laser scanner 3 itself has a mechanism that can rotate horizontally, and is also equipped with a mechanism that can automatically rotate to track an object.

The laser scanner 3 has a built-in distance measuring section and an attitude detecting section, which will be described later, and the distance measuring section emits distance measuring light onto the object to be measured or the measurement range, and measures the distance by receiving the reflected distance measuring light. . Further, the attitude detection section can detect the attitude of the laser scanner 3 with respect to the vertical (or horizontal) with high accuracy.

The operating device 4 has a communication function to communicate with the laser scanner 3 via a necessary means such as wired or wireless.
Images, measurement conditions, measurement results, etc. are transmitted from the laser scanner 3 to the operating device 4, and the images, measurement conditions, measurement results, etc. are stored in the operating device 4.

FIG. 2 is a schematic diagram showing the main configuration of the laser scanner 3 of FIG. 1. As shown in FIG.
As shown in FIG. 2, the laser scanner 3 includes a distance measurement light emitting section 11, a light receiving section 12, a distance measurement calculation section 13, an imaging section 14 (camera), an emission direction detection section 15, a motor driver 16, and an attitude detection section 17. , a first communication section 18, an arithmetic control section 19, a first storage section 20, an imaging control section 21, and an image processing section 22, which are housed in a housing 9 and integrated.

Note that the distance measuring light emitting section 11, the light receiving section 12, the distance measuring calculation section 13, etc. constitute a distance measuring section.
The distance measuring light emitting unit 11 has an emitting optical axis 26, and a light emitting element 27, such as a laser diode (LD), is provided on the emitting optical axis 26.
Further, a projection lens 28 is provided on the emission optical axis 26. Furthermore, the first reflecting mirror 29 as a deflecting optical member provided on the emitting optical axis 26 and the second reflecting mirror 32 as a deflecting optical member provided on the receiving optical axis 31 cause the emitting optical axis 26 to be , are deflected so as to coincide with the light receiving optical axis 31. The first reflecting mirror 29 and the second reflecting mirror 32 constitute an exit optical axis deflecting section.

The light emitting element 27 emits a pulsed laser beam, and the ranging light emitting section 11 emits the pulsed laser beam emitted from the light emitting element 27 as ranging light 23.
The reflected distance measuring light 24 from the object to be measured (ie, the measurement point) is incident on the light receiving section 12 .
The light receiving section 12 has a light receiving optical axis 31, and the light receiving optical axis 31 coincides with the emitting optical axis 26 that is deflected by the first reflecting mirror 29 and the second reflecting mirror 32, as described above.
Note that the state in which the emission optical axis 26 and the light reception optical axis 31 match is defined as the distance measuring optical axis 40 (see FIG. 1).

An optical axis deflector 35 is disposed on the deflected emission optical axis 26, that is, on the light receiving optical axis 31. A straight optical axis passing through the center of the optical axis deflection section 35 is a reference optical axis O. The reference optical axis O coincides with the emitting optical axis 26 or the receiving optical axis 31 when the optical axis is not deflected by the optical axis deflecting section 35.

An imaging lens 34 is disposed on the light-receiving optical axis 31 that has passed through the optical axis deflection unit 35 and entered, and a light-receiving element 33, for example, a photodiode (PD), is provided.
The imaging lens 34 forms an image of the reflected distance measuring light 24 on the light receiving element 33 . The light receiving element 33 receives the reflected distance measuring light 24 and generates a light receiving signal.
The light reception signal is input to the distance measurement calculation section 13. The distance measurement calculation unit 13 measures the distance to the measurement point based on the light reception signal.

FIG. 3 is a schematic diagram showing the main configuration of the optical axis deflection section 35 shown in FIG. 2. As shown in FIG.
The optical axis deflection section 35 will be explained with reference to FIG. 3.
The optical axis deflection section 35 is composed of a pair of optical prisms 36a and 36b. The optical prisms 36a and 36b each have a disk shape, are arranged perpendicularly to the light receiving optical axis 31, overlapped, and arranged in parallel.
It is preferable to use Fresnel prisms as each of the optical prisms 36a and 36b in order to downsize the apparatus.

The center part of the optical axis deflection part 35 serves as a distance measurement light deflection part 35a, which is a first optical axis deflection part through which the distance measurement light 23 is transmitted and emitted, and the part other than the center part is the reflection light 23. The distance measuring light deflection section 35b is a second optical axis deflection section through which the distance measuring light 24 is transmitted and incident.

The Fresnel prisms used as the optical prisms 36a and 36b are composed of parallel prism elements 37a and 37b and a large number of prism elements 38a and 38b, and have a disk shape. The optical prisms 36a, 36b and each prism element 37a, 37b and prism element 38a, 38b have the same optical characteristics.

The prism elements 37a and 37b constitute the distance measurement light deflection section 35a, and the prism elements 38a and 38b constitute the reflected distance measurement light deflection section 35b.

The Fresnel prism may be made from optical glass, or may be molded from optical plastic material. Cheap Fresnel prisms can be manufactured by molding optical plastic materials.

The optical prisms 36a and 36b are arranged to be independently rotatable about the light receiving optical axis 31, respectively. The optical prisms 36a and 36b have their rotational direction, amount of rotation, and rotational speed independently controlled, thereby deflecting the emitting optical axis 26 of the emitted ranging light 23 in an arbitrary direction and receiving the light. The receiving optical axis 31 of the reflected distance measuring light 24 is deflected parallel to the emitting optical axis 26.

The external shapes of the optical prisms 36a and 36b are circular with the light-receiving optical axis 31 as the center. Considering the spread of the reflected ranging light 24, the optical prisms 36a and 36b are arranged so that a sufficient amount of light can be obtained. The diameter of 36b is set.

A ring gear 39a is fitted on the outer periphery of the optical prism 36a, and a ring gear 39b is fitted on the outer periphery of the optical prism 36b.

A drive gear 41a meshes with the ring gear 39a, and the drive gear 41a is fixed to the output shaft of a motor 42a. Similarly, a drive gear 41b meshes with the ring gear 39b, and the drive gear 41b is fixed to the output shaft of a motor 42b. The motors 42a, 42b are electrically connected to the motor driver 16.

The motors 42a and 42b are those whose rotation angle can be detected, or those which rotate in accordance with the drive input value, such as pulse motors. Alternatively, the amount of rotation of the motor may be detected using a rotation angle detector that detects the amount of rotation (rotation angle) of the motor, such as an encoder. The amount of rotation of the motors 42a, 42b is detected, and the motor driver 16 controls the motors 42a, 42b individually.
Incidentally, encoders may be directly attached to the ring gears 39a, 39b, respectively, and the rotation angles of the ring gears 39a, 39b may be directly detected by the encoders.

The driving gears 41a, 41b and the motors 42a, 42b are provided at positions that do not interfere with the distance measuring light emitting section 11, for example, below the ring gears 39a, 39b.

The light projection lens 28, the first reflecting mirror 29, the second reflecting mirror 32, the distance measuring light deflection section 35a, etc. constitute a light projection optical system, and the reflected distance measuring light deflection section 35b, the image forming The lens 34 and the like constitute a light receiving optical system.

The distance measurement calculation unit 13 controls the light emitting element 27 to emit a pulsed laser beam as the distance measurement light 23. The emitting optical axis 26 of the distance measuring light 23 is deflected by the prism elements 37a and 37b (the distance measuring light deflecting section 35a) so that the distance measuring light 23 is directed toward the measurement point.

The reflected distance measuring light 24 reflected from the object to be measured enters through the prism elements 38a and 38b (the reflected distance measuring light deflection section 35b) and the imaging lens 34, and is received by the light receiving element 33. Ru. The light-receiving element 33 sends a light-receiving signal to the distance-measuring calculation unit 13, and the distance-measuring calculation unit 13 determines the measurement point (where the distance-measuring light is irradiated) for each pulsed light based on the light-receiving signal from the light-receiving element 33. The distance measurement data is stored in the first storage unit 20.
By scanning the distance measurement light 23 and performing distance measurement for each pulsed light, distance measurement data for each measurement point can be obtained.

The injection direction detection unit 15 detects the rotation angle of the motors 42a, 42b by counting drive pulses input to the motors 42a, 42b.
Alternatively, the rotation angles of the motors 42a, 42b are detected based on signals from encoders. Further, the emission direction detection section 15 calculates the rotational positions of the optical prisms 36a and 36b based on the rotation angles of the motors 42a and 42b.
Furthermore, the emission direction detection unit 15 calculates the declination and emission direction of the ranging light based on the refractive index and rotational position of the optical prisms 36a and 36b, and the calculation results are input to the calculation control unit 19. .

The calculation control unit 19 calculates the horizontal angle and vertical angle of the measurement point from the declination and emission direction of the distance measurement light, and associates the horizontal angle and vertical angle with the distance measurement data for each measurement point. Three-dimensional data of a point can be obtained.

The imaging unit 14 is a camera that has an imaging optical axis 43 parallel to the reference optical axis O of the laser scanner 3 and has an angle of view of, for example, 50°, and acquires image data including the scan range of the laser scanner 3. do.
The relationship between the imaging optical axis 43, the exit optical axis 26, and the reference optical axis O is known. Further, the imaging unit 14 can acquire moving images or continuous images.

The imaging control section 21 controls imaging by the imaging section 14 . When the imaging unit 14 captures a moving image or continuous images, the imaging control unit 21 synchronizes the timing of acquiring frame images constituting the moving image or continuous images with the timing of scanning with the laser scanner 3. ing. The arithmetic control unit 19 also associates the image with point cloud data, which is scan data.

Moreover, as a mode of measurement performed by the laser scanner 3, by fixing the optical axis deflection unit 35 at each required deflection angle and performing distance measurement, it is possible to perform distance measurement at a specific measurement point.
Further, the direction angle (horizontal angle, vertical angle) during distance measurement can be obtained based on the detection result of the emission direction detection section 15.

Further, the inclination and inclination direction of the laser scanner 3 with respect to the horizontal can be detected by the attitude detection section 17, and the measurement result can be corrected to horizontal reference data based on the detection result of the attitude detection section 17.
That is, the laser scanner 3 can be used like a total station.

Furthermore, the emission direction angle of the distance measurement light during distance measurement can be detected by the rotation angle of the motor, and by associating the emission direction angle during distance measurement with the distance measurement data, three-dimensional distance measurement data can be obtained. I can do it.
Therefore, the laser scanner 3 can function as a laser scanner that acquires point cloud data having three-dimensional position data.

The surveying apparatus 1 and the head-mounted display 100 in FIG. 1 include a computer, and the computer includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc. (not shown). , bus, etc.

(Main configuration of head mounted display 100, etc.)
FIG. 4 is a schematic block diagram showing the main configuration of the head mounted display 100 of FIG. 1.
As shown in FIG. 4, the head mounted display 100 has an "HMD side control unit 101", and the control unit 101 has an "HMD side communication device 102" which communicates with the surveying device 1 etc. of FIG. It controls the cameras 111a and 111b, the HMD-side illumination unit 112, the HMD-side display 113, and the like.

The control unit 101 also controls the HMD-side various information storage unit 120 shown in FIG.
FIG. 5 is a schematic block diagram showing the main contents of the "HMD side various information storage section 120" in FIG. 4. These configurations will be described later.

(Main configuration of laser scanner 3, etc.)
FIG. 6 is a schematic block diagram showing the main configuration of the laser scanner 3 of FIG. 1.
As shown in FIG. 6, the laser scanner 3 has a "laser scanner side control section 301", and the control section 301 controls the "laser scanner main body 302" which is the main component of the laser scanner 3 in FIG. At the same time, it also controls the "laser scanner side various information storage section 310".
FIG. 7 is a schematic block diagram showing the main configuration of the "laser scanner side various information storage section 310" in FIG. 6. As shown in FIG. These configurations will be described later.

(Example of operation of point cloud information generation system 500)
8 and 9 are schematic flowcharts showing an example of the operation of the point cloud information generation system 500.
In this embodiment, in order for the worker P to obtain three-dimensional point cloud data between point A and point B of one specific pillar 600 at a construction site, the worker P uses the head-mounted display 100 is attached, and the surveying device 1 performs a scan, thereby acquiring 3D point cloud data of the area between point A and point B, and displaying the acquired 3D point cloud data on the HMD side display 113. As an example, a main operation example of the present system 500 will be specifically described below.

First, in step (hereinafter referred to as "ST") 1 in FIG. 8, the worker P installs the surveying device 1 at a predetermined position and self-installs the head mounted display (HMD) 100, as shown in FIG. be attached to the head of

Next, proceed to ST2. In ST2, the worker P turns on the HMD-side cameras 111a and 111b of the head-mounted display 100, and also turns on the HMD-side illumination section 112. Since the HMD-side display 113 is of a "transmissive type", the worker P can view the outside through the display 113.

In addition, in ST3, the "HMD coordinate information generation unit (program) 121" shown in FIG. Three-dimensional coordinate information of the pillar 600 that the person P is viewing on the HMD-side display 113 is generated and stored as HMD coordinate information (MR coordinate data) in the "HMD coordinate information storage unit 122" in FIG. 5.

Next, proceed to ST4. In ST4, as the worker P moves his right hand and points his index finger toward the pillar 600 as shown in FIG. In ST5, the "HMD coordinate information generation unit (program) 121" shown in FIG. 5 operates, and the HMD side illumination unit 112 and the like automatically operate to generate right-hand HMD coordinate information using the structured illumination method. It is stored in the "HMD coordinate information storage section 122" in FIG. 5.

Next, the process advances to ST6. In ST6, the worker P points the index finger of the right hand toward the "starting point A" of the pillar 600 shown in FIG. 1, which is the object to be scanned, and turns on the "trigger switch" (not shown).
This action of the worker P is an example of execution instruction information.
Then, proceeding to ST7, the "HMD coordinate data generation unit (program) 125" in FIG. information" is generated and stored in the "scan start point coordinate information storage section 126" in FIG.

Next, proceed to ST8. In ST8, the worker P points the index finger of the right hand toward the "end point B" of the pillar 600, which is the object to be scanned, and turns the "trigger switch" ON.
This action of the worker P is an example of execution instruction information.
Then, proceed to ST9. In ST9, the "HMD coordinate data generation unit 125" in FIG. 5 generates direction information from the index finger inclination information, etc., specifies, for example, point B of the pillar 600, and generates "scan end point coordinate information", It is stored in the "scan end point coordinate information storage section 127" in FIG.

This completes the process of specifying the scanning target range (point A to point B in FIG. 1) by the head mounted display 100.
This range from point A to point B is an example of a portion of the object.

Next, the operation of the laser scanner 3 of the surveying apparatus 1 in FIG. 1 will be explained.
The surveying device 1 always measures the distance to the head-mounted display 100 and stores "HMD distance information" in the "HMD distance information storage section 311" in FIG. 7 .
Therefore, the laser scanner 3 constantly acquires distance information from the head-mounted display 100.

The laser scanner 3 also constantly communicates with the head mounted display 100, and the head mounted display 100 in FIG. The information in the storage unit 127 is acquired.
Then, "scanner-side HMD coordinate information,""scanner-side scan start point coordinate information," and "scanner-side scan end point coordinate information" are generated, respectively.
Then, they are stored in the "scanner-side HMD coordinate information storage section 312,""scanner-side scan start point coordinate information storage section 313," and "scanner-side scan end point coordinate information storage section 314," respectively, in FIG.

In this way, the coordinate information stored in the head mounted display 100 is stored in the laser scanner 3 side.

Next, the "laser scanner side corrected coordinate information generation unit (program) 315" shown in FIG. , "Corrected scanner-side HMD coordinate information" and "Corrected scanner-side start point coordinate information" corrected with the coordinate position of the laser scanner 3 from each coordinate information of "Scanner-side scan start point coordinate information" and "Scanner-side scan end point coordinate information" coordinate information" and "corrected scanner-side scan end point information".

Next, the respective pieces of information generated as described above are stored in the "corrected scanner side HMD coordinate information storage section 316," the "corrected scanner side start point coordinate information storage section 317," and the "corrected scanner side scan end point information storage section 318." Remember.

In this way, the laser scanner 3 is synchronized with the data of the head-mounted display 100, and always generates and stores coordinate information based on the laser scanner 3, which incorporates the coordinate information of the head-mounted display 100.

Next, the "scan target identification processing unit (program) 319" of the laser scanner 3 shown in FIG. With reference to , the coordinate information of the range from point A to point B of the pillar 600 to be scanned is specified using the laser scanner 3 standard, and this range is scanned (local scan) to obtain three-dimensional point cloud data. .

At this time, the optical axis deflection unit 35 shown in FIGS. 2 and 3 operates to locally scan the above-mentioned range, and an example of the local scan pattern will be described below.
10 and 11 show an example of a scan pattern when the optical axis deflection unit 35 and the like perform local scanning.
The scan pattern 53 shown in FIG. 10 is a petal-shaped scan pattern, and the scan patterns 54a and 54b shown in FIGS. 11A and 11B are linear scan patterns.

In this way, according to the present embodiment, the configuration is such that only a local area specified as a scan target can be partially scanned.

In this way, the three-dimensional point cloud data acquired by the laser scanner 3 is displayed on the HMD-side display 113 of the head-mounted display 100.

As described above, according to the present embodiment, when the worker P wears the head-mounted display 100 and simply specifies the scanning target range with his or her finger, the laser scanner 3 automatically scans only the target range with precision. Scanning can be performed quickly and easily, and three-dimensional point cloud data of a target range can be obtained.

Further, in this embodiment, as described above, the coordinate information including the head mounted display 100 is held with the laser scanner 3 as a reference.
Therefore, when the head-mounted display 100 rotates as a result of the worker P in FIG. The horizontal mechanism rotates in the same way.

Therefore, in this embodiment, the laser scanner 3 can also be configured to rotate in synchronization with the line of sight of the worker P, and the laser scanner 3 can also be configured to automatically follow the object to be scanned. .

Furthermore, in this embodiment, even when the head-mounted display 100 does not rotate, the laser scanner 3 can be configured to face the same area as the "angle of view" imaged by the HMD-side cameras 111a and 111b.

Further, in this embodiment, the head mounted display 100 generates coordinate information, and then the laser scanner 3 acquires the coordinate information of the head mounted display 100, and the laser scanner 3, the head mounted display 100, and the pillar 600 Although the entire coordinate information included is generated based on the laser scanner 3, the present invention is not limited to this, and may be configured as follows.

That is, the laser scanner 3 can be used without acquiring the coordinate information of the head-mounted display 10 using the camera (imaging section 14) of the laser scanner 3, the HMD-side camera 111a of the head-mounted display 100, the HMD-side illumination section 112, etc. may be configured to generate the entire coordinate information.

In this case, the worker P wearing the head-mounted display 100 then specifies the scanning part (point A to point B) of the pillar 600 with his index finger, and these steps are the same as in the embodiment described above. be.

(First modification of the embodiment)
FIG. 12 is a schematic diagram showing a modification of the point cloud information generation system 500 according to the embodiment.
Since many of the configurations of this modification are common to those of the above-described embodiment, their description will be omitted, and the following description will focus on the differences.
As shown in FIG. 12, in this modification, a highly reflective object 201 having a unique shape, which is an example of a highly reflective portion having at least one of a unique shape or a unique arrangement, is placed above the head mounted display 200. It is arranged so that it stands up.

In the case of this modification, when the laser scanner 3 specifies the position of the head-mounted display 200, it is possible to target a part of the highly reflective object 201 of the head-mounted display 200, which enables accurate measurement. Become.

Furthermore, by providing the highly reflective object 201 in this way, not only the position of the head mounted display 200 but also its inclination can be determined with high accuracy, and the coordinates (or line of sight) specified by the head mounted display 200 can be detected. Accuracy will also be improved.

(Second modification of embodiment)
Many of the configurations of the second modified example of the present embodiment are the same as those of the above-described embodiments, so their explanation will be omitted, and the following explanation will focus on the differences.
In the embodiment described above, the field of view of the worker P is displayed on the display 113 of the head-mounted display 100 of the worker P, but in this modification, the corresponding "design drawing" information is displayed in three-dimensional form using a computer or the like. The information is processed into information (for example, CG (Computer Graphics)), combined, and displayed.

Therefore, in reality, at a construction site, the vicinity of a structure on a blueprint that does not yet exist can be specified as a scanning range, and the laser scanner 3 can scan the area.
In addition, when inspecting components by comparing them with design data after actually installing them at a construction site, you can specify the components to be inspected on the design drawings and scan them installed in the actual space, making it easy to intuitively inspect the components. You can specify the inspection target automatically.

Furthermore, if the position of a component on the blueprint differs from the planned installation location and is ``shifted'', you can visually check the amount of difference and scan before completing the installation to change the installation position after completion. Work can be done easily and quickly.

(Third modification of this embodiment)
Many of the configurations of the third modified example of the present embodiment are the same as those of the above-described embodiments, so their explanation will be omitted, and the following explanation will focus on the differences.

In the embodiment described above, the worker P specifies the object such as the pillar 600 displayed on the head-mounted display 100 as the object to be scanned, and the worker P's hand does not need to actually touch the pillar 600 or the like. It becomes.
However, in this modification, the worker P can actually touch the pillar 600 to specify the object to be scanned or its range.
Therefore, the worker P can specify the scanning range more accurately.

In the present embodiment described above, the case where it is realized as an apparatus has been described as an example, but the present invention is not limited to this, and the present invention is not limited to this. (Trademark) disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), magneto-optical disk (MO), semiconductor memory, etc., and may be stored and distributed in storage media.

Further, the storage medium may be any storage medium that can store a program and is readable by a computer. The storage format of the storage medium is not particularly limited.

In addition, the OS (operating system), MW (middleware) such as database management software, network software, etc. that are running on the computer based on the instructions of the program installed on the computer from the storage medium realize this embodiment. A part of each process may be executed.

Furthermore, the storage medium in the present invention is not limited to a medium independent of a computer, but also includes a storage medium in which a program transmitted via a LAN, the Internet, etc. is downloaded and stored or temporarily stored.

Further, the computer in the present invention may execute each process in the present embodiment based on a program stored in a storage medium, and may be a device including one personal computer (PC) or the like, or may be a device including a plurality of devices. may be a system connected to a network.

Furthermore, the computer in the present invention is not limited to a personal computer, but also includes arithmetic processing units, microcomputers, etc. included in information processing equipment, and is a general term for devices and devices that can realize the functions of the present invention through programs. ing.

The embodiments of the present invention have been described above. However, the present invention is not limited to the above embodiments, and various changes can be made without departing from the scope of the claims.

DESCRIPTION OF SYMBOLS 1... Surveying device, 2... Tripod, 3... Laser scanner, 4... Operating device, 5... Turntable, 7... Lever, 9... Housing, 11... - Distance measurement light emitting unit, 12... Light receiving unit, 13... Distance measurement calculation unit, 14... Imaging control unit, 15... Emission direction detection unit, 16... Motor driver, 17... - Attitude detection unit, 18... First communication unit, 19... Arithmetic control unit, 20... First storage unit, 21... Imaging control unit, 22... Image processing unit, 23... - Distance measuring light, 24... Reflected ranging light, 26... Outgoing optical axis, 27... Light emitting element, 28... Light projecting lens, 29... First reflecting mirror, 31... Light receiving optical axis, 32... Second reflecting mirror, 33... Light receiving element, 34... Imaging lens, 35... Optical axis deflection unit, 40... Distance measuring optical axis, 43... Imaging optical axis, 100, 200... Head mounted display, 101... HMD side control unit, 102... HMD side communication device, 110... Main unit, 111a, 111b... HMD side camera, 112... HMD side illumination section, 113... HMD side display, 120... HMD side various information storage section, 121... HMD coordinate information generation section, 122... HMD coordinate information storage section, 125. ... HMD coordinate data generation unit, 126... Scan start point coordinate information storage unit, 127... Scan end point coordinate information storage unit, 201... Highly reflective object, 301... Laser scanner side control unit, 302...Laser scanner body, 310...Laser scanner side various information storage unit, 311...HMD distance information storage unit, 312...Scanner side HMD coordinate information storage unit, 313...Scanner side scan start Point coordinate information storage section, 314... Scanner side scan end point coordinate information, 315... Laser scanner side corrected coordinate information generation section, 316... Correction scanner side HMD coordinate information storage section, 317... Correction scanner Side start point coordinate information storage unit, 318... Correction scanner side scan end point information storage unit, 319... Scan target identification processing unit, 600... Pillar, 21... Imaging control unit, O... Reference optical axis, P...operator

Claims (10)

A point cloud information generation system comprising a point cloud information generation device that generates three-dimensional point cloud information of a target, and an XR (cross reality) generation device that can be worn by a user,
The XR generation device includes at least an imaging unit that acquires imaging information corresponding to the user's line of sight, and a display unit,
The point cloud information generation device generates three-dimensional point cloud information about the object specified by the execution instruction information based on the execution instruction information about the object displayed on the display unit. point cloud information generation system. The point cloud information generation system according to claim 1, wherein the three-dimensional point cloud information is generated for a portion of the object specified based on the execution instruction information. The point cloud information generation device has an optical axis deflection unit that emits distance measurement light for generating point cloud information and deflects the emission direction of the distance measurement light, and the optical axis deflection unit emits distance measurement light for generating point cloud information. 3. The point cloud information generation system according to claim 2, wherein the distance measuring light is irradiated only on a portion of the object by deflecting the distance light. The point cloud according to any one of claims 1 to 3, wherein the XR generation device is provided with a high reflection part having at least one of a unique shape and a unique arrangement. Information generation system. The point cloud information generation system according to claim 1, wherein the point cloud information generation system is configured to display blueprint information on the display unit. 3. The point cloud information generation system according to claim 2, wherein the point cloud information generation system is configured to display blueprint information on the display unit. 4. The point cloud information generation system according to claim 3, wherein the point cloud information generation system is configured to display blueprint information on the display unit. 5. The point cloud information generation system according to claim 4, wherein the point cloud information generation system is configured to display blueprint information on the display unit. A method for controlling a point cloud information generation system comprising a point cloud information generation device that generates three-dimensional point cloud information of a target, and an XR generation device that can be worn by a user, the method comprising:
The imaging unit of the XR generation device acquires imaging information corresponding to the user's line of sight,
The point cloud information generation device generates three-dimensional point cloud information about the object specified by the execution instruction information based on the execution instruction information about the object displayed on the display unit. A control method for a point cloud information generation system. A point cloud information generation system comprising a point cloud information generation device that generates three-dimensional point cloud information of a target, and an XR generation device that can be worn by a user,
a function for the imaging unit of the XR generation device to acquire imaging information corresponding to the user's line of sight;
Based on the execution instruction information about the object displayed on the display unit, the point cloud information generation device realizes a function of generating three-dimensional point cloud information about the object specified by the execution instruction information. A control program for a point cloud information generation system.

Images