JP2018097588A - Three-dimensional space specifying device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately specify a three-dimensional space in which an object to be detected is present.SOLUTION: The device comprises: an interface section 22 for displaying each of a plurality of images which an imaging range is overlapping; an input unit 10 for accepting, at each of the plurality of images, the designation of the range in which an object to be detected is present; and an analysis section 24 for specifying, based on the designation of the range in each of the plurality of images, the three-dimensional space in which the object to be detected is present.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a three-dimensional space specifying device, method, and program, and more particularly, to a three-dimensional space specifying device, method, and program for specifying a three-dimensional space in which a detection target exists.

  Conventionally, by extracting the silhouette of a detection object (a two-dimensional region where the detection object exists in a camera image) reflected in a plurality of camera images, the position of the object in a three-dimensional space can be determined as a voxel. There is known a method for obtaining in the space expressed by (Non-Patent Document 1).

Shinya Taki, Daisaku Arita, Rinichiro Taniguchi, “Generation of Arbitrary Viewpoint Color Image Based on 3D Model Using Visual Volume Crossing Method”, Fire Country Information Symposium, pp.68-75, 2003.

  However, the technique described in Non-Patent Document 1 is premised on shooting with a fixed camera, and background difference is performed to extract a silhouette. You can improve the robustness by learning in advance about changes in the background image such as sunshine changes, but in the case of a camera image mounted on a moving body (car), the background image changes from moment to moment. Therefore, it is difficult to learn, and the accuracy of extracting the foreground image area from the background decreases. That is, there is a technical problem that it is difficult to apply the present technology because there is a possibility that it is determined that there is a detection target in an erroneous region that originally has no detection target.

  The present invention has been made in consideration of the above problems, and can accurately identify a three-dimensional space in which a detection target exists using an image or a three-dimensional point cloud measured from a moving object. An object of the present invention is to provide a dimension space identification device, method, and program.

  In order to achieve the above object, the three-dimensional space identification device of the present invention displays each of a plurality of images with overlapping imaging ranges, and specifies a region where a detection target exists in each of the plurality of images. The interface unit is configured to include an accepting interface unit, and an analysis unit that identifies a three-dimensional space in which the detection target exists based on designation of a region in each of the plurality of images.

  In the three-dimensional space specifying method of the present invention, the interface unit displays each of a plurality of images having overlapping imaging ranges, receives designation of a region where the detection target exists in each of the plurality of images, and the analysis unit However, based on the designation of the area in each of the plurality of images, the three-dimensional space in which the detection object exists is specified.

  Moreover, the program of this invention is for making a computer function as each part of said three-dimensional space identification apparatus.

  According to the present invention, each of a plurality of images having overlapping imaging ranges is displayed, and in each of the plurality of images, designation of a region where a detection target exists is received, and a three-dimensional space where the detection target exists is specified. By doing so, the effect that the three-dimensional space in which a detection target object exists can be pinpointed accurately is acquired.

It is a block diagram which shows an example of schematic structure of the equipment detection apparatus of 1st Embodiment and 2nd Embodiment. It is a flowchart which shows an example of the flow of the preliminary process performed by the equipment detection apparatus of 1st Embodiment and 2nd Embodiment. It is a flowchart which shows an example of the flow of the interactive detection process performed by the equipment detection apparatus of 1st Embodiment and 2nd Embodiment. It is a figure which shows an example of the GUI screen which presents a camera image. It is a flowchart which shows an example of the flow of the reanalysis target point group determination process performed by the equipment detection apparatus of 1st Embodiment. It is a figure which shows a mode that the area | region was designated by operation of the user on the camera image. It is a figure for demonstrating the method to detect the point cloud in the designated area | region. It is a figure for demonstrating the method of extracting the image containing an image designation | designated 3D point. It is a figure which shows an example of the GUI screen which presents a candidate image. It is a figure which shows a mode that the area | region was designated by the user's operation on the selected candidate image. It is a figure for demonstrating the method to specify three-dimensional space by the designated area | region. It is a figure for demonstrating the method to determine the three-dimensional point group of the three-dimensional space specified by the designated area | region. It is a flowchart which shows an example of the flow of the reanalysis target point group determination process performed by the equipment detection apparatus of 2nd Embodiment. It is a figure which shows an example of a virtual image. It is a figure which shows an example of the GUI screen which shows a some candidate image. It is a figure which shows a mode that the area | region was designated by the user's operation on the selected candidate image. It is a figure for demonstrating the method to specify three-dimensional space by the designated area | region. It is a figure which shows an example of an ortho image. It is a figure which shows an example of the ortho image which displayed the imaging | photography range. It is a figure which shows a mode that the area | region was designated by the user's operation on an ortho image. It is a figure which shows an example of the virtual image of a viewpoint with little influence of occlusion. It is a figure for demonstrating the method of calculating | requiring the virtual image of a viewpoint with little influence of an occlusion. It is a figure for demonstrating the method of calculating | requiring the virtual image of a viewpoint with little influence of an occlusion. It is a figure which shows a mode that designation | designated on the image was performed by the prior art.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that this embodiment does not limit the present invention.

[Summary of Invention]
First, an outline of the embodiment of the present invention will be described.

With the aging of infrastructure equipment, development of equipment state recognition technology using 3D point cloud is expected to reduce the management cost of huge equipment. Here, the equipment state recognition technique refers to, for example, a technique for automatically measuring the inclination of a power pole, the amount of deflection, and the minimum ground height of an electric wire or communication line. For example, there is a technique as disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2014-15336), and hereinafter referred to as an infrastructure equipment detection technique.
However, even if these infrastructure equipment detection technologies are used, it is difficult to estimate with 100% accuracy depending on measurement conditions.

  This is because, for example, if the distance between the measurement vehicle (MMS) equipped with a laser scanner and the subject is large, the density of the measured three-dimensional points is small and the surface area to be measured is also small. Because it becomes difficult to do.

  Therefore, for the results estimated by the infrastructure equipment detection technology, if the information is correct while the equipment manager confirms whether the information is correct or incorrect as post-processing using the viewer software (GUI interface) Work to register in the equipment management database is required.

  At this time, the information of the equipment that has been detected in error may be ignored (or deleted by simple operation such as clicking by visual confirmation) without registering, but for equipment that has been detected undetected, it is again three-dimensional. Analysis must be performed using point clouds.

  In the first place, it is difficult to improve the accuracy even if the automatic detection technology is applied again to the scene (3D point cloud data) where the result of the infrastructure equipment detection technology is wrong due to the poor quality of the measured point cloud. It is necessary to provide useful information for improving accuracy. Specifically, it is important to specify what the detection target object is included in the scene and to indicate in which position of the input scene (three-dimensional point group) the detection target object exists.

  From the above, in order to improve the detection accuracy, it is necessary to specify the detection target using a GUI interface or the like, and to determine the reanalysis range of the 3D point cloud and execute the detection process again. is there.

  At this time, the type of the detection object can be easily specified by the GUI, but it is difficult to specify the three-dimensional space. As shown in FIG. 24, “area specification” on one camera image measured by MMS (Mobile Mapping System) leaves uncertainty in the depth direction, so the reanalysis range in the three-dimensional space is determined. Not. Here, “designation of the area” means that the processing range is specified by drawing a line segment such as a figure such as a circle, square, rectangle, or a curve.

  Therefore, in the embodiment of the present invention, it is possible to specify a reanalysis range as a three-dimensional space by specifying an area on a two-dimensional image by using two or more camera images. In addition, an image to be designated by the user is automatically selected or generated on the system side and displayed, thereby enabling an operation for efficiently updating the facility management DB.

[Outline of Embodiment]
The equipment detection apparatus of this embodiment aims to detect equipment around a road by inputting a 3D point group that is a set of 3D points representing a position on the surface of an object and a plurality of images. Is. Specifically, the facility detection apparatus according to the present embodiment detects utility poles, overhead lines, manholes, gratings, and the like as facilities existing around the road.

  The three-dimensional information in the present embodiment may be latitude, longitude, and sea level (height) information, or may be a three-dimensional Euclidean coordinate system or a polar coordinate system with a specific position set by the user as the origin. In the following example, a three-dimensional Euclidean coordinate system at the origin set by the user (each direction is assumed to be an X, Y, Z coordinate) is assumed. The unit of each coordinate is expressed in meters (m), centimeters (cm), and millimeters (mm), but other units may be used.

  A three-dimensional point is a point where each point is given the time when the point group was photographed, laser reflection intensity, color information such as red, blue, and green, etc., on the above three-dimensional coordinates. There is no restriction on the information given to the three-dimensional point, but at least position information (X, Y, Z coordinates) and time are given, and the three-dimensional point group is a collection of two or more three-dimensional points. It is a set.

[First Embodiment]
<Configuration of Facility Detection Device 100 of First Embodiment>
Next, the structure of the equipment detection apparatus of 1st Embodiment is demonstrated.

  As shown in FIG. 1, the facility detection apparatus 100 according to the first embodiment includes an input unit 10, a calculation unit 20, and an image output unit 90.

  The input unit 10 receives a user operation on the GUI screen output by the image output unit 90.

  The calculation unit 20 includes an interface unit 22, an analysis unit 24, an image generation unit 26, and a storage unit 28.

  The interface unit 22 draws the GUI screen output from the image output unit 90 and grasps the content of the user operation accepted by the input unit 10.

  The analysis unit 24 detects a facility around the road by analyzing a three-dimensional point group, which is a set of three-dimensional points representing a position on the surface of the object, prepared in advance.

  The image generation unit 26 generates a highlighted image that is highlighted with reflection of the detection result of the analysis unit 24 with respect to a previously shot image, and acquires an image in which the shooting range overlaps with the highlighted image. The generated highlight image and the acquired image are output to the interface unit 22 so as to be displayed on the GUI screen.

  The storage unit 28 includes a three-dimensional point cloud storage unit 30, an image storage unit 32, a measurement route storage unit 34, and an equipment information storage unit 36.

  The three-dimensional point group storage unit 30 stores a three-dimensional point group that is a set of three-dimensional points that represent positions on the surface of an object, which are measured in advance by a device that measures the three-dimensional point group. The 3D point cloud measuring device can measure the distance between the subject and the sensor, such as laser scanner, laser range finder, ultrasonic sensor, infrared sensor such as Microsoft Kinect (registered trademark), or ultrasonic sensor. Device. For example, a laser scanner is mounted on a GPS-equipped car or the like and is measured while moving, so that an outdoor environmental feature is a subject, such as a guardrail, road ground, cable, and building. Then, the three-dimensional position of the subject surface is measured. In the present embodiment, an MMS (Mobile Mapping System) in which a GPS and a laser scanner are mounted on a vehicle is assumed as an apparatus for measuring a three-dimensional point group. In addition, the apparatus which measures a three-dimensional point group may output the three-dimensional point group calculated | required from the depth information acquired with the stereo camera.

  The image storage unit 32 stores a plurality of camera images photographed by a camera together with measurement by a device that measures a three-dimensional point group. Camera parameters (focal length, shooting position, camera posture, shooting time) obtained at the time of shooting are given to the plurality of camera images.

  The measurement route storage unit 34 stores route information traveled by a vehicle on which the device is mounted when measurement is performed by the device that measures the three-dimensional point group.

  The facility information storage unit 36 stores information on facilities around the road detected by the analysis unit 24. Moreover, the facility information storage unit 36 stores a facility management map that stores information on facilities around the road. In the facility management map, for example, information on each facility around the road in a three-dimensional space is stored.

  The interface unit 22 includes an equipment management GUI function drawing unit 38, a user GUI operation content grasping unit 40, and an on-GUI image drawing unit 42.

  The facility management GUI function rendering unit 38 renders the facility management GUI function operated by the user on the GUI screen output by the image output unit 90. For example, a button for a click operation by the user is drawn on the GUI screen.

  The user GUI operation content grasping unit 40 grasps the content of the operation on the GUI screen by the user.

  The on-GUI image drawing unit 42 draws the image output by the image generation unit 26 on the GUI screen output by the image output unit 90.

  The analysis unit 24 includes an infrastructure equipment detection unit 44 and a reanalysis processing target point group determination unit 46.

  The infrastructure equipment detection unit 44 performs a point cloud analysis on the 3D point cloud stored in the 3D point cloud storage unit 30 to detect equipment around the road. As an analysis method for the three-dimensional point group, a conventionally known method may be used. For example, the method described in Patent Document 1 may be used.

  In addition, the infrastructure equipment detection unit 44 performs a point cloud analysis on the reanalysis processing target 3D point group determined by the reanalysis processing target point group determination unit 46 as described later, and detects the equipment around the road. To detect.

  The reanalysis processing target point group determination unit 46 is designated based on the region designation by the user for the image displayed on the GUI screen from the three-dimensional point group stored in the three-dimensional point group storage unit 30. A three-dimensional point included in the region is detected.

  The reanalysis processing target point group determination unit 46 specifies a three-dimensional space in which a facility to be detected exists based on a region designation by a user for a plurality of images displayed on the GUI screen, and detects the facility. A three-dimensional point group to be reanalyzed is determined.

  The image generation unit 26 includes a highlighted display image generation unit 48 and a user-designated auxiliary image generation unit 50.

  The highlight image generation unit 48 compares the facility around the road detected by the infrastructure facility detection unit 44 with the information about the facility around the road stored in the facility information storage unit 36 and stores it in the image storage unit 32. An image in which highlighting is performed according to the comparison result is generated for the displayed image, and is output to the on-GUI image rendering unit 42 as an image to be displayed on the GUI screen.

  The user-designated auxiliary image generation unit 50 acquires an image of the imaging range including the three-dimensional point detected by the reanalysis processing target point group determination unit 46 from the image stored in the image storage unit 32, and displays the image on the GUI screen. Is output to the image rendering unit 42 on the GUI.

  The facility detection apparatus 100 of the present embodiment includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM that stores programs and various data for executing pre-processing and interactive detection processing described later, and (Read Only Memory). The CPU of the present embodiment functions as each unit included in the calculation unit 20 by executing the program.

<Operation | movement of the equipment detection apparatus 100 of 1st Embodiment>
Next, operation | movement of the equipment detection apparatus 100 of 1st Embodiment is demonstrated.

  First, while a vehicle equipped with a device for measuring a three-dimensional point cloud is traveling, the three-dimensional point group is measured at each time and photographed by a camera, and the obtained three-dimensional point group is stored in a three-dimensional point group. A plurality of camera images to which camera parameters obtained at the time of photographing are added are stored in the image storage unit 32 while being stored in the unit 30.

  Next, as shown in FIG. 2, pre-processing is executed in the equipment detection apparatus 100 of the present embodiment.

  In step S100, the infrastructure equipment detection unit 44 performs a point cloud analysis on the 3D point cloud stored in the 3D point cloud storage unit 30 to detect equipment around the road, and the detected road surroundings. The facility information is stored in the facility information storage unit 36.

  In step S <b> 102, the highlighted image generation unit 48 displays an area representing the facility of each camera image stored in the image storage unit 32 for each of the facilities around the road detected in step S <b> 100. Calculate as Accordingly, in response to the fact that the equipment cannot be detected with 100% accuracy by automatic detection, it is possible to visually check and find a wrong one or a detection omission.

  In step S104, the information on each facility stored in the facility management map stored in the facility information storage unit 36 is compared with the information on the facilities in the vicinity of the road detected in step S100. The calculated portion is calculated as a highlight display area in each camera image stored in the image storage unit 32. Further, for each camera image stored in the image storage unit 32, an image in which the highlighted display area calculated in steps S102 and S104 is highlighted is generated, stored in the image storage unit 32, and the preprocessing routine is finished. To do. As a result, the detection result can be displayed in an easy-to-understand manner on the camera image, so that the visual confirmation can be performed efficiently.

  Next, as shown in FIG. 3, interactive detection processing is executed in the equipment detection apparatus 100 of the present embodiment.

  First, in step S110, the shooting time of the camera image stored in the image storage unit 32 is updated as the shooting time of the processing target. Alternatively, the first shooting time of the camera image stored in the image storage unit 32 is set as the shooting time to be processed.

  In step S112, the highlight image generation unit 48 acquires the camera image and camera parameters corresponding to the shooting time updated in step S110 from the image storage unit 32, and outputs the camera image to the on-GUI image drawing unit 42. To do.

  In step S114, the on-GUI image drawing unit 42 draws the camera image output by the image generation unit 26 on the GUI screen output by the image output unit 90 (see FIG. 4). At this time, the highlighted display area in the camera image is highlighted by blinking, painting, or the like.

  In step S <b> 116, the input unit 10 accepts a click operation on the erroneously detected part (highlighted display area) by the user, and deletes the equipment information of the erroneously detected part from the equipment information storage unit 36. In this way, erroneous detection of equipment can be deleted only by clicking on the camera image.

  In step S118, processing for determining a reanalysis target point group is performed. Step S118 is realized by the processing routine shown in FIG.

  In step S130, the input unit 10 accepts an operation for designating a region where equipment to be detected by the user exists on the camera image on the GUI screen (see FIG. 6). Thereby, when there is an equipment detection omission, it is possible to specify an area where the equipment exists.

  In step S132, the reanalysis processing target point group determination unit 46 stores the three-dimensional point group storage unit 30 based on the region specified in step S130 and the camera image and camera parameters acquired in step S112. A three-dimensional point (hereinafter also referred to as an image designated 3D point) within the designated area is detected from the three-dimensional point group. For example, as shown in FIG. 7, when each pixel in the region designated by the user is extended in the depth direction, a point group consisting of three-dimensional points that are closest to the shooting position is detected. At this time, there may be a case where an image designated 3D point also appears in an object existing behind due to a point cloud loss or an error in a user designated area.

  In step S134, the reanalysis processing target point group determination unit 46, based on the camera image and camera parameters stored in the image storage unit 32, captures the camera image in which the image designated 3D point detected in step S132 is captured. Are extracted as candidate images (see FIG. 8), and each extracted candidate image is drawn on the GUI screen by the GUI image drawing unit 42 and presented to the user (see FIG. 9).

  In step S136, selection of a candidate image by the user is accepted. For example, the user selects any one candidate image by a click operation (see FIG. 9).

  In step S138, the input unit 10 accepts an operation for designating an area in which a facility to be detected by the user exists on the selected candidate image (see FIG. 10).

  In step S140, the reanalysis processing target point group determination unit 46 determines the area specified in step S130, the area specified in step S138, the camera image and camera parameters acquired in step S112, and the above step. Based on the candidate image selected in S136 and the camera parameters, 3 in the three-dimensional space that is an overlapping area of the two specified areas from the three-dimensional point group stored in the three-dimensional point group storage unit 30. A dimension point group is detected (see FIG. 11).

  For example, as shown in FIG. 12, when the pixels in the region designated in step S130 are extended in the depth direction, the range of ΔD from the position of the three-dimensional point that is closest to the shooting position, and in step S138, When a pixel in the specified area is extended in the depth direction, a three-dimensional space defined by the range of ΔD from the position of the three-dimensional point closest to the shooting position is specified, and the specified three-dimensional space The three-dimensional point group is detected.

  In step S142, the highlight image generation unit 48 emphasizes the detected three-dimensional point group based on the camera image and camera parameters acquired in step S112 and the three-dimensional point group detected in step S140. The displayed camera image is generated, and the on-GUI image drawing unit 42 draws the highlighted camera image on the GUI screen, presents it to the user, and ends the processing routine. Note that the candidate image selected in step S136 may be highlighted.

  As described above, it is necessary to reanalyze the equipment that becomes a detection failure by using a three-dimensional point group. However, it is possible to detect it with high accuracy by manually teaching the range to be reanalyzed.

  Then, in step S120 of the interactive detection process shown in FIG. 3, the input unit 10 accepts designation of the type of equipment to be detected by the user (see FIG. 4 above) and accepts a reanalysis execution instruction. A point cloud analysis is performed again on the three-dimensional point group detected in step S140 to detect equipment around the road, and information on the detected equipment around the road is stored in the equipment information storage unit 36. .

  In step S122, it is determined whether or not there is a detection failure facility. When a click operation on the “next time” button (see FIG. 4) by the user is accepted, it is determined that there is no equipment that is not detected, and the process proceeds to step S124. On the other hand, if not, the process returns to step S118, and the designation of the area is accepted again.

  In step S <b> 124, it is determined whether or not the shooting time that is the processing target is the last time of the camera image stored in the image storage unit 32. If the shooting time to be processed is not the last time of the camera image stored in the image storage unit 32, the process returns to step S110, and the shooting time to be processed is set to the shooting time of the next camera image. Update to On the other hand, when the photographing time to be processed is the last time of the camera image stored in the image storage unit 32, the facility management map stored in the facility information storage unit 36 in step S126. Is updated and the interactive detection process is terminated.

  As described above, according to the facility detection apparatus 100 of the first embodiment, each of a plurality of images with overlapping imaging ranges is displayed, and the facility to be detected is detected in each of the plurality of images by a user operation. By accepting the specification of the existing area and specifying the three-dimensional space where the equipment to be detected exists, the three-dimensional space where the equipment that is not detected can be accurately specified, and the reanalysis of the three-dimensional point cloud It can be performed.

[Second Embodiment]
Next, the equipment detection apparatus of 2nd Embodiment is demonstrated. In addition, since the equipment detection apparatus of 2nd Embodiment is the structure similar to the equipment detection apparatus of 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

  The second embodiment is different from the first embodiment in that a virtual image viewed from a virtual viewpoint is generated, presented to the user, and an area designation is received.

<Configuration of Facility Detection Device 100 of Second Embodiment>
The image generation unit 26 of the facility detection apparatus 100 according to the second embodiment generates a highlighted image that reflects the detection result of the analysis unit 24 with respect to a previously captured image, and also displays the highlighted image. A virtual image representing the shooting range is generated, and the generated highlighted image and the generated virtual image are output to the interface unit 22 so as to be displayed on the GUI screen.

  The reanalysis processing target point group determination unit 46 specifies a three-dimensional space in which the equipment to be detected exists based on the region designation by the user for the highlighted image and the virtual image displayed on the GUI screen. A three-dimensional point group to be reanalyzed for detection is determined.

  The user-designated auxiliary image generation unit 50 acquires a virtual image viewed from a virtual viewpoint, which represents a shooting range of a camera image for which a region is designated by the user, and displays an image on the GUI as an image displayed on the GUI screen. Output to 42.

<Operation | movement of the equipment detection apparatus 100 of 2nd Embodiment>
Next, operation | movement of the equipment detection apparatus 100 of 2nd Embodiment is demonstrated. In addition, about the process similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  First, as shown in FIG. 2, in the facility detection apparatus 100 according to the second embodiment, pre-processing is executed as in the first embodiment.

  And as shown in the said FIG. 3, in the equipment detection apparatus 100 of 2nd Embodiment, an interactive detection process is performed similarly to 1st Embodiment.

  At this time, the process of determining the reanalysis target point group in step S118 is realized by the process routine shown in FIG.

  In step S130, the input unit 10 accepts an operation for designating a region where equipment to be detected by the user exists on the camera image on the GUI screen (see FIG. 6 above).

  In step S132, the reanalysis processing target point group determination unit 46 stores the three-dimensional point group storage unit 30 based on the region specified in step S130 and the camera image and camera parameters acquired in step S112. A three-dimensional point (image designated 3D point) within the range of the designated region is detected from the three-dimensional point group that has been designated.

  In step S200, the user-specified auxiliary image generation unit 50 calculates a three-dimensional area of the shooting range of the camera image based on the camera image and camera parameters acquired in step S112 (see FIG. 14).

  In step S <b> 202, the user-specified auxiliary image generation unit 50 uses the three-dimensional points included in the three-dimensional area of the imaging range calculated in step S <b> 200 from the three-dimensional point group stored in the three-dimensional point group storage unit 30. A group is acquired, and a virtual image obtained by viewing the acquired three-dimensional point group from each of a plurality of virtual viewpoints is generated as a candidate image. Then, each of the generated candidate images is drawn on the GUI screen by the GUI image drawing unit 42 and presented to the user (see FIG. 15). For example, a virtual image viewed from directly above, a virtual image viewed from an oblique direction, and a virtual image viewed from the vertical direction with respect to the travel locus of the MMS are generated.

  In step S204, the selection of the virtual image by the user is accepted. For example, the user selects any one virtual image by a click operation (see FIG. 15 above).

  In step S <b> 206, the input unit 10 accepts an operation for designating a region in which a facility to be detected by the user exists on the selected virtual image (see FIG. 16).

  In step S208, the reanalysis processing target point group determination unit 46 determines the area specified in step S130, the area specified in step S206, the camera image and camera parameters acquired in step S112, and the above step. Based on the virtual image selected in S204 and the viewpoint direction of the virtual image, a three-dimensional space that is an overlapping region of the two specified regions from the three-dimensional point group stored in the three-dimensional point group storage unit 30 Among them, a three-dimensional point group is detected (see FIG. 17).

  For example, as shown in FIG. 12, the range of ΔD from the position of the three-dimensional point that is closest to the shooting position when the pixels in the region specified in step S130 are extended in the depth direction, and step S138. A three-dimensional space defined by the range of ΔD from the position of the three-dimensional point closest to the shooting position when the pixels in the region designated by the depth are extended in the depth direction, and the specified three-dimensional space A three-dimensional point group is detected.

  In step S142, the highlight image generation unit 48 emphasizes the detected three-dimensional point group based on the camera image and camera parameters acquired in step S112 and the three-dimensional point group detected in step S208. The displayed camera image is generated, and the on-GUI image drawing unit 42 draws the highlighted camera image on the GUI screen, presents it to the user, and ends the processing routine. Note that the virtual image selected in step S204 may be highlighted.

  In addition, about the other structure and effect | action of the equipment detection apparatus 100 of 2nd Embodiment, since it is the same as that of 1st Embodiment, description is abbreviate | omitted.

  As described above, according to the equipment detection device according to the second embodiment, each of the camera image and the virtual image with overlapping shooting ranges is displayed, and each of the camera image and the virtual image is displayed by a user operation. By receiving the designation of the area where the detection target equipment exists and specifying the three-dimensional space where the detection target equipment exists, it is possible to accurately specify the three-dimensional space where the detection failure equipment exists. Reanalysis of the dimension point group can be performed.

  Note that this embodiment is an example, and the specific configuration is not limited to this embodiment, and includes design and the like within a scope that does not depart from the gist of the present invention, and can be changed according to circumstances. Needless to say.

  For example, the facility detection apparatus 100 according to the second embodiment may generate only an ortho image viewed from directly above, instead of generating a plurality of virtual images. For example, the user-specified auxiliary image generation unit 50 calculates the shooting range of the camera image from the camera parameters of the camera image acquired in step S112, and stores the three-dimensional point group stored in the three-dimensional point group storage unit 30. Then, a three-dimensional point group included in the three-dimensional space of the calculated photographing range is acquired, and an ortho image obtained by viewing the acquired three-dimensional point group from directly above is generated (see FIG. 18). At this time, if road surface equipment (for example, manholes, lane markings, and gratings) is a detection target, the ground is in the reanalysis range, and therefore, a point group that exists above the ground may be removed. Then, the user-designated auxiliary image generation unit 50 determines the shooting range of the camera image on the ortho image from the camera parameters of the camera image acquired in step S112 (see FIG. 19). Then, the generated ortho image is drawn on the GUI screen by the GUI image drawing unit 42 and presented to the user, and the input unit 10 has equipment to be detected by the user on the ortho image. An operation for designating an area is accepted (FIG. 20).

  In addition, when generating a plurality of virtual images, the facility detection apparatus 100 of the second embodiment may use a virtual image viewed from a viewpoint with less influence of occlusion as a candidate image (see FIG. 21). Of the point cloud included in the area specified by the user, the area of the point cloud that exists in a wide area is the area where there is a high probability that the equipment to be detected exists, so a virtual image of the viewpoint that reflects this area widely is displayed. You may make it be a candidate image. For example, clustering is performed using a k-neighboring point group for a point group composed of image designated 3D points, and it is determined that the probability that a cluster is a certain size or larger is a detection target (see FIG. 22). Then, as shown in FIG. 23, various candidate images are generated with respect to the direction of the virtual camera viewpoint from the center of gravity position of the cluster of the image designated 3D point, and the virtual of the viewpoint where the image designated 3D point is not affected by occlusion as much as possible. The image may be a candidate image.

  Moreover, although the case where the equipment around the road is used as the detection target has been described as an example, the present invention is not limited to this, and a thing other than the equipment around the road may be used as the detection target.

DESCRIPTION OF SYMBOLS 10 Input part 20 Operation part 22 Interface part 24 Analysis part 26 Image generation part 28 Storage part 30 Three-dimensional point cloud storage part 32 Image storage part 34 Measurement route storage part 36 Equipment information storage part 38 Equipment management GUI function drawing part 40 User GUI Operation content grasping unit 42 GUI image drawing unit 44 Infrastructure facility detection unit 46 Reanalysis processing target point group determination unit 48 Highlighted image generation unit 50 User-specified auxiliary image generation unit 90 Image output unit 100 Facility detection device

Claims (8)

An interface unit that displays each of a plurality of images having overlapping shooting ranges, and receives designation of an area where a detection target exists in each of the plurality of images;
An analysis unit that identifies a three-dimensional space in which the detection target exists based on designation of a region in each of the plurality of images;
A three-dimensional space specifying device.   The three-dimensional space specifying device according to claim 1, wherein each of the plurality of images is an image to which a shooting position is given. A user-specified auxiliary image generation unit;
The interface unit displays an image, and in the image, accepts designation of a region where a detection target exists,
The analysis unit detects a three-dimensional point included in the designation of the region in the image from a three-dimensional point group that is a set of three-dimensional points representing a position on the surface of the object prepared in advance.
The user-specified auxiliary image generation unit acquires an image of a shooting range including a three-dimensional point detected by the analysis unit,
The three-dimensional space identification device according to claim 1 or 2, wherein the interface unit displays an image acquired by the user-specified auxiliary image generation unit, and receives designation of a region where the detection target exists in the image. A user-specified auxiliary image generation unit;
The interface unit displays an image, and in the image, accepts designation of a region where a detection target exists,
The analysis unit calculates a three-dimensional region corresponding to the imaging range of the image,
The user-specified auxiliary image generation unit includes a point group included in the three-dimensional region calculated by the analysis unit from a three-dimensional point group that is a set of three-dimensional points representing a position on the surface of the object. , Representing the acquired point cloud and generating a virtual image viewed from a virtual viewpoint,
3. The three-dimensional space identification according to claim 1, wherein the interface unit displays a virtual image generated by the user-specified auxiliary image generation unit, and receives designation of a region where the detection target exists in the virtual image. apparatus.   The three-dimensional space specifying device according to claim 4, wherein the virtual image is an ortho image as viewed from directly above a point group included in the three-dimensional region calculated by the analysis unit. The detection object is a facility around the road,
The analysis unit further detects a facility around a road by analyzing a point group composed of three-dimensional points included in a three-dimensional space in which the identified detection target exists. The three-dimensional space identification device according to any one of the preceding claims. The interface unit displays each of a plurality of images having overlapping shooting ranges, and receives designation of a region where a detection target exists in each of the plurality of images,
The analysis unit specifies a three-dimensional space in which the detection target exists based on designation of a region in each of the plurality of images.   The program for functioning a computer as each part of the three-dimensional space identification apparatus of any one of Claims 1-6.