KR20180032138A - Method for Surveying and Monitoring Mine Site by using Virtual Reality and Augmented Reality - Google Patents

Abstract

The present invention relates to a method for surveying a mine and monitoring a site using virtual reality (VR) and augmented reality (AR) which can acquire accurate VR state information. The method for surveying a mine and monitoring a site using VR and AR comprises: a reality image transmission step (S100) of linking an imaging camera (100) (coordinate recognizable by GPS and 360 degrees rotatable and tiltable) moved and installed on a required position in a mine to a VR system (200) to operate the imaging camera (100) to transmit a reality image acquired for bench terrain (22) or a transport vehicle (40) for ore transport by the imaging camera (100) to the VR system by wireless communication; a virtual reality data generation step (S200) of processing the reality image by the VR system (200) to generate virtual reality data for information of a state of the bench terrain (22) or a state of the transport vehicle (40) for ore transport; and an augmented reality screen realization step (S300) of transmitting the virtual reality data to an augmented reality device possessed by a site supervisor by wireless communication to display an augmented reality screen including the virtual reality data.

Description

Technical Field [0001] The present invention relates to a method and apparatus for monitoring a mine site using VR and augmented reality,

The present invention relates to a method and apparatus for setting a virtual camera such as a VR (virtual) modeling information, such as an actual screen acquired from a video camera at a position where a supervisor needs a field surveillance or surveying and an optical body of a limestone mine collected from an actual screen, The present state of the equipment used in the mine and the slope in the mine (for example, in case of open cut lime mine, limestone cement Slope) or tunnel status in real time.

Recently, VR (Virtual Reality, Virtual Reality, Virtual Reality, Virtual Reality, Virtual Reality), which uses a special wearable devise such as a special eyeglass or headset to simulate what is possible in a software program of a computer through a human sense of sight, Reality technology has been developed and further, as disclosed in Korean Registered Patent Publication (B1) 10-1575016 (Announcement issued on December 01, 2015) and Japanese Registered Patent Publication (B2) P5603377 (Announcement issued on Aug. 29, 2014) AR (Augmented Reality) technology that combines virtual worlds with additional information in real time into a single image by displaying virtual objects overlaid on the real world viewed by eyes is becoming a reality.

Although it is not a real reality, it means a simulation technology that enables a user to experience an environment such as reality through a three-dimensional sight (Sight). Such a virtual reality is an additional experience that is difficult to obtain based on a virtual space and objects This virtual reality allows a user to immerse himself in a virtual environment consisting of three dimensions but allows the user to experience a realistic experience in the virtual environment As a result, the virtual reality can be applied not only to games, animation, and the like but also to various environments. Especially, the virtual reality is attracting attention as a next generation display technology suitable for a ubiquitous environment.

Particularly, in the field of mining technology to which the present invention is applied, the introduction or application level of the latest technology is very low compared to other technical fields, and the case of introducing the above-described virtual reality or augmented reality technology into the mining scene .

For example, in a mine site where bench-cut (open-cut) blasting / mining such as limestone mines is performed, it is necessary to monitor the state of the change in bench topography and the state of change as a result of bench-cut blasting, While it is necessary to provide real-time monitoring of the operation of vehicles transporting ores (limestone ore), it is necessary to provide a means for integrated monitoring through virtual reality technology or augmented reality technology. However, It is difficult to expect active development.

Also, in the case of underground mining development, if the modeling of mining or tunnel modeling for precise mining reaches a certain level, it is necessary to use the modeling data for production timing planning technology. In this case, The virtual reality data on the operation of the vehicle body through the change and the tunnel will be obtained as augmented reality and its usefulness will be greatly improved.

Korean Registered Patent Publication (B1) 10-1575016 (Dec. 01, 2015) Japanese Patent Registration No. B2 P5603377 (Aug. 29, 2014)

SUMMARY OF THE INVENTION It is an object of the present invention to solve the general and common problems of the prior art described above and it is an object of the present invention to provide a mine which performs a bench-cut (open-cut) blasting such as a limestone mine, The present invention provides a means for obtaining virtual reality data by transferring a bench topography state changed by bench-cut blasting in the field and a video image obtained through an image camera for an ore transporting vehicle to a VR system by wireless communication , Providing a technical means for displaying the real image image and the virtual reality data processed in the VR system through the personal augmented reality device possessed by the field supervisor and providing accurate virtual reality status information for the three- And to provide 3D modeling means capable of obtaining 3D modeling means.

In addition, by extending the 3D modeling technology, the bench topography state changed by bench-cut blasting at the mine site performing bench-cut (open-cut) blasting such as limestone mine, The volume of the ore pile is calculated from the three-dimensional image of the pile, the volume of the ore being transported in the image image of the ore transporting vehicle is calculated, the relationship between the calculated volume values is confirmed, It also provides a means to reflect on subsequent bench-cut designs for batch operations or to establish optimized ore transport plans (post vehicle deployment plans) following a subsequent bench-cut blast, further extending virtual and augmented reality technologies And the future image of the future plan of the bench-cut to be subsequently blasted into the virtual reality It is another technical task of the present invention to provide a technical means for displaying through a personal augmented reality device currently owned by a field supervisor.

In order to solve the above-described technical problem, the present invention provides a mining survey and a site monitoring method using VR and augmented reality, ) Is operated in conjunction with the VR system to transmit the acquired real image image to the VR system by wireless communication with respect to the bench top or the ore transporting vehicle via the image camera;

A virtual reality data generation step of processing the real image image in a VR system to generate virtual reality data on a bench topography state or a transportation vehicle state for ore transportation; And

And an augmented reality screen display step of displaying the augmented reality screen in a form of an augmented reality screen including virtual reality data by transmitting the virtual reality data to a personal augmented reality device held by a field supervisor by wireless communication and performing a mine survey using the augmented reality And on-site monitoring methods.

Here, in the case of the application of the bench topography state, when a specific point of the bench structure is selected by the video camera interlocked with the VR system in the real image transmission step, the virtual reality data generation step generates the distance information Virtual reality data information on height information, coordinate information, or geometry information including slope inclination is generated, and in the augmenting reality screen realization step, virtual reality data information on geometry information on a specific point is obtained It is preferable that the image is transmitted to the personal augmented reality device and output to the augmented reality like the real scene image. In the case of the underground mining, not the outdoor light, the image camera and the VR system It can be operated.

In addition, in the case of an application example of the transportation vehicle state for ore transportation, when a specific transportation vehicle is selected by the image camera interlocked with the VR system in the real image transmission step, the virtual reality data generation step Information such as vehicle number, driver information, vehicle speed, or performance on the day is generated as virtual reality data information, and in the augmented reality screen implementation step, the information is transmitted to the personal augmented reality device so as to be output to the augmented reality It would be desirable to configure it.

Furthermore, in order to perform accurate 3D modeling on a three-dimensional structure such as a bench topographical state, not only can three-dimensional modeling be performed using stereoscopic images obtained by a plurality of (two or more) image cameras, , A laser instrument is attached to the image camera, a measurement area is determined on the VR screen acquired by the image camera, and laser measurement is performed within a predetermined range of the measurement area using the laser instrument, Quot; value "

In order to perform accurate 3D modeling of the ore pile after the blasting of the bench, if the ore pile after the bench blasting is selected by the image camera interlocked with the VR system, the 3D pile of the ore pile is 3D- It is more preferable to generate virtual reality data information on the volume (expected transport volume) information and transmit it to the personal augmented reality device like the actual scene image.

On the other hand, if a specific transportation vehicle in the mine is selected by the video camera linked with the VR system, the transportation information and the performance information of the transportation vehicle are selected. Then, the 3D modeling is performed on the volume of the ore carried on the transportation vehicle, It is more preferable to calculate the ore volume and to configure the virtual reality data information including the transport ore volume to be transmitted to the personal augmented reality device like the actual vehicle image image and output.

Of course, in order to provide optimized mine operation information beyond the above mentioned simple monitoring activities, the estimated transport volume value calculated from the 3D modeling result of the ore pile after bench blasting, the volume of the ore carried on the transport vehicle The relationship between the transport ore volume values calculated from the 3D modeling results for a given benchmark-cut blasting design and the expected mining ore volume according to the planned bench topographic change state (processed by the VR system and output to the AR device) And corrects errors in the volume calculation process according to their relevance and accuracy, recalculates the optimal ore transport vehicle batch schedule, or recalculates the optimal ore transport vehicle batch schedule in consideration of the set ore transportation schedule It may also be configured to recompute the optimization data of the bench-cut design.

According to the present invention, detailed information (virtual reality data) about the bench topography state and the orbit transport vehicle for ore transporting by bench-cut blasting at the mine site performing bench-cut (open-cut) blasting such as limestone mine, And a real-time image obtained through a video camera, and can be displayed in an augmented reality manner through a personal augmented reality device possessed by a field supervisor, thereby facilitating easier on-site supervision. In addition, It is possible to obtain an accurate virtual reality state information on the structure.

Further, according to the present invention, by extending the 3D modeling technique, a bench topography state changing by bench-cut blasting at a mine site performing a bench-cut (open-cut) blasting such as a limestone mine, The relationship between the volume estimate of the ore pile calculated from the three-dimensional image of the ore pile remaining on the bench and the calculated volume of the ore being calculated from the image of the ore carrier transport vehicle is checked, Cut design, or to establish an optimized ore transport plan (transport vehicle layout plan) after a subsequent bench-cut blast, and furthermore, to provide a virtual reality And augmented reality technology, it is possible to apply not only the real image image of the mine site but also the future plan image of the subsequent bench- It is possible to display the information on a personal augmented reality device owned by a field supervisor by implementing the virtual reality as a virtual reality so that the mining scene monitoring can be extended not only to the present time but also to a future time, .

1 is a basic conceptual diagram of a preferred embodiment of a state in which the present invention is performed.
2 is a flow chart of an embodiment of the present invention.
3 is a diagram illustrating a state in which virtual reality data information on geometry information for a specific point K is transmitted to the personal augmented reality device 300 and output to the augmented reality together with the actual field image image.
4 is a diagram illustrating a state in which, when a specific transportation vehicle 40 is selected, its driving information is generated as virtual reality data information and output to an augmented reality together with an actual vehicle image image.
5 and 6 are explanatory diagrams for explaining a state in which the laser camera 150 is attached to the image camera 100 to perform three-dimensional modeling.
Figures 7 to 10 illustrate a further illustrative embodiment of the present invention, a VR & AR exemplary screen state, and a flow diagram.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art to which the present invention pertains. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.

According to a preferred embodiment of the present invention, as shown in Figures 1 to 6,

A method for monitoring and monitoring a mine using a virtual reality (VR) and augmented reality (AR), comprising the steps of: capturing an image camera (100) The VR system 200 is operated in cooperation with the VR system 200 to transmit the obtained real image image to the bench top 22 or the ore transporting vehicle 40 via the video camera 100 by wireless communication, (S100);

A virtual reality data generation step (S200) of processing the real image image in the VR system (200) to generate virtual reality data for a bench top (22) state or an ore transporting vehicle (40) state information; And

And a step of constructing an augmented reality screen (S300) in which the virtual reality data is transmitted to a personal augmented reality device (300) possessed by a field supervisor by wireless communication and displayed in the form of an augmented reality screen including virtual reality data We provide mine surveys and field surveillance methods using augmented reality.

Here, in the case of the application of the bench topography state, in the real image image transmission step S100, the image of the bench topography 22 is displayed by the image camera 100 interlocked with the VR system 200, When the specific point K is selected, the virtual reality data generation step S200 generates the virtual reality data for the point POINT K, the height information, the coordinate information, or the geometry information including the slope inclination, Data information is generated and the virtual reality data information on the geometry information for the specific point K is transmitted to the personal augmented reality device 300 in the augmented reality screen implementation step S300, It is preferable that the image is output to the augmented reality together with the actual scene image image.

Of course, in the case of underground mining rather than open-air mining, the video camera, VR system, and personal augmented reality device can be operated in the same manner as described above with respect to the tunnel topography and mine structure status.

When the transport vehicle 40 is selected by the video camera 100 interlocked with the VR system 200 in the real image transmission step S100, In the virtual reality data generation step (S200), the driving information (such as the vehicle number, the driver information, the vehicle speed, or the performance of the day, etc.) for the transportation vehicle 40 is generated as the virtual reality data information, In the screen realization step S300, it is preferable that the virtual reality data information is transmitted to the personal augmented reality device and output to the augmented reality together with the actual vehicle image as illustrated in Fig.

Meanwhile, the real image displayed on the personal augmented reality device 300 in the step of implementing the augmented reality screen (S300) may be obtained by the personal augmented reality device 300. In this case, 22 or the visual or optical means (a camera with a built-in personal augmented reality device; for example, a camera incorporated in the smartphone illustrated in Fig. 4) to the ore transporting vehicle 40, and Otherwise, if the field supervisor is located at a remote place, the real image displayed on the personal augmented reality device 300 may be transmitted through the VR system 200 in the augmented reality screen implementation step S300.

Further, in order to perform accurate 3D modeling on a three-dimensional structure such as the state of a bench topography, in another preferred embodiment according to the present invention, image images acquired by a plurality of (two or more) image cameras are used in three dimensions So that three-dimensional modeling can be performed. 5 and 6, a laser instrument 150 is attached to the image camera 100, and a measurement area (ABCD in FIG. 5) is displayed on the VR screen acquired by the image camera 100, (For example, an area where four points are determined and set based on the upper boundary line 22ua and the lower boundary line 22da of the bench front), and then the laser instrument 150 is used as shown in FIG. 6 (Laser scan) on the entire boundary lines 22da, 22db, 22dc, 22ua, 22ub, 22uc within the predetermined measurement area range (ABCD area in FIG. 5) (Measurement result) of the measurement result. Here, the measurement result thus measured can be stored in a point cloud, and further, it can be used to construct a stereoscopic image structure by superimposing on a photographed image image.

According to a further embodiment of the present invention shown in FIGS. 7 to 10, in order to perform accurate 3D modeling of the ore pile 25 after bench blasting, a video camera (not shown) associated with the VR system 200 100, when the ore dummy 25 after the bench blasting is selected, 3D rendering is performed on the ore dummy 25 to generate virtual reality data information on the ore dummy volume (estimated transport volume) information, It is more preferable to transmit the image data to the augmented reality device 300 and output it together with the actual field image image. In this case, since the boundary line of the ore pile volume is not entirely exposed to the outside, a plurality of cameras or laser instruments are used However, accurate measurement is impossible in reality. However, real image images and bench blasting of bench topography (22) (Information stored in the VR system as virtual reality data) such as a virtual image by the estimated blasting boundary line (refer to 22a ', 22b', 22c 'in FIG. 7) It is possible to reconstruct a 3D virtual image of a broken ore pile due to the current blasting, and it is possible to calculate a 3D volume using the 3D virtual image. 7, it can be understood that reference numeral '10' denotes a topsoil, '20' denotes an upper bench, and '30' denotes a bench road.

On the other hand, if the specific transportation vehicle 40 in the mine is selected by the video camera 100 linked to the VR system 200, only the transportation information (including simple performance information such as the number of operations) 3D modeling is performed on the volume of the water-receiving ore 25a loaded on the transport vehicle 40 to calculate the transport ore volume, and the virtual reality data information including the transport ore volume information is transmitted to the personal augmented reality device 300 It is more preferable that the image is transmitted as shown in FIG. 8 together with the actual vehicle image image. Even in this case, even if a plurality of cameras or laser measuring instruments are used, accurate measurement can not be practically realized because the entire boundary line of the volume of the transport ore 25a loaded on the loading box of the transportation vehicle is not exposed to the outside, A separate 3D virtual image for the portion of the transportation ore 25a currently being transported is obtained by using additional information (information stored in the VR system as virtual reality data) such as a virtual image by the real image image of the empty state And it is possible to calculate the three-dimensional volume using this.

In addition, according to a further embodiment of the present invention, the expected transport volume value calculated from the 3D modeling result for the ore pile after bench blasting so that the mine operation information optimized can be provided beyond the aforementioned simple monitoring activity, To confirm the association between the transport ore volume values calculated from the 3D modeling results on the volume of ore transported and loaded and the expected bench top change state in accordance with the subsequent bench-cut blasting design (processed by the VR system, , And corrects errors in the volume calculation process, recalculates the optimal ore transport vehicle scheduling according to its relevance and accuracy, or recalculates the optimum ore transport schedule Optimization data re-calculation of subsequent bench-cut design considering transport vehicle layout plan To this end, as shown in FIGS. 9 and 10, a predetermined bench top change state according to the subsequent bench-cut blasting design is processed in the VR system and output through the AR device as a personal augmented reality device A subsequent bench estimated mining ore volume calculation (VR) / output (AR) step S2100 for calculating the expected bench mining ore volume and outputting it as virtual reality data; A subsequent bench blasting step (S2200); Actual blasting mined ore pile volume calculation (VR) / output (AR) step S2300; (VR) / output (AR) step (S2400) on the transport vehicle is carried out, and the predicted transport volume value calculated from the 3D modeling result of the ore pile after bench blasting, the estimated transport volume value of the ore After the comparison and analysis step S2500 to ascertain the association between the transport ore volume values calculated from the 3D modeling results of the volume, if there is an error (error existence ?: YES), the volume calculation process error correction If the predetermined bench top change state (output by the VR system and output to the AR device) according to the subsequent bench-cut blasting design, as previously mentioned, proceeds to step S2600, if not (error existence ?: NO) (S2700a) or re-calculating the transportation vehicle layout schedule by checking the accuracy with the expected mining ore volume value (S2700b).

In the present specification, a specific physical configuration of the VR system 200 or software detailed configuration for processing the virtual reality data therefrom is omitted, and the personal augmented reality device 300 used in the on- (Augmented reality) of the synthesized image is beyond the scope of the present invention, and a detailed description thereof will be omitted. However, it is understood that these basic technologies do not interfere with understanding and practicing the present invention, as there are various implementations in various technology types at present, which are different from commercialization level.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, Of the right.

1: mine
22: Bench topography
25: Ore pile
25a: Transport ore
40: Transport vehicle
100: Video camera
200: VR system
300: personal augmented reality device
S100: Actual image image transmission step
S200: Virtual reality data generation step
S300: Step of implementing an augmented reality screen

Claims (6)

As a method of mining survey and site monitoring using VR (Virtual Reality) and Augmented Reality (AR), a video camera 100 (capable of recognizing coordinates by GPS, capable of rotating 360 degrees and tilting) VR system 200 and transmits the acquired real image image to the VR system 200 by wireless communication through the video camera 100 for the bench top 22 or the ore transporting vehicle 40 A step S100 of transmitting a real image;
A virtual reality data generation step (S200) of processing the real image image in the VR system (200) to generate virtual reality data for a bench top (22) state or an ore transporting vehicle (40) state information; And
And a step of constructing an augmented reality screen (S300) in which the virtual reality data is transmitted to a personal augmented reality device (300) possessed by a field supervisor by wireless communication and displayed in the form of an augmented reality screen including virtual reality data Mining survey and field monitoring method using augmented reality. 2. The method according to claim 1, wherein, in the case of the application of the bench topography condition, a specific point K of the bench topography 22 is detected by the video camera 100 interlocked with the VR system 200 in the step S100 of transmitting the real- Virtual reality data information on the geographical information for the point POINT K is generated in the virtual reality data generation step S200 and the specific point K is generated in the augmented reality screen realization step S300, Is transmitted to the personal augmented reality device (300) and outputted to the augmented reality together with the actual scene image, and the method for monitoring the mine survey and the field using the augmented reality. 3. The method according to claim 2, wherein, in the case of application of the transport vehicle state for ore transport, the specific transport vehicle (40) is transported by the video camera (100) interlocked with the VR system (200) When the virtual reality data is generated, the driving information for the transportation vehicle 40 is generated as the virtual reality data information in the virtual reality data generation step S200. In the augmented reality screen realization step S300, Device and is output to an augmented reality together with an actual vehicle image image. 3. The method of claim 2, wherein in order to perform accurate 3D modeling on a three-dimensional structure, such as a state of a bench topography,
A laser measuring instrument 150 is attached to the image camera 100 to set a survey area on the VR screen acquired by the image camera 100 and then to use the laser instrument 150 to set a predetermined survey area range Dimensional structure by performing a laser survey within the VR and augmented reality. ≪ RTI ID = 0.0 > [0002] < / RTI > Method according to any one of claims 1 to 4, characterized in that in order to carry out accurate 3D modeling of the ore pile (25) after bench blasting, the bench blasting is performed by the video camera (100) interlocking with the VR system (200) 3D rendering of the ore dummy 25 is performed to generate virtual reality data information for the ore dummy volume information and transmits it to the personal augmented reality device so that the real scene image Are configured to be output together, and
When the specific transportation vehicle 40 in the mine is selected by the video camera 100 interlocked with the VR system 200, only the transportation information for the transportation vehicle is obtained, and the transportation ore 25a to calculate the volume of transport ore, and the virtual reality data information including the transport ore volume information is transmitted to the personal augmented reality device 300 and is output together with the actual vehicle image image Mining survey and field monitoring using VR and augmented reality. 6. The method of claim 5,
The following bench topographic change state according to the subsequent bench-cut blasting design is processed in the VR system and outputted through the AR device as a personal augmented reality device, and the subsequent bench estimated mining ore volume is calculated and output as virtual reality data. A volume calculation (VR) / output (AR) step S2100;
A subsequent bench blasting step (S2200);
Actual blasting mined ore pile volume calculation (VR) / output (AR) step S2300; And
(VR) / output (AR) step (S2400) on the transport vehicle, the predicted transport volume value calculated from the 3D modeling result of the ore pile after bench blasting, the volume of ore transported in the transport vehicle (S2500) in order to confirm the correlation between the transporting ore volume values calculated from the 3D modeling results of the 3D modeling result
If there is an error (existence of an error? YES), the volume calculation process error correction step S2600 is performed,
By checking the accuracy of the predicted mining ore volume value according to the predetermined bench top change state according to the subsequent bench-cut blasting design in the step S2100, (S2700a) or the transportation vehicle layout schedule is re-calculated (S2700b).

Images