JP2020098481A - Display system - Google Patents

Abstract

To enable a positional relationship of installed objects to be grasped by a worker even when design data is nonexistent.SOLUTION: A display system S comprises: an installed object sensor 1 for detecting an installed object behind a wall; a stereo camera 22 and a 9-axis sensor 23 for detecting position information at a detection point by the installed object sensor 1; a storage device 32 for storing installed object information I in which the installed objects detected by the installed object sensor 1 and the position information detected by the stereo camera 22 and 9-axis sensor 23 are associated; a smart glass 2 for displaying the installed object information I stored by the storage device 32 by superimposition; and a controller 3 for causing installed object information pieces I1, I2, out of the installed object information I stored by the storage device 32, that correspond to visual fields V1, V2 of the smart glass 2 over a display 21 to be displayed by the smart glass 2.SELECTED DRAWING: Figure 1

Description

The technology disclosed herein relates to a display system.

A so-called head-mounted display device (head mounted display: HMD) is widely known as a display device worn by a user. This HMD is roughly classified into a non-transmissive HMD that allows a user to visually recognize only information without transmitting external light, and a transmissive HMD that allows information to be superimposed and displayed while transmitting external light.

Among the HMDs classified as described above, a transmissive HMD may be used in a display system for work support in order to take advantage of the advantage of not impairing the user's visual field. In this case, the information for supporting the work can be visually recognized by the user without hindering the work by the user.

As a technique created from such a point of view, for example, Patent Document 1 discloses a display system that contributes to preventing damage to a gas pipe or a water pipe when excavating a road, for example. Specifically, the display system according to Patent Document 1 acquires the current position of the HMD using GPS or an AR marker, and acquires it from a database that stores image data indicating the piping structure of a gas pipe or a water pipe. The image of the pipe corresponding to the current position is read. By displaying the read image of the pipe on the transmissive HMD, the operator can grasp the position of the pipe and the like.

JP, 2016-194763, A

By the way, when performing maintenance inspections on the laying behind the wall, such as piping embedded in the wall of the building, construct a part of the wall around the laying that is to be inspected. Or, it is required to construct while avoiding the laying object that is not subject to inspection.

In order to meet such a demand, it is conceivable to let an operator grasp the positional relationship of the laid object in advance. Although a measure for that purpose is to prepare a design drawing prior to maintenance and inspection, this method is troublesome and inconvenient.

Therefore, it is conceivable to use the display system according to Patent Document 1 instead of preparing the design drawing. However, in order for this display system to function, image data showing the piping structure is indispensable.

The display system according to Patent Document 1 already has electronic data (so-called design data) indicating a design drawing and is useful when it can be used as image data, but a relatively old building is constructed. The inventors of the present application have noticed that, when such design data does not exist, the function does not work and does not contribute to work support.

The technique disclosed here is made in view of such a point, and the purpose thereof is to make an operator grasp the positional relationship of a laid object even when there is no design data.

The present disclosure relates to display systems. This display system includes a laying object detecting device for detecting a laying object on the back of a wall, a position information detecting device for detecting positional information of a detection location by the laying object detecting device, and a laying object detected by the laying object detecting device. And a storage device for storing the installation information associated with the position information detected by the position information detecting device, and a transmissive head-mounted display for superimposing and displaying the installation information stored by the storage device. An apparatus and a control device for displaying, on the head-mounted display device, laid-object information corresponding to a field of view through the head-mounted display device, out of the laid-object information stored by the storage device.

Here, the “field of view through the head-mounted display device” refers to the field of view through the display of the head-mounted display device, for example.

According to the above configuration, the control device displays the laying object information corresponding to the field of view through the head-mounted display device among the laying object information stored in the storage device. Therefore, even if the field of view of the worker changes, it is possible to allow the worker to visually recognize the installation information that reflects such a change. Therefore, the worker can work while looking over a wide range. This is effective in ensuring work efficiency.

Further, the above configuration does not require electronic data indicating a design drawing or the like. Therefore, even when there is no design data, it becomes possible for the operator to grasp the positional relationship of the laid object.

Further, according to the above configuration, by storing the laying object information in the storage device, the check result of the work target can be recorded. This saves time and effort for recording, which is advantageous in ensuring work efficiency.

Further, the laid object detecting device may include a radar sensor using ultra-wideband wireless.

In general, ultra-wideband wireless has excellent transparency. Therefore, by using the ultra wideband wireless, it becomes possible to detect the laying object behind the wall with high accuracy.

Further, the laid object detection device may display laid object information corresponding to the orientation of the head-mounted display device on the head-mounted display device.

According to the above configuration, the control device displays the laying object information corresponding to the orientation of the head-mounted display device among the laying object information stored in the storage device. For example, when the field of view of the worker changes, such as when the worker looks around, the orientation of the head-mounted display device changes according to the change. Therefore, by displaying the laying object information corresponding to the orientation of the head-mounted display device on the head-mounted display device, the worker can visually recognize the laying object information corresponding to the field of view of the worker. ..

The head-mounted display device may be configured as smart glasses.

A general smart glass can be attached to a construction helmet. Therefore, according to the said structure, the convenience improves at the time of construction.

As described above, according to the display system, it becomes possible for the operator to grasp the positional relationship of the laid object even when there is no design data.

FIG. 1 is a diagram illustrating an outline of a display system. FIG. 2 is a block diagram illustrating the configuration of the display system. FIG. 3 is a diagram illustrating an example of a procedure for acquiring the installation information. FIG. 4 is a diagram illustrating a display mode of the installation information.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The following description is an example.

<Outline of display system>
FIG. 1 is a diagram illustrating an overview of the display system S, and FIG. 2 is a block diagram illustrating a configuration of the display system S. As shown in FIGS. 1 and 2, the display system S includes a laying object sensor 1, a smart glass 2 and a controller 3. The detection result of the laying object sensor 1 is stored in the controller 3, and the stored contents are displayed. It can be displayed on the smart glasses 2. The display content on the smart glasses 2 is visually recognized by the worker who wears the smart glasses 2.

Hereinafter, each element which comprises the display system S is demonstrated in order.

-Layers sensor-
The laid object sensor 1 is configured as a radar sensor using ultra-wideband wireless, and can be pressed against the wall of a building to detect the laid object behind the wall. Here, the location detected by the laid object sensor 1 (so-called “sensor position”) is specified by the smart glasses 2 described later.

The laid object sensor 1 corresponds to the "laid object detecting device" in the present embodiment. The detection result by the laying object sensor 1 as the laying object detecting device is appropriately transmitted to the storage device 32 of the controller 3 wirelessly or by wire.

-Smart glasses-
The smart glasses 2 exemplify a transmissive head-mounted display device, and a display 21 for superimposing and displaying various information, a stereo camera 22 for measuring a distance, and a posture of the display 21 are detected. 9-axis sensor 23 for

Specifically, the display 21 is configured as a pair of left and right display devices, and can display Augmented Reality (AR) content based on a signal from the controller 3. The AR content displayed on the display 21 will be superimposed on the real space viewed through the display 21.

The stereo camera 22 is configured to include a pair of left and right lenses, and can measure the distance to the subject based on the optical image formed through each lens. The subject here includes at least the above-described laying object sensor 1, the detection location by the laying object sensor 1, and an AR marker 4 described later.

The 9-axis sensor 23 is configured to include a so-called gyroscope, and can detect the attitude of the smart glasses 2. By combining the detection result of the stereo camera 22 and the detection result of the 9-axis sensor 23, it becomes possible to three-dimensionally detect the position information of the detection position of the installation sensor 1.

The stereo camera 22 and the 9-axis sensor 23 correspond to the "positional information detection device" in this embodiment. The detection results of the stereo camera 22 and the 9-axis sensor 23 are appropriately transmitted to the controller 3.

-Controller-
The controller 3 includes a processor 31 that performs various processes and the storage device 32 described above, and is electrically connected to each of the laying object sensor 1 and the smart glasses 2. The controller 3 corresponds to the “control device” in the present disclosure.

The processor 31 compares the laying object detected by the laying object sensor 1 with the position information detected by the stereo camera 22 and the 9-axis sensor 23 (specifically, the position information of the location detected by the laying object sensor 1). The related installation information is generated, and the generated installation information is stored in the storage device 32. That is, the laid object information according to the present embodiment is information provided for each coordinate in the three-dimensional space, and can be defined in a so-called “sensing space”.

The relative positional relationship between this sensing space and the actual space viewed through the display 21 (hereinafter, also referred to as “real space”) is determined by, for example, disposing the AR marker 4 in the real space. be able to. In this case, the stereo camera 22 detects the AR marker 4 so that the position of the AR marker 4 in the real space is detected and the coordinate in the sensing space can be determined as the relative coordinate with respect to the AR marker 4. ..

The storage device 32 includes an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a volatile or non-volatile memory, and can store the laying object information generated by the processor 31. The storage contents of the storage device 32 are read by the processor 31 as appropriate.

The smart glasses 2 described above can superimpose and display the laid object information stored in the storage device 32 in a superimposed manner. That is, the smart glasses 2 can select the laying object information as the AR content and superimpose it on the real space. Here, the display mode of the installation information in the smart glasses 2 is controlled by the controller 3.

Specifically, the controller 3 outputs the laying object information corresponding to the view through the smart glasses 2 (specifically, the view through the display 21 of the smart glasses 2) among the laying object information stored by the storage device 32. Display on smart glass 2.

More specifically, the controller 3 detects the orientation of the smart glasses 2 based on the detection result of the 9-axis sensor 23, and estimates the field of view through the display 21 based on the orientation of the smart glasses 2 detected in this way. To do. Then, the controller 3 selects, from the laying object information stored by the storage device 32, the laying object information that is within the estimated field of view, and displays it on the display 21.

As described above, the smart glasses 2 according to the present embodiment can display the sensing space in which the laying object information is defined in an overlapping manner with the real space viewed through the display 21. By providing a worker with a mixed reality (MR) space in which a real space and a sensing space are overlapped with each other, various work support can be realized at the time of maintenance and inspection of a laid structure.

<Procedures for obtaining laying object information>
FIG. 3 is a diagram illustrating an example of a procedure for acquiring the installation information. As shown by the chain line in FIG. 3, it is assumed that a pipe-like laid object is provided behind the wall.

As shown in the upper part of FIG. 3, the worker attaches the AR marker 4 to the surface of the wall W to be inspected and brings the laid object sensor 1 closer to the surface of the wall W.

Then, as shown in the middle and lower parts of FIG. 3, the worker moves the laid object sensor 1 along the entire surface of the wall W. Thereby, the laying object P on the back of the wall is detected. The stereo camera 22 of the smart glasses 2 tracks the laying object sensor 1 in parallel with the detection of the laying object, so that the position information of the detection location by the laying object sensor 1 is three-dimensionally detected. The controller 3 causes the storage device 32 to store the installation object information I in which the installation object detected by the installation object sensor 1 and the position information detected by the smart glasses 2 are associated with each other.

<Display mode of laid object information>
FIG. 4 is a diagram illustrating a display mode of the laid object information I. The installation information I illustrated in FIG. 4 is equal to the installation information I illustrated in FIG.

As shown in the upper part of FIG. 4, when the line of sight of the worker M is directed to the right side of the drawing, the wall W on the right side of the drawing falls within the field of view V1. In this case, the controller 3 causes the display 21 to display the laying object information I1 corresponding to the field of view V1 among the laying object information I stored in the storage device 32.

On the other hand, as shown in the lower part of FIG. 4, when the worker M shakes his/her head and looks around the surroundings, when the line of sight is directed to the left side of the paper, the wall W on the left side of the paper falls within the field of view V2. In this case, the controller 3 causes the display 21 to display the laid object information I2 corresponding to the field of view V2 of the laid object information I stored in the storage device 32.

In this way, the controller 3 displays the laying object information I1 and I2 corresponding to the fields of view V1 and V2 through the smart glasses 2 among the laying object information I stored in the storage device 32. Therefore, even if the field of view of the worker M changes, the worker M can visually recognize the installation object information I1 and I2 that reflect such changes. Therefore, the worker M can work while looking over a wide range. This is effective in ensuring work efficiency.

Further, the display system S according to the present embodiment does not require electronic data indicating a design drawing or the like. Therefore, even when the design data does not exist, it becomes possible for the operator M to grasp the positional relationship of the laid object.

On the other hand, if design data such as 3D-CAD data is present, by comparing the design data with the installation information I, the difference between the design drawing and the actual installation can be grasped. It will be possible.

Furthermore, according to the display system S according to the present embodiment, by storing the laying object information I in the storage device 32, the laying object check result can be recorded. This saves time and effort for recording, which is advantageous in ensuring work efficiency.

In addition, ultra wideband radio is generally excellent in transparency. Therefore, by constructing the laid object sensor 1 as a radar sensor using ultra-wideband wireless, it becomes possible to detect the laid object behind the wall with high accuracy.

Further, as shown in FIG. 4, the controller 3 displays the laying object information I1 and I2 corresponding to the orientation of the smart glasses 2 among the laying object information I stored in the storage device 32. For example, when the field of view of the worker M changes, such as when the worker M looks around, the orientation of the smart glasses 2 changes according to the change of the field of view. Therefore, by displaying the laying object information I1, I2 corresponding to the direction of the smart glass 2 on the smart glass 2, the worker M visually recognizes the laying object information I1, I2 corresponding to the field of view V1, V2 of the worker M. It will be possible.

Further, the smart glasses 2 as a head-mounted display device can be mounted on a construction helmet. Therefore, the convenience at the time of construction is improved.

<<Other Embodiments>>
In the above embodiment, the position information detected by the stereo camera 22 and the 9-axis sensor 23 is configured to be detected as three-dimensional position information, but the configuration is not limited to this. For example, it may be configured to be detected as two-dimensional position information.

Moreover, although the said embodiment was set as the structure which used AR marker 4, it is not limited to this structure. For example, an object that can be imaged by the stereo camera 22 may be used as a characteristic point and used as a substitute for the AR marker.

1 Laying object sensor (Laying object detection device)
2 smart glasses (head mounted display)
22 Stereo camera (positional information detection device)
23 9-axis sensor (position information detection device)
3 controller (control device)
32 Storage device I Laying object information I1 Laying object information I2 Laying object information S Display system V1 Field of view V2 Field of view W Wall

Claims (4)

A laying object detection device that detects laying objects behind the wall,
A position information detection device for detecting position information of a detection location by the laid object detection device,
A laying object detected by the laying object detection device, and a storage device that stores laying object information in which the position information detected by the position information detecting device is associated.
A transmissive head-mounted display device that superimposes and displays the installation information stored by the storage device,
A control device that causes the head-mounted display device to display the laid-object information corresponding to the field of view through the head-mounted display device among the laid-object information stored by the storage device. Display system. The display system according to claim 1,
The display system, wherein the laid object detecting device includes a radar sensor using ultra-wideband wireless. The display system according to claim 1 or 2,
The display system, wherein the laid object detecting device causes the head mounted display device to display laid object information corresponding to the orientation of the head mounted display device. The display system according to any one of claims 1 to 3,
The display system, wherein the head-mounted display device is configured as a smart glass.

Images