JP2016012168A - Information sharing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To effectively perform sharing of information that allows for connection between a designer of a device and an operator who performs maintenance of the device.SOLUTION: A portable terminal device 2 includes: an imaging unit 22 that picks up an image of a device 5; and a view point data generation unit 26 that generates view point data showing a position and direction of the portable terminal device 2 with respect to the device 5 on the basis of a picked up real image of the device 5. A PC 1 includes an instruction data generation unit 18 that generates instruction data including instruction image data of an instruction image showing instruction given to an operator who uses the portable terminal device 2 and display position data specified on the basis of the view point data and appearance data showing appearance of the device 5 in order to display the instruction image superimposed on the real image of the device 5.

Description

  The present invention relates to an information sharing system for sharing information for cooperation between a designer of a device and a worker who performs maintenance of the device between the designer and the worker.

  For device maintenance, it is indispensable to collaborate with workers who are actually in charge of device maintenance work on site and device designers who are familiar with the device. In addition, since the designer is usually in an office away from the site, the communication between the designer and the worker is performed by a mobile phone conversation referring to the blueprint, and transmission of voice and video using the Internet telephone service. It is performed in the form. For example, when using a mobile phone, an operator obtains an instruction from the designer by using the mobile phone as a designer to consult about maintenance of the apparatus with reference to the design drawing. In addition, when using the Internet telephone service, the worker sends the data of the moving image obtained by photographing the device and the audio data explaining the moving image to the designer via the Internet to consult with the designer. Get instructions from.

  Examples of other cooperation between designers and workers include systems disclosed in Patent Document 1 and Patent Document 2. In Patent Document 1, hypertext data, which is a combination of fixed image screen data representing the entire plant and collected state data of each part of the plant, is transferred from the monitoring target side to the office side via a data transmission path, and the office Display on the side. Patent Document 2 superimposes the business server status updated in advance on the business server monitoring video taken by the monitoring camera in the vicinity of the corresponding business server with reference to the marker attached to the business server. It is disclosed that an augmented reality image is displayed. By using these systems, the worker can consult the designer about the maintenance of the apparatus based on the information sent from the worker side to the designer side.

JP 10-41861 A (published February 13, 1998) JP 2012-18605 A (published January 26, 2012)

  In some cases, a designer who has consulted with an operator wants to know the state of the desired part of the device in order to determine an appropriate countermeasure. For example, when it is necessary to know the state of other parts related to a part of a device that has been reported by the worker, information obtained from the worker is not sufficient.

  However, in the method using the cellular phone or Internet telephone service described above, since the voice exchange refers to the image of the device, the part of the device that the designer wants to know the state of is spoken (verbal) depending on the device configuration. ) May be difficult to convey to workers. Patent Documents 1 and 2 disclose a method for transmitting the status of the apparatus from the apparatus side (worker side) to the office side (designer side). There is no disclosure of the method of transmitting from the side to the worker side.

  As described above, according to the conventional technology, it is not easy to efficiently share information on the apparatus between the designer and the worker.

  The present invention has been made to solve the above-described problems, and its purpose is to efficiently share information for cooperation between a device designer and a worker who performs maintenance of the device. It is in.

  In order to solve the above-described problem, an information sharing system according to the present invention includes an imaging unit that images a monitoring target, and a portable terminal for the monitoring target based on an actual image of the monitoring target captured by the imaging unit. A portable terminal device having viewpoint data creating means for creating viewpoint data indicating the position and direction of the device, and instruction image data of an instruction image indicating an instruction given to a worker who uses the portable terminal device, and the instruction image Instruction data for creating instruction data including display position data in the stereoscopic arrangement region of the monitoring target specified based on the viewpoint data and the appearance data representing the appearance of the monitoring target to be displayed overlaid on the actual image The display position on the actual image captured by the imaging unit. The display position specified by the data, is characterized by having further an instruction image display processing means for performing processing for displaying overlapping the instruction image shown by the instruction image data.

  Note that the “three-dimensional arrangement area of the monitoring target” means that the captured monitoring target is displayed in a two-dimensional manner, but in reality it is a three-dimensional monitoring target that is displayed in a two-dimensional manner. It represents an area.

  In the above configuration, in the mobile terminal device, in the data creation device, the instruction data creation means provides the instruction data (instruction image data and display position data for giving an instruction to the worker based on the viewpoint data and the appearance data. Is created). Thereby, the designer who uses the data creation device can create the instruction data from the viewpoint of the worker based on the viewpoint data received from the worker who uses the mobile terminal device.

  On the other hand, in the portable terminal device, the instruction image display processing means displays the instruction position by the instruction image data on the display position specified by the display position data on the real image to be monitored based on the instruction data created by the data creation device. When the process of displaying the designated instruction image in a superimposed manner is performed, the instruction image is displayed so as to be superimposed on the actual image captured by the imaging unit.

  Therefore, the designer can transmit the instructions to the worker more clearly (intuitively) by using the instruction data as shared information, and can smoothly meet with the worker.

  In the information sharing system, the portable terminal device creates a first storage unit that stores the appearance data, comparative appearance data for comparison based on the image data of the real image, and the comparison appearance data Difference data creating means for creating difference data that is a difference from the appearance data; and first update means for updating the appearance data stored in the first storage unit based on the difference data; The data creation device further includes a second storage unit that stores the appearance data, and a second update unit that updates the appearance data stored in the second storage unit based on the difference data. It is preferable.

  In the above configuration, when the difference data is created by the difference data creation means in the mobile terminal device, the appearance data is updated based on the difference data by the first update means. Also in the data creation device, the appearance data is updated based on the difference data by the second updating means. Thereby, the shared appearance data is similarly updated in the mobile terminal device and the data creation device. Further, by using the difference data in this way, it is possible to reduce the data size when the difference data is transmitted from the mobile terminal device to the data creation device.

  In the information sharing system, the mobile terminal device divides the updated appearance data into a plurality of pieces of divided data having different densities, and sequentially converts the divided data from the sparse divided data to the dense divided data. It is preferable to further include a transmission processing means for transmitting to the device for storing the appearance data.

  In the above configuration, the divided data divided as described above is transmitted by the transmission processing means. Thereby, the content of appearance data can be estimated only by transmission of division data in the stage where transmission of all appearance data is not completed. Therefore, the work efficiency of the designer and the worker can be improved. Further, since it is possible to avoid transmitting large data at a time, even if the communication line for transmitting the divided data has a low quality, the data can be transmitted without squeezing the line.

  The present invention, which is configured as described above, has an effect of efficiently sharing information for cooperation between an apparatus designer and an operator who performs maintenance of the apparatus.

It is a block diagram which shows the structure of the information sharing system which concerns on one Embodiment of this invention. It is a figure which shows the image of 3D model data produced with the personal computer in the said information sharing system. It is a figure which shows the image of texture data. It is a figure which shows the virtual image which represented the state of the apparatus seen from the viewpoint of the worker virtually. It is the figure obtained by adding the instruction | indication given to a worker to the said virtual elephant. It is a figure which shows the image of the apparatus displayed with the portable terminal device in the said information sharing system in which the said instruction | indication was shown. It is the figure to which the imaging range of the apparatus which an operator image | photographs was added. It is the figure where the imaging | photography direction of the apparatus which an operator image | photographs was added. (A)-(c) is the figure obtained by adding the imaging | photography location and imaging | photography direction of an apparatus to the said virtual elephant. (A) is a figure which shows the texture shared by a designer and a worker, (b) is a figure which shows the texture produced based on the image image | photographed with the said portable terminal device, (c) (D) is a figure which shows the difference between the textures of (a) and (b), respectively, (e) is a figure which shows the texture updated based on the said difference. It is a figure which shows the said 3D model in which the part was updated (the state in the middle in progressive update). FIG. 3 shows a 3D model that has been fully updated (in the last state in a progressive update). It is a figure which shows the texture in which the part was updated (the state in the middle in progressive update). FIG. 5 shows a texture that has been completely updated (in the last state in a progressive update).

Embodiment
An embodiment of the present invention will be described with reference to FIGS.

  FIG. 1 is a block diagram showing a configuration of an information sharing system 100 according to the present embodiment.

  As shown in FIG. 1, the information sharing system 100 is a personal computer (indicated by “PC” in the figure, hereinafter simply referred to as “PC”) for creating various data on the designer side of the apparatus 5 (described later). 1, a mobile terminal device 2 that creates data to be provided to a designer based on the various types of data, and a network 3 that connects the PC 1 and the mobile terminal device 2 so that they can communicate with each other. In addition, the information sharing system 100 includes a server 4 that stores the various types of data as necessary.

  Note that basic 3D model data and texture data (both described later in detail) are supplied in advance to the PC 1 and the mobile terminal device 2 from the outside. The basic 3D model data and texture data are created when the device 5 is designed. In the present embodiment, the 3D model data is data (3D model data, stereoscopic appearance data) in which the stereoscopic appearance of the apparatus 5 is represented by combining triangles and quadrangles using a known modeling technique. . Further, in the present embodiment, the texture data is created based on a diagram (front view, side view, etc.) that represents the appearance of the device 5 in plan view, and the texture data is a plan view of the appearance of the device 5. It is assumed that the figure data (planar appearance data) is expressed as a model, and further represents the texture of the surface of the device 5.

  The device 5 (monitoring target) is a device to be monitored by a worker for maintenance, and is installed at a site such as a factory. The designer of the device 5 is in an office away from the site where the device 5 is installed, and gives an instruction to the worker to the portable terminal device 2 using the PC 1.

  The PC 1 (data creation device, mainly creating instruction data described later) includes a CPU, a memory, a storage device (HDD, SSD, etc.), a communication interface, etc., and a general-purpose computer on which an OS (Operating System) is mounted. It is. The PC 1 (data creation device) includes a data creation unit 10 that creates data, a storage unit 11 (second storage unit) that stores the data, a transmission / reception unit 12 that communicates with the mobile terminal device 2, and the like. It has a call unit 13 for making a call via the transmission / reception unit 12, a display monitor 14, and an input device 15 for performing an input operation.

  The data creation unit 10 includes an instruction data creation unit 18 that creates instruction data for instructions given to workers, and an update processing unit 19 that performs update processing of 3D model data and texture data.

  The instruction data creation unit 18 (instruction data creation means) creates instruction data relating to instructions given to the worker. The instruction data includes data (instruction image data) representing an instruction image representing the instruction as an image, data representing the instruction content (instruction content data), and position data (display position data) for displaying the instruction image and the instruction content. ). Furthermore, the instruction data creation unit 18 uses the viewpoint data sent from the mobile terminal device 2 to create a virtual three-dimensional figure (virtual elephant) viewed from the same viewpoint as the viewpoint at which the worker viewed the apparatus 5. Instruction data is created so that it can be overlaid. Details of the viewpoint data and the virtual elephant will be described later.

  The update processing unit 19 (second update unit) performs processing for updating 3D model data and texture data stored in the storage unit 11 based on data for update from the mobile terminal device 2.

  The mobile terminal device 2 is a terminal device including a wireless communication function, an imaging function, a display function, and an input function as basic functions, like a tablet terminal. In order to realize these basic functions, the mobile terminal device 2 includes a transmission / reception unit 20 that performs wireless communication with the network 3, a call unit 21 that performs a call via the transmission / reception unit 20, and an imaging unit that captures a subject. 22, a display panel 23 that displays an image, a touch panel 24 that receives a touch input on the display panel 23, and a control unit 25 that controls these units. The imaging unit 22 has not only an imaging function that converts an optical image of a subject into an electrical signal and outputs the electrical signal, but also an imaging function that captures the imaging data as a still image or a moving image. The control unit 25 also controls writing of data received by the transmission / reception unit 20 to the storage unit 29 and transmission of data read from the storage unit 29 by the transmission / reception unit 20.

  In addition, the mobile terminal device 2 creates difference data for basic data (shared data described later) and provides the difference data to the PC 1. In order to create such data, the mobile terminal device 2 generates a viewpoint data creating unit 26 that creates viewpoint data, an instruction image display processing unit 27 that performs processing for displaying instruction data, and imaging data (images). A difference data creating unit 28 that creates difference data (difference data) changed between the 3D model data and the texture data based on the data), and a storage unit 29 (first storage unit).

  The viewpoint data creation unit 26 (viewpoint data creation means) is data (viewpoint) indicating the viewpoint (position and direction (angle) of the mobile terminal device 2 (lens of the imaging unit 22)) when the worker views the device through the imaging unit 22. Data) is created using AR (Augmented Reality) technology. The viewpoint data is data indicating which position (how far away from the apparatus 5 is from what height) and from which direction the worker is viewing the apparatus 5 through the imaging unit 22.

  The instruction image display processing unit 27 (instruction image display processing means) performs a process of displaying the above-described instruction image on the display panel 23 based on the instruction data provided from the PC 1.

  The difference data creation unit 28 (difference data creation means) is based on the shooting data (still image) obtained by the shooting function of the imaging unit 22, and the apparatus 5 is used as the basis or the 3D model data and texture at the time of previous shooting. A difference changed from the data is created as difference data.

  The storage unit 29 is provided for storing 3D model data, texture data, and data created in the mobile terminal device 2. Specifically, the storage unit 29 stores basic texture data and 3D model data, and instruction data, and stores viewpoint data, shooting data, and difference data created in the mobile terminal device 2.

  The transmission / reception unit 20 transmits various data to the PC 1. Further, the transmission / reception unit 20 (transmission processing means) performs processing of dividing the transmission data and sequentially transmitting the data so as to progressively transmit data for updating the 3D model data and the texture data.

  The network 3 is a communication network such as the Internet or an intranet. The PC 1 and the mobile terminal device 2 are connected to each other, and a server 4 is connected as necessary. The mobile terminal device 2 is connected to the network 3 via an access point or the like for wireless communication. The server 4 stores data shared between the PC 1 and the portable terminal device 2 for purposes such as backup.

  Here, the operation of the information sharing system 100 configured as described above will be described. First, operations from creation of viewpoint data on the worker side to creation of instruction data on the designer side will be described.

  FIG. 2 is a diagram illustrating an image of 3D model data created by the PC 1. FIG. 3 is a diagram illustrating an image of texture data. FIG. 4 is a diagram showing a virtual image that virtually represents the state of the device 5 viewed from the viewpoint of the worker. FIG. 5 is a diagram illustrating an instruction image obtained by adding an instruction given to a worker to a virtual image. FIG. 6 is a diagram showing an image of the device 5 displayed on the mobile terminal device 2 in which the above instruction is shown. FIG. 7 is a diagram in which a shooting range of the device 5 for shooting by the worker is added. FIG. 8 is a diagram in which the shooting direction of the device 5 for shooting by the worker is added.

  The basic 3D model data and texture data are already shared by the PC 1 and the mobile terminal device 2 as described above.

  The worker creates viewpoint data using the mobile terminal device 2. In creating the viewpoint data, first, an operator images the device 5 using the mobile terminal device 2 (sees the device 5 through the imaging unit 22). In this state, for example, when the AR technology is used, the viewpoint data creation unit 26 reads and analyzes the marker attached to the device 5 from the imaging data output from the imaging unit 22. As a result of this analysis, the viewpoint data creation unit 26 calculates the position and direction of the mobile terminal device 2 with respect to the device 5 with reference to the marker, and creates viewpoint data.

  Note that the viewpoint data creation unit 26 may create viewpoint data using a technique other than the AR technique. For example, the viewpoint data creation unit 26 measures the distance of the device 5 from the mobile terminal device 2 using a distance sensor, and detects the angle of the mobile terminal device 2 with respect to the device 5 using a geomagnetic sensor. Data may be created.

  The viewpoint data creation unit 26 stores the created viewpoint data in the storage unit 29 and instructs the control unit 25 to transmit it to the PC 1 as necessary. Thus, the viewpoint data is transmitted to the PC 1 by the transmission / reception unit 20. The viewpoint data transmitted to the PC 1 is stored in the storage unit 11.

  The designer creates instruction data by the instruction data creation unit 18 based on the received viewpoint data. The instruction data creation unit 18 first creates virtual data temporarily based on the viewpoint data, and creates instruction data suitable for the created virtual data, that is, can be superimposed on the virtual data.

  The instruction data creation unit 18 viewed the virtual stereoscopic image (virtual image) of the device 5 from the same direction as viewed from the viewpoint of the operator indicated by the viewpoint data based on the 3D model data and the texture data. For example, virtual data is created as shown in FIG. The instruction data creation unit 18 temporarily stores the virtual data of the virtual image created in this way in the storage unit 11 and reads it when creating the instruction data.

  In creating the instruction data, the instruction data creating unit 18 adds the instruction data to the virtual data read from the storage unit 11. Specifically, the instruction data creation unit 18 includes instruction image data and instruction content data respectively indicating instruction images and instruction contents that can be added, that is, superimposed on a virtual image displayed on the monitor 14 based on the virtual data. The instruction data including the display position data for specifying the display position to which the instruction image and the instruction content are added (overlapped) is created. Information for creating the instruction data is obtained when the designer inputs the information using the input device 15. The instruction content is a description by characters (such as text), but may be expressed by symbols other than characters. When the instruction content can be expressed by an image, the instruction image may also serve as the instruction content.

  The instruction data shown in FIG. 5 forms an instruction image in the form of a balloon message and includes instruction contents. The instruction data is added so as to indicate the part of the device 5 that the worker wants to see. Since the instruction image includes the instruction content, instruction image data as an instruction image that is expressed integrally with the instruction content is created.

  The instruction data creating unit 18 stores the instruction image data (including the instruction content data) created in this way and instruction data including display position data for specifying the position where the instruction image is displayed in the storage unit 11. . The instruction data includes the instruction image data and the coordinates (position information) for displaying the instruction image as display position data, so that the instruction data can be appropriately superimposed on the virtual data and the image displayed on the mobile terminal device 2. . That is, the instruction data does not include the virtual data described above.

  The instruction data is shared between the PC 1 (designer) and the portable terminal device 2 (worker) by being transmitted to and stored in the portable terminal device 2 (and the server 4 as necessary).

  In the portable terminal device 2, the instruction image display processing unit 27 is displayed in a state where the worker is imaging the device 5 based on the instruction data stored in the storage unit 29 (and the server 4 as necessary). The instruction panel indicated by the instruction data is displayed on the display panel 23 so as to overlap the actual image of the device 5 displayed on the panel 23. In the example shown in FIG. 6, the instruction content is included in the instruction image displayed by the above-described balloon message at the position determined by the display position data on the actual image of the device 5. The worker performs work according to such instruction content.

  As the instruction data, in addition to the data including the message shown in FIG. 5, for example, data indicating an imaging location and an imaging direction in which the worker images the apparatus 5 is indicated. In the example shown in FIG. 7, the shooting location is indicated by an elliptical broken line on the virtual image of the device 5. In this example, if it is a precondition that the designer and the worker agree that the area indicated by the broken line indicates the shooting location, the broken instruction image indicates the instruction content. Can also be said. Therefore, as the instruction data, instruction image data of a broken line instruction image and display position data for determining the display position of the instruction image are created.

  In the example shown in FIG. 8, the position of the worker on the virtual image of the apparatus 5 and the shooting direction are indicated by a face picture and an arrow, respectively. In this example, assuming that there is an agreement between the designer and the worker that the face picture represents the worker's position and that the arrow represents the shooting direction, the face It can be said that the picture and the arrow instruction image also express the instruction content. Therefore, as the instruction data, instruction image data of a face picture and an arrow instruction image and display position data for determining the display position of the instruction image are created.

  The instruction data creating unit 18 creates such instruction data and provides it to the mobile terminal device 2, so that the mobile terminal device 2 does not display the balloon message on the actual image as shown in FIG. An instruction image indicating the shooting location and shooting direction is displayed. The designer and the worker talk with each other via the call unit 13 of the PC 1 and the call unit 21 of the mobile terminal device 2 while referring to the shared shooting instruction as described above, and are displayed in the instruction image. Check the instructions.

  In order to give an instruction to the worker based on the instruction data, it is desirable for the designer and the worker to match the viewpoints, but the viewpoints of both do not necessarily match.

  In addition, the instruction data creation unit 18 creates virtual data and saves it in the storage unit 11, but the virtual data need not be created and saved. For example, based on the 3D model data and the texture data, the instruction data creation unit 18 calculates the position of the device 5 as seen from the same direction as seen from the viewpoint of the worker indicated by the viewpoint data. Based on the above, a three-dimensional area (space) in which the device 5 is arranged may be created as data representing a cube or a vertex of the cube. If the cube and the vertex are displayed based on this data, at least the direction in which the operator is looking at the device 5 can be expressed by a simple process. The virtual data also represents the three-dimensional arrangement area of the device 5.

  Next, the operation at the time of updating the 3D model data of the information sharing system 100 will be described.

  FIGS. 9A to 9C are diagrams obtained by adding the shooting location and shooting direction of the apparatus to the virtual image. On the mobile terminal device 2, a real image is used instead of a virtual image.

  In a state where the actual shape of the device 5 changes (deformation, breakage, etc.), the 3D model indicated by the 3D model data is different from the actual device 5 and thus it is necessary to update (correct) the 3D model data. In order to update the 3D model data, an image of the device 5 taken from multiple viewpoints is necessary. Although it is possible to shoot images from multiple points by the instruction method as shown in FIGS. 5, 7, and 8, a moving image shooting instruction with movement and direction using animation is created as an instruction image. be able to.

  For this reason, the instruction data creation unit 18 creates instruction data including instruction contents for photographing the device 5 from multiple viewpoints based on information input by the designer. For example, in the example illustrated in FIG. 9, the upper center of the device 5 is photographed from the front of the device 5 (FIG. 9A), and the upper portion of the device 5 is photographed from the right side of the device 5 ( b)), an instruction to photograph the upper part of the apparatus 5 from the right rear side of the apparatus 5 ((c) in FIG. 9) is indicated by a continuous movement (instruction image). In this example, an instruction to shoot a moving image is shown, but (a) to (c) in FIG. 9 may be combined into one instruction image (still image). Thereby, it is possible to give an instruction to the operator to shoot the device 5 from the three directions shown in FIGS.

  In addition to the above instruction image, a message explaining an instruction to shoot the device 5 and the type of image to be shot (still image or moving image) may be added. This photographing instruction and message may be added as a balloon message (instruction image) as shown in FIG. 6, or may be added as instruction contents shown in text. Thereby, the voice instruction from the designer to the worker can be omitted.

  When such an instruction image is displayed on the display panel 23 by the instruction image display processing unit 27, the worker takes an image of the apparatus 5 in accordance with the instruction. Shooting data obtained as a result is temporarily stored in the storage unit 29.

  The difference data creation unit 28 creates 3D model data (comparative appearance data) and texture data (comparative appearance data) for comparison using a known modeling technique based on a moving image or a multi-viewpoint still image. First, the case of 3D model data (comparative appearance data) will be described. Further, the difference data creation unit 28 compares the 3D model data with the shared 3D model data to extract the difference (changed part) between the two, and based on the extracted part, the difference data Create Further, the difference data creation unit 28 (first update unit) updates the 3D model data stored in the storage unit 29 so as to reflect the changed portion based on the created difference data. Then, the difference data creation unit 28 stores the created difference data in the storage unit 29 and instructs the control unit 25 to transmit the stored difference data to the PC 1 as necessary. Thereby, the difference data is transmitted to the PC 1 by the transmission / reception unit 20. The difference data transmitted to the PC 1 may be stored in the storage unit 11, but is not necessarily stored.

  The update processing unit 19 updates the 3D model data so as to reflect the changed portion indicated by the difference data, and stores the updated 3D model data in the storage unit 11 of the PC 1. In this way, the updated 3D model data is shared between the designer and the worker.

  Subsequently, an operation at the time of updating the texture data of the information sharing system 100 will be described.

  FIG. 10A is a diagram showing texture data shared by the designer and the worker, and FIG. 10B shows a texture created based on an image taken by the portable terminal device. FIG. FIGS. 10C and 10D are diagrams showing the difference between the textures of FIGS. 10A and 10B, respectively. FIG. 10E is updated based on the difference. FIG.

  Next, the case of texture data will be described. The difference data creation unit 28 creates texture data for comparison based on the shooting data (still image) in the storage unit 11. Further, the difference data creation unit 28 compares the texture data with the shared texture data to extract a difference (change part) between the two, and creates difference data based on the extracted part. To do. Moreover, the difference data creation unit 28 updates the texture data stored in the storage unit 29 so as to reflect the changed portion based on the created difference data. Then, similarly to the difference data of the 3D model data, the difference data creation unit 28 stores the created difference data in the storage unit 29 and transmits the stored difference data to the PC 1 as necessary. Instruct. Thereby, the difference data may be transmitted to the PC 1 and stored in the storage unit 11, but it is not necessarily stored.

  The update processing unit 19 updates the texture data so as to reflect the changed portion indicated by the difference data, and stores it in the storage unit 11 of the PC 1. In this way, the updated texture data is shared between the designer and the worker.

  For example, as a result of comparison between the shared texture (texture data) shown in FIG. 10A and the comparative texture (texture data) shown in FIG. 10B, (c) in FIG. ) And difference data of the changed portion as shown in (d) are created and provided from the mobile terminal device 2 to the PC 1. The update processing unit 19 of the PC 1 creates new texture data reflecting the difference data like the texture shown in FIG.

  Note that the data for updating the 3D model data and texture data is not necessarily difference data. For example, when the above-described comparison texture data is transmitted to the PC 1 based on an instruction from the difference data creation unit 28 to the control unit 25, the update processing unit 19 compares the texture data stored in the storage unit 11. Update based on the difference from the texture data for use.

  Finally, transmission processing by the PC 1 of the updated 3D model data and texture data will be described.

  FIG. 11 is a diagram showing the 3D model in a partially updated state, that is, a state in the middle of a progressive update described later. FIG. 12 is a diagram showing the 3D model in the last state in which all of the data has been updated, that is, in a progressive update. FIG. 13 is a diagram illustrating a texture partially updated, that is, a texture in the middle of a progressive update. FIG. 14 is a diagram showing the texture in the final state in which all the data has been updated, that is, in the progressive update.

  If the update result of the 3D model data and the texture data of the device 5 is not shared between the designer and the worker, it is not possible to start the cooperative work between the two based on the 3D model data and the texture data. Further, even if the worker requests the designer to update the 3D model data and the texture data, until the transmission of data (including difference data) for updating from the mobile terminal device 2 to the PC 1 is completed, the PC 1 The update process cannot be started.

  On the other hand, for example, the transmitter / receiver 20 creates outline vertex data, outline data, and internal line data for the 3D model data, and transmits them to the mobile terminal device 2 in this order. A plurality of pieces of divided data divided so as to have different densities are transmitted from the portable terminal device 2 to the PC 1 in order from sparse divided data to dense divided data, so that the PC 1 changes from a sparse state to a dense state. Thus, the 3D model data is updated progressively.

  Specifically, first, as shown in FIG. 11, the outline of the 3D model is updated by the vertex data and the outline data, and the 3D including the inside is updated by the internal line data as shown in FIG. All of the models are updated.

  In addition, the transmission / reception unit 20 creates a plurality of divided data of different frequency band components for the texture data, and transmits the divided data to the mobile terminal device 2 in order of increasing the frequency band of the divided data from the low frequency band. . As a result, when the divided data having different densities are transmitted sequentially (progressively) to the PC 1, the texture data is updated in the PC 1 so as to change from sparse to dense. Specifically, first, the sparse overall image of the texture is updated by the low frequency component data as shown in FIG. 13, and the texture of the texture is updated by the higher frequency band component data as shown in FIG. A dense picture is updated. When only the difference data is transmitted for the 3D model data and the texture data, the transmission / reception unit 20 outputs only the low frequency component of the difference portion.

  In this way, update data and differential data are transmitted progressively, so that designers and workers who have received each data can complete the data transmission before the mutual data sharing is completed. You can start the next task.

  As described above, in the information sharing system 100 according to the present embodiment, the PC 1 includes the instruction data creation unit 18 and the update processing unit 19, and the mobile terminal device 2 includes the transmission / reception unit 20, the viewpoint data creation unit 26, and the instruction. An image display processing unit 27 and a difference data creation unit 28 are provided.

  Thereby, the designer can create instruction data from the viewpoint of the worker based on the viewpoint data received from the worker. Therefore, the designer can transmit the instructions to the worker more clearly (intuitively) by using the instruction data as shared information, and can smoothly meet with the worker.

  In addition, by transmitting difference data from the worker to the designer for updating the shared data (3D model data and texture data), the data size is smaller than when the photographed data photographed by the worker is transmitted as it is. can do. The designer can update the shared data based on the difference data.

  Further, by using the 3D model data as shared data, it is possible to update an apparent change in the external shape (deformation, damage, etc.) of the device 5. Based on the updated 3D model data, the entire apparatus 5 can be confirmed. In addition, changes on the surface of the device 5 (such as dirt) can be updated by using the texture data as shared data. Based on the updated texture data, the details of the device 5 can be confirmed.

  Then, transmission of data (including difference data) for updating the shared data from the mobile terminal device 2 is progressively performed from sparse to dense, thereby completing transmission completely (update is completed). By the time, the contents of the data can be estimated from the received data. Thereby, the work efficiency of a designer and a worker can be improved. Also, since it is possible to avoid transmitting large data at a time, even if the network 3 is of low quality, it is possible to transmit data without squeezing the line.

  In the above embodiment, the appearance data (3D model data and texture data) is stored in both the PC 1 and the portable terminal device 2, but the appearance data is stored only in the PC 1, and the portable terminal device 2 is stored. So you can always use real images.

[Example of software implementation]
The data creation unit 10 of the PC 1 and the transmission / reception unit 20 of the mobile terminal device 2, the viewpoint data creation unit 26, the instruction image display processing unit 27, and the difference data creation unit 28 are logic circuits (IC chips) or the like formed in an integrated circuit (IC chip) or the like. Hardware), or software using a CPU.

  In the latter case, the PC 1 and the portable terminal device 2 include a CPU that executes instructions of a program that realizes each function, a ROM or a storage device (recording medium) in which the program and various data are recorded so as to be readable by the CPU, and the program RAM and the like are provided. Then, the CPU reads the program from the recording medium and executes it. As the recording medium, a “non-temporary tangible medium” can be used. Further, the program may be supplied to the PC 1 and the mobile terminal device 2 via any transmission medium (such as a communication network) capable of transmitting the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

  In particular, a program that realizes the viewpoint data creation unit 26, the instruction image display processing unit 27, and the difference data creation unit 28 is installed in the mobile terminal device 2 such as a tablet terminal. Therefore, these programs may be made available from general sites that provide application programs.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  INDUSTRIAL APPLICABILITY The present invention can be suitably used for applications in which cooperation between an apparatus designer and an operator in charge of apparatus maintenance is efficiently performed.

1 PC (data creation device)
2 Mobile terminal devices (equipment)
4 Server (equipment)
5 Equipment (monitoring target)
11 Storage unit (second storage unit)
18 Instruction data creation unit (instruction data creation means)
19 Update processing unit (second update means)
20 Transmitter / receiver (transmission processing means)
22 Imaging unit 26 Viewpoint data creation unit (viewpoint data creation means)
27 Instruction image display processing unit (instruction image display processing means)
28 Difference data creation unit (difference data creation means, first update means)
29 storage unit (first storage unit)
100 Information sharing system

Claims (3)

An imaging unit for imaging a monitoring target;
A portable terminal device having viewpoint data creating means for creating viewpoint data indicating the position and direction of the portable terminal device with respect to the monitoring target based on the actual image of the monitoring target imaged by the imaging unit;
Based on instruction image data of an instruction image indicating an instruction to be given to a worker who uses the mobile terminal device, and appearance data representing the appearance of the monitoring target in order to display the instruction image on the actual image. A data creation device having instruction data creation means for creating instruction data including the display position data specified by
The portable terminal device performs an instruction to perform a process of superimposing and displaying the instruction image indicated by the instruction image data at a display position specified by the display position data on the actual image captured by the imaging unit. An information sharing system further comprising image display processing means. The portable terminal device
A first storage unit for storing the appearance data;
Creating comparison appearance data for comparison based on the image data of the actual image, and creating difference data that is a difference between the comparison appearance data and the appearance data;
First update means for updating the appearance data stored in the first storage unit based on the difference data;
The data creation device includes:
A second storage unit for storing the appearance data;
2. The information sharing system according to claim 1, further comprising: a second update unit configured to update the appearance data stored in the second storage unit based on the difference data. The portable terminal device
A transmission process for dividing the updated appearance data into a plurality of pieces of divided data having different densities, and sequentially sending the divided data from the sparse divided data to the dense divided data to a device that stores the appearance data. The information sharing system according to claim 2, further comprising means.