WO2018193880A1

WO2018193880A1 - Plant equipment recognition system and plant equipment recognition method

Info

Publication number: WO2018193880A1
Application number: PCT/JP2018/014742
Authority: WO
Inventors: 村田　幸弘; 洋関; 洋平杉本
Original assignee: 日立Ｇｅニュークリア・エナジー株式会社
Priority date: 2017-04-21
Filing date: 2018-04-06
Publication date: 2018-10-25

Abstract

The purpose of the present invention is to provide a plant equipment recognition system capable of accurately recognizing plant equipment even in a plant wherein wireless equipment or markers cannot be installed, or where the equipment differs in form from the design data or the form thereof has been modified from the design data. Provided is a plant equipment recognition system for recognizing, using mobile terminals provided with cameras, plant equipment to be used , said system being configured to comprise: a feature value acquisition unit for acquiring, as a first feature value, information relating to piping from images obtained by using the cameras to capture images of the plant equipment to be used and the piping present in the vicinity of said equipment; and a feature value comparison unit for comparing the first feature value with a second feature value relating to the piping and acquired from the design data.

Description

Plant equipment recognition system and plant equipment recognition method

The present invention relates to a plant equipment recognition system and a plant equipment recognition method.

In plant facilities such as nuclear power facilities, specifications and work details for each plant equipment to be worked are pre-determined when performing pre-operation tests and maintenance management after operation for plant equipment such as motors and valves. Knowing this is useful for improving work efficiency. To that end, it is necessary to recognize in advance what kind of plant equipment is to be worked on by recognizing individual installation positions of plant equipment and / or recognizing individual images. Become.

Conventionally, positioning technology using GPS (Global Positioning System), mobile phone base stations, wireless LAN base stations, and the like has been widely used for position recognition. As an image recognition technique, a self-position specifying method for identifying a current position using a marker is known. There is also known a technology for recognizing an object itself and recognizing the object by image matching with a model separately prepared as a comparison target. For example, using a camera mounted on a moving body, the feature amount in the image is analyzed, and the position of the moving body at each time point associated with the movement and the relative positional relationship of each feature portion are analyzed using the SFM method. There is a technique in which matching is performed by comparing feature quantities obtained by calculation (see, for example, Patent Document 1).

In Patent Literature 1, “a plurality of real images taken in the vicinity of a reference point from an in-vehicle camera are input, and a feature portion is extracted from them as first feature information. The second feature corresponding to the map DB. The feature information of the first and second feature information is compared and matched to estimate the position of each feature, and based on the position, the SFM method is used. The current position of the host vehicle is estimated by calculating the relative positional relationship between the position of the host vehicle at each point of time associated with the movement and the position of each of the features.

There is also a technique for recognizing an object by extracting a feature value from a captured image by using an image obtained by ortho-correcting a captured actual image and performing matching with three-dimensional coordinate data to be compared (for example, patents). Reference 2). Patent Document 2 states that “feature points are extracted for the current real image ortho-corrected by the camera and the three-dimensional coordinate data acquired from the server, and the extracted feature points are matched with each other. Is used to estimate the self-position of the mobile terminal. "

JP 2012-127896 A Japanese Unexamined Patent Publication No. 2016-170060

By the way, radio and typified by GPS cannot be used inside plant facilities such as nuclear facilities. Therefore, in the plant facility, it is not always possible to use a conventional self-localization method that often uses radio or a marker to recognize the plant facility device. Further, the plant equipment is often different in shape from CAD (Computer-Aided Design) data at the time of design or the shape is changed. For this reason, matching between the actual image of the plant equipment and the image based on the design data cannot be obtained, and the plant equipment cannot be accurately recognized.

The present invention is a plant facility capable of accurately recognizing a plant facility device even in a plant facility where a wireless facility or a marker cannot be installed, and even when the shape is different from the design data or the shape is changed. An object is to provide a device recognition system and a plant facility device recognition method.

In order to solve the above problems, for example, the configuration described in the claims is adopted.
The present application includes a plurality of means for solving the above problems.
In a plant equipment recognition system for recognizing plant equipment to be worked in plant equipment using a mobile terminal equipped with a camera,
A feature amount acquisition unit that acquires information about piping as a first feature amount from an image obtained by photographing the plant facility device to be worked and the piping existing around the plant facility device with a camera;
A feature amount comparison unit that compares the first feature amount and a second feature amount relating to the piping acquired from design data;
It is characterized by providing.

According to the present invention, even in a plant facility where a radio facility or a marker cannot be installed, and even when the shape is different from or changed from the design data (CAD data at the time of design), the plant facility device to be worked on Can be accurately recognized.

It is an example of the block diagram which shows the structure of the plant equipment apparatus recognition system which concerns on 1st Embodiment of this invention. It is an example of the figure which shows an example of the display screen displayed on the display part of a mobile terminal. It is an example of the flowchart which shows the flow of a process of the plant equipment apparatus recognition method which concerns on 1st Embodiment of this invention. It is an example of the figure which shows the example of a display of the display screen of a mobile terminal in the specific process of the plant facility equipment recognition method which concerns on 1st Embodiment (the 1). It is an example of the figure which shows the example of a display of the display screen of a mobile terminal in the specific process of the plant facility equipment recognition method which concerns on 1st Embodiment (the 2). It is an example of the schematic diagram which shows an example of piping containing the curved pipe part (elbow) connected to the valve | bulb. It is an example of the flowchart which shows the flow of the recognition process of the piping part in the plant equipment apparatus recognition system which concerns on 2nd Embodiment. It is an example of the figure which shows the example of a display of the confirmation screen in the recognition process of a piping part. It is an example of the flowchart which shows the flow of the recognition process of the valve | bulb part in the plant equipment apparatus recognition system which concerns on 2nd Embodiment. It is an example of the figure which shows the example of a display of the confirmation screen in the recognition process of a valve | bulb part. It is an example of the block diagram which shows the structure of the plant equipment apparatus recognition system which concerns on 3rd Embodiment of this invention.

Hereinafter, modes for carrying out the present invention (hereinafter referred to as “embodiments”) will be described in detail with reference to the drawings. The present invention is not limited to the embodiment. In the following description and each drawing, the same reference numerals are used for the same elements or elements having the same functions, and duplicate descriptions are omitted.

<About plant equipment>
In plant facilities such as nuclear facilities, plant facilities are usually connected with piping for the purpose of transporting fluids such as liquid, gas, and powder, and protecting electrical wiring and cables. Examples of plant facility equipment include equipment such as motors and valves. In the plant facility, a test is performed on each plant facility device before operation, and maintenance and management of each plant facility device is performed after operation.

In pre-operation testing and maintenance management after operation, specifications and work contents, etc., vary depending on the individual plant equipment, such as the type of plant equipment to be worked on and the installation position (installation location) of the same type of plant equipment. Is different. Therefore, for plant equipment such as motors and valves, in order to improve work efficiency when conducting pre-operation testing and maintenance management after operation, the specifications and work contents of each plant equipment are known in advance. It is preferable.

In order to know in advance the specifications and work contents of individual plant equipment, what kind of plant equipment is to be worked on by recognizing the individual installation positions of the plant equipment and recognizing individual images. It is necessary to recognize in advance whether it is a device. However, generally, a radio or a marker represented by GPS cannot be used inside a plant facility such as a nuclear facility. Further, the plant equipment is often different in shape from the CAD data at the time of design or the shape is changed.

In the following environment, that is, even in a plant facility where radio equipment and markers cannot be installed, and even if the shape is different from the CAD data at the time of design or the shape is changed, the plant to be worked on An embodiment of a system (method) capable of accurately recognizing facility equipment will be described.

<First Embodiment>
[Configuration of plant equipment recognition system]
FIG. 1 is an example of a block diagram illustrating a configuration of a plant facility equipment recognition system according to the first embodiment of the present invention. The plant equipment recognition system 1 according to the present embodiment includes a mobile terminal 10 and a comparison feature database (DB) creation unit 20, and is a work target for performing tests before operation and maintenance management after operation. Automatically recognizes plant equipment. The plant facility equipment to be worked is also the plant equipment to be recognized. In the present embodiment, the case where the plant equipment to be worked (recognized) is a valve to which a pipe for the purpose of transporting fluid such as liquid, gas, and powder is connected.

The mobile terminal 10 has a digital system constructed by a microcomputer (microcontroller) in which a processor, a memory, and the like are integrated, and includes an area specifying unit 11, a feature amount acquisition unit 12, an equipment information display control unit 13, and a feature amount comparison. Each function unit of the control unit 14 is included. Details of each functional unit of the mobile terminal 10 will be described later.

The comparison feature quantity database creation unit 20 includes a CAD data storage unit 21, an imaging data storage unit 22, and a feature quantity creation unit 23. The CAD data storage unit 21 stores and holds CAD data at the time of design (that is, design data). Examples of CAD data include three-dimensional design data such as the pipe diameter, pipe length, and distance from the floor or wall to the pipe in the plant equipment. The imaging data storage unit 22 stores and holds imaging data in the field by a camera 122 described later.

The feature quantity creation unit 23 creates (extracts) a comparative feature quantity for piping using each storage data of the CAD data storage unit 21 and the imaging data storage unit 22, that is, CAD data at the time of design and imaging data at the site. To do. Features for comparison include pipe diameter, pipe length, pipe height (distance from floor or wall to pipe), pipe orientation (vertical / horizontal), positional relationship between multiple pipes, and plant equipment Piping information etc. can be illustrated.

(mobile computer)
Below, the detail of each function part of the mobile terminal 10, ie, the area specific | specification part 11, the feature-value acquisition part 12, the equipment information display control part 13, and the feature-value comparison control part 14 is demonstrated.

The area specifying unit 11 includes a GPS 111, an electronic compass 112, a beacon 113, a WiFi (registered trademark) 114, a marker reading unit 115, and an area information collecting unit 116, and a plant facility device that is a work target (recognition target) is installed. Identify the recognition target area.

The GPS 111 acquires position information of the mobile terminal 10 using satellite communication. The electronic compass 112 is an example of an azimuth meter, and acquires azimuth information of the mobile terminal 10 by detecting geomagnetism. The beacon 113 acquires the position information of the mobile terminal 10 through communication with a plurality of beacon master devices installed in the recognition target area. WiFi (registered trademark) 114 wirelessly connects the mobile terminal 10 to a local area network (LAN). The marker reading unit 115 reads a plurality of markers pasted in the recognition target area in order to specify the position of the mobile terminal 10.

The area information collection unit 116 collects various information acquired by the electronic compass 112, the beacon 113, WiFi (registered trademark) 114, and the marker reading unit 115. Based on various information collected by the area information collection unit 116, the area identification unit 11 provides a rough overview of the mobile terminal 10 from the entrance of the plant building to the recognition target area where the recognition target plant equipment is installed. Get location information. The identification of the recognition target area by the area identification unit 11 is limited to an environment where radio waves can be used that are not electromagnetically shielded.

The feature quantity acquisition unit 12 includes a structure sensor 121, a camera (imaging device) 122, an electronic compass 123, a feature quantity calculation unit 124, and a feature quantity database 125, and images and point clouds obtained by photographing with the camera 122. Using the data, information regarding the piping is acquired as the first feature amount. The space in the recognition target area is usually an electromagnetically shielded space. Therefore, the feature quantity acquisition unit 12 enters the recognition target area, that is, the feature quantity related to the pipe connected to the plant equipment to be recognized in a situation where the shield is electromagnetically shielded and wireless cannot be used. To get.

The structure sensor 121 is an infrared camera, and acquires two-dimensional information of the distance from the mobile terminal 10 to the object to be photographed. The camera 122 includes a CCD (Charge-Coupled Device) image sensor, a CMOS (Complementary-Metal-Oxide-Semiconductor) image sensor, and the like, and the equipment in the recognition target area including the plant equipment to be recognized and piping connected thereto. Shoot. The electronic compass 123 acquires azimuth information of the mobile terminal 10 by detecting geomagnetism. The feature amount calculation unit 124 calculates a feature amount related to the captured pipe as the first feature amount based on information given from the structure sensor 121 and the camera 122. The feature value database 125 is a database of feature value values related to piping calculated by the feature value calculation unit 124.

The facility information display control unit 13 includes a facility information database 131, a three-dimensional facility database 132, an electronic compass 133, an AR (Augmented Reality) display unit 134, and a facility information display unit 135.

The facility information database 131 is a database of facility information including specifications for each plant facility device and work instruction content related to the plant facility device to be worked on by an operator. Present. This facility information database 131 can include data relating the positional relationship with piping and plant facility equipment such as structures included in plant facilities, for example, products such as mounts and embedded hardware, temporary materials such as scaffolds, and passages. . Thereby, the structure which exists in the vicinity of the mobile terminal 10 can be estimated.

The three-dimensional equipment database 132 is a database of data equivalent to the CAD data at the time of design stored in the CAD data storage unit 21, for example, three-dimensional data obtained by processing CAD data. The electronic compass 133 detects geomagnetism and acquires the orientation information of the mobile terminal 10.

The AR display unit 134 displays facility information such as work instruction contents of plant facility equipment using the AR function based on the information given from the three-dimensional facility database 132 and the electronic compass 133. The facility information display unit 135 displays facility information such as work instruction contents of the recognized plant facility equipment to be worked.

The feature quantity comparison control unit 14 includes a comparison feature quantity database 141 and a feature quantity comparison unit 142.

The comparison feature quantity database 141 is a feature quantity created by the feature quantity creation unit 23 of the comparison feature quantity database creation unit 20, that is, information about piping acquired from three-dimensional design data (CAD data at the time of design). This is databased as the second feature value (comparison feature value). The comparison feature quantity database 141 includes, as the second feature quantity, in addition to the pipe diameter, pipe length, and distance from the floor or wall to the pipe in the plant equipment, the positional relationship between the plant equipment and the pipe, Moreover, the positional relationship between the pipes in the plant facility can also be included.

The feature amount comparison unit 142 compares the first feature amount of the feature amount database 125 with the second feature amount of the comparison feature amount database 141, and has a feature amount that matches in the corresponding recognition target area. To list. In the comparison between the first feature value and the second feature value, the setting of a threshold value serving as a matching index can be adjusted on the mobile terminal 10. Since the comparison feature value database 141 includes information on pipes connected to the plant equipment, it is possible to indirectly estimate (specify) the target plant equipment by specifying a plurality of pipes. Can do.

In the present embodiment, the comparison feature quantity database 141 is implemented on the mobile terminal 10, but is not limited thereto, and a comparison feature quantity database creation terminal is provided separately from the mobile terminal 10. It can also be implemented by the comparison feature quantity database creation terminal.

(Display screen of mobile device)
Here, the display screen of the mobile terminal 10 will be described. FIG. 2 is an example of a diagram illustrating an example of a display screen displayed on the display unit of the mobile terminal 10.

The mobile terminal 10 displays an image captured by the camera 122 mounted on the mobile terminal 10 or a two-dimensional image (distance two-dimensional information) acquired by the structure sensor 121 on the image display screen 151 in the substantially central portion of the display unit 15. Is displayed. In this example, the image display screen 151 includes

pipes

1512 and 1513 provided across the valve 1511, pipes 1514 branched by the valve 1511, and pipes 1515 provided in parallel with the

pipes

1512 and 1513. The state displayed as an image is illustrated.

The mobile terminal 10 is often equipped with a touch panel (touch panel sensor) that combines a display device such as a liquid crystal panel or an organic EL panel and a position input device such as a touch pad as the display unit 15. In the display unit 15 including a touch panel, the operator can select a desired pipe by touching the pipe 1512/1513/1514/1515 on the image display screen 151 with a finger or the like.

The mobile terminal 10 can display the plant number “AAA” for identifying the plant facility and the area number “BBB” for identifying the recognition target area on the upper part of the display unit 15. Further, the mobile terminal 10 can display feature quantities such as the pipe diameter “aaa”, the pipe length “bbb”, and the pipe height “ccc” for the pipe selected by the operator. At this time, the feature quantities such as the pipe diameter, the pipe length, and the pipe height can be acquired from CAD data stored and held in advance in the CAD data storage unit 21 (see FIG. 1).

Note that the pipe height is limited to the case where the height from the floor of the pipe can be recognized. As a case where the height from the floor of the pipe can be recognized, a case where the pipe and the floor are photographed at the same time and two-dimensional information of the distance can be acquired by the structure sensor 121 can be exemplified.

In addition, the mobile terminal 10 displays a list of pipe candidates or plant equipment candidate candidates narrowed down by the feature amount comparison by the feature amount comparison unit 142 (see FIG. 1) on the upper right side of the image display screen 151 on the list display screen 152. Can be displayed as The mobile terminal 10 can further display the property of the selected pipe or plant equipment as a property screen 153 on the lower right side of the image display screen 151.

On the image display screen 151, a SCAN button 154 for instructing acquisition of a captured image by the camera 122 and acquisition of two-dimensional information of the distance from the mobile terminal 10 to the photographing object by the structure sensor 121 is displayed. be able to. The SCAN button 154 corresponds to a shutter of the imaging device (camera). When the operator presses the SCAN button 154, the captured image and the two-dimensional information of the distance can be acquired. The position where the SCAN button 154 is displayed is not limited to the display position shown in FIG.

(Image of floor, wall and pipe extraction)
With the structure sensor 121 shown in FIG. 1, two-dimensional information (two-dimensional image) of the distance from the mobile terminal 10 to the object to be imaged can be obtained. The structure sensor 121 can obtain the two-dimensional information of the distance by, for example, irradiating the imaging object with infrared rays and measuring the light reflected by the imaging object. Then, a plane or a curved surface can be extracted from the distance information obtained as the two-dimensional information according to the amount of change in distance. Thereby, a floor, a wall, and piping can be identified.

As an image processing method, for example, when a plane is obtained from two-dimensional information of distance, a method of extracting a point cloud of a plane portion using a PCL (Point Cloud Library) module and calculating a plane approximation formula can be used. The PCL module is a library that summarizes open source 3D point cloud data processing.

Also, as the distance from the wall (distance to the wall), the distance from the intersection line formed by two adjacent walls can be used. More specifically, the distance between the two lines is calculated by calculating two plane expressions and deriving an intersection line and extracting the start point and the end point of the pipe and deriving a linear expression. As cylinder pipe recognition, for example, in order to extract the pipe diameter, the distance between two boundary lines on the side surface of the cylinder is obtained. Since the cylinder side surface is a curved surface, it is possible to determine whether the two boundary lines are the side surfaces of the cylinder by considering the normal vector.

Also, as the distance from the pipe to the wall, it is possible to grasp the current position and the distance to the surrounding walls by imaging the walls existing in all directions. In many cases, the butt portion of the wall of the plant equipment is orthogonal, and the location in the recognition target area can be estimated from the distance to the surrounding wall.

[Plant equipment recognition method]
Next, a plant facility equipment recognition method according to the first embodiment of the present invention will be described. The plant equipment recognition method according to the present embodiment is controlled by a processor that constructs a digital system of the mobile terminal 10 in the mobile terminal 10 of the plant equipment recognition system 1 according to the first embodiment having the above-described configuration. To be implemented.

FIG. 3 is an example of a flowchart showing a processing flow of the plant facility equipment recognition method according to the first embodiment of the present invention. 4 and 5 are diagrams showing a display example (part 1) and a display example (part 2) of the display screen of the mobile terminal 10 in the specific process of the plant facility equipment recognition method according to the first embodiment. It is an example.

Prior to execution of processing of the plant facility equipment recognition method according to the first embodiment, the comparison feature database creation unit 20 uses CAD data at the time of design and imaging data at the work site (recognition target). It is assumed that a process for creating a comparison feature amount of piping connected to the plant equipment is performed. The comparison feature quantity of piping created by the comparison feature quantity database creation unit 20 is assumed to be databased as a comparison feature quantity database 141.

First, the processor uses the function of the area specifying unit 11 to set the initial position of the mobile terminal 10 that performs image recognition by feature amount matching (step S11). Specifically, the processor specifies and sets the initial position of the mobile terminal 10 using the function of the area specifying unit 11 that can be used for the purpose of specifying the position of GPS, WiFi, marker, beacon, and the like. For example, when the plant building is specified from the outside of the plant building, the position is specified outdoors, and therefore positioning (position specifying) using GPS is possible.

In the above example, the method of setting the initial position of the mobile terminal 10 using the function of the area specifying unit 11 under the control of the processor is illustrated. However, the method is not limited to this setting method. For example, when the plant number or the area number is known in advance, a method of setting the initial position of the mobile terminal 10 by manual input by an operator can be taken.

Next, under the operation of the operator, the processor photographs the plant equipment that is desired to be identified (recognized) and the piping connected to the equipment (step S12). Specifically, the processor captures the plant equipment and the piping existing in the vicinity thereof with the camera 122. FIG. 4A is an example in which the plant equipment to be specified and piping existing in the vicinity thereof are photographed by the camera 122 of the mobile terminal 10. The display screen in FIG. 4A corresponds to the display screen in FIG. A SCAN button 154 is also displayed on the image display screen 151.

Next, the processor determines whether or not the SCAN button 154 displayed on the image display screen 151 has been pressed by the operator (step S13). When the SCAN button 154 is pressed (Yes in S13), the camera The image information by 122 and the two-dimensional information of the distance by the structure sensor 121 are acquired (step S14).

Next, the processor recognizes the pipes that are reflected in the periphery of the recognition-target plant equipment that is the imaged work object, based on the image information acquired in step S14 and the two-dimensional information about the distance (step S15). In this recognition process, when a plurality of pipes are shown around the plant equipment to be recognized, the operator designates the pipe to be specified on the touch panel and repeatedly executes the recognition work.

FIG. 4B shows a display image when one pipe 1512 is selected. At this time, the selected pipe is displayed on the image display screen 151 with, for example, paint (shaded in FIG. 4B). Thereby, the worker can grasp at a glance which piping is being checked. The pipe diameter of the pipe 1512 is displayed on the upper part of the display unit 15 of the mobile terminal 10, and attribute information regarding the pipe 1512 is displayed on the lower right side of the display unit 15.

Next, the processor acquires the feature amount of the pipe designated by the operator from the captured image using the function of the feature amount acquisition unit 12 (step S16), and then calculates the feature amount calculated from the captured image, Matching with the comparison feature quantity in the comparison feature quantity database 141 acquired in advance is performed (step S17). Subsequently, the processor determines whether or not the matching result between the feature amount calculated from the captured image and the comparison feature amount matches (step S18). If the matching result does not match (No in S18), the processor returns to step S12. Steps S12 to S18 are repeated.

If the matching results match (Yes in S18), the processor displays a pipe candidate list corresponding to the current pipe diameter on the list display screen 152 on the right side of the image display screen 151 as shown in FIG. 5A ( Next, it is determined whether or not to continue recognition of other pipes (step S20). Whether or not to continue to recognize other piping can be determined based on whether or not the operator has selected other piping on the touch panel.

Here, if as many pipes as possible connected to the plant equipment are recognized, the amount of information to be narrowed increases, so that the possibility of uniquely identifying the plant equipment to be worked (recognized) increases. For this reason, if there is any unrecognized pipe existing around the plant equipment at this stage, the operator preferably selects the pipe again on the touch panel to obtain the pipe information.

When the operator selects another pipe on the touch panel, the processor determines that the recognition of the other pipe is continued (Yes in S20), returns to Step S16, and repeats the processes from Step S16 to Step S20. Further, when the operator does not select another pipe on the touch panel, the processor determines that the recognition of the other pipe is not continued (No in S20), and as shown in FIG. The equipment list (plant facility equipment candidate list) associated with the piping information is displayed (step S21).

Next, the processor determines whether or not the plant equipment that is the work target (recognition target) is uniquely identified (step S22). Here, if the processor is not uniquely identified (No in S22), the plant equipment is selected under the touch operation of the touch panel by the operator (Step S23), and then the selected plant equipment is selected. The device information is displayed (step S24).

In the device information display process, the property information of the plant equipment is displayed as the device information, and the related work list display process is performed. Then, when the worker touches and selects the corresponding work item, the work procedure manual of the plant equipment to be worked is displayed. If the plant equipment is uniquely identified (Yes in S22), the processor directly proceeds to step S24 and performs processing for displaying the equipment information of the identified plant equipment.

As described above, in the first embodiment, information relating to piping is obtained from the image obtained by photographing the piping existing around the plant equipment to be worked with the camera 122 mounted on the mobile terminal 10 as the first information. Acquired as a feature value. Then, the feature quantity comparison unit 142 compares the first feature quantity and the second feature quantity relating to the piping acquired from the three-dimensional design data. As a result, based on the comparison result of the feature value comparison unit 142, the piping near the plant equipment to be worked is identified, the plant equipment is narrowed down from the identified piping, and is recognized as the plant equipment to be worked (specification). / Estimation).

The recognition method according to the first embodiment is different from the method of recognizing the target plant equipment itself by the image matching technique. Therefore, according to the recognition method, the plant equipment can be accurately recognized even when the target plant equipment is different in shape from the three-dimensional design data or the shape is changed. Moreover, since the image obtained by photographing with the camera 122 equipped in the mobile terminal 10 is used, the plant facility device to be worked can be accurately recognized even in the plant facility where the wireless facility and the marker cannot be installed. .

Moreover, in 1st Embodiment, while displaying the information regarding piping on the facility information display part 135 based on the comparison result of the feature-value comparison part 142, the installation of the plant equipment apparatus of the work object narrowed down from the information regarding the said piping Information is displayed on the facility information display unit 135. Then, by displaying on the facility information display unit 135 the specifications of the plant equipment and the details of the work instructions as the equipment information of the plant equipment to be worked on, the worker can work according to the specifications and the contents of the work instructions. Since it only has to be carried out, it is possible to improve the work efficiency at the time of testing before operation and maintenance after operation.

In the first embodiment, since the feature amount obtained from the 3D design data to be compared is stored in the 3D equipment database 132 in a specific format, the data stored in the mobile terminal 10 Security can be increased.

Further, according to the first embodiment, it is possible to specify (estimate) the position information of the mobile terminal 10 by specifying a plurality of pipes or plant equipment devices existing in the vicinity of the mobile terminal 10. Therefore, based on the position information of the mobile terminal 10, information regarding structures existing around the mobile terminal 10 can be displayed on the facility information display unit 135. Examples of the structures existing in the periphery include products such as mounts and embedded hardware, temporary materials such as scaffolds, passages, and the like.

In addition, in an environment where an electronic compass 133, which is an example of a direction meter, can be used, the orientation of the mobile terminal 10 can be recognized. Therefore, an image captured by the camera 122 from the position information of the mobile terminal 10 and the orientation information can be obtained. Can be estimated. The AR display unit 134 can display the equipment information of the equipment information database 131 and then superimpose the 3D image of the 3D equipment database 132. When performing work for plant equipment, work errors can be reduced by displaying the work instruction content superimposed on the image captured by the camera 122 using the AR technology.

Second Embodiment
2nd Embodiment is a modification of 1st Embodiment, and is an example in case the piping connected to the valve which is an example of the plant equipment apparatus of work object (recognition object) contains a curved pipe part. The curved pipe portion of the pipe is an L-shaped pipe called an elbow, and is a connecting joint pipe that branches and merges the pipes by connecting the straight pipe portions. In-plant piping generally includes a plurality of straight pipe portions and a plurality of curved pipe portions, and is installed while bending the piping direction in a horizontal direction or a vertical direction. An example is shown in FIG.

FIG. 6 is an example of a schematic diagram illustrating an example of piping including a bent pipe portion (elbow) connected to a valve. In this example, a pipe structure in which three systems of

pipes

1516, 1517 and 1518 are connected to the valve 1511 is illustrated. Moreover, in this piping structure, the case where each piping diameter of three piping 1516, 1517, 1518 differs is illustrated. The pipe 1518 includes, for example, five straight pipe portions 1518_1, 1518_2, 1518_3, 1518_4, 1518_5 and four curved pipe portions 1518_6, 1518_7, 1518_8, 1518_9.

The curved pipe portion 1518_6 connects the straight pipe portion 1518_1 whose direction is horizontal and the straight pipe portion 1518_2 whose direction is vertical. The curved pipe part 1518_7 connects the straight pipe part 1518_2 whose direction is vertical and the straight pipe part 1518_3 whose direction is horizontal. The curved pipe portion 1518_8 connects the straight pipe portion 1518_3 whose direction is horizontal and the straight pipe portion 1518_4 whose direction is vertical. The curved pipe portion 1518_9 connects a straight pipe portion 1518_4 whose direction is vertical and a straight pipe portion 1518_5 whose direction is horizontal.

In FIG. 6, the pipe 1518 is illustrated as a pipe including a curved pipe portion (elbow), and the pipe 1518 is illustrated with respect to a pipe portion on the downstream side as viewed from the valve 1511, but the pipe 1516 and the pipe 1517 are also illustrated. Similar to the pipe 1518, there may be a case where one or a plurality of bent pipe parts are included.

The configuration of the plant equipment recognition system according to the second embodiment is basically the same as the configuration of the plant equipment recognition system according to the first embodiment shown in FIG. In the plant equipment recognition system according to the first embodiment, the three systems of

pipes

1512, 1513, and 1514 connected to the valve 1511 are photographed together by the camera 122, and together on the image display screen 151 of the display unit 15. It is displayed (see FIG. 2), and the operator selects a desired pipe on the image display screen 151.

However, if any of the plurality of pipes connected to the valve is hidden behind a temporary material such as a scaffold, for example, as shown in FIG. When the pipe portion is included, it is difficult to photograph all of the plurality of pipes connected to the valve together with the camera 122.

Therefore, in the second embodiment, a plurality of pipes connected to the valve are individually photographed (one by one) as shown in FIG. Then, every time one shot is taken, by performing the same specific processing as in the case where the operator selects a desired pipe on the image display screen 151 in the first embodiment, for each taken pipe, A pipe in the vicinity of a valve, which is an example of the plant equipment to be worked, is specified, and the valve is recognized (specified / estimated) from the specified pipe as the plant equipment to be worked.

In the second embodiment, when a pipe 1518 including one or a plurality of curved pipe portions is a target pipe for specific processing, individual straight pipe portions and curved pipe portions are set as matching targets, and pipe specific processing is performed. . That is, when the pipe to be specified includes a straight pipe portion and a curved pipe portion, not only the straight pipe portion 1518_1 directly connected to the valve 1511 but also the straight pipe portions 1518_2, 1518_3, 1518_4 on the downstream side thereof. 1518_5 and curved pipe portions 1518_6, 1518_7, 1518_8, and 1518_9 are also subject to matching.

In this pipe specific processing, the pipe diameter, pipe length, pipe height (distance from the floor or wall), pipe direction (vertical direction / horizontal direction) for each of the straight pipe portion and the curved pipe portion of the pipe 1518 Are stored in the comparison feature quantity database 141 as a comparison feature quantity (second feature quantity). In this case, the feature values are databased for both the straight pipe part and the curved pipe part. However, since the pipe diameter and shape are generally constant, the straight pipe part is generally used for the curved pipe part. It is also possible to make the feature quantity into a database only for the above.

Specifically, in the second embodiment, the camera 122 mounted on the mobile terminal 10 performs shooting one by one for the

pipes

1516, 1517, and 1518 connected to the valve 1511. Taking the case of the pipe 1518 as an example, as in the case of the first embodiment, the feature amount acquisition unit 12 acquires the feature amounts of the straight pipe portions 1518_1, 1518_2, 1518_3, 1518_4, and 1518_5, and the feature amount database. Stored in 125. Then, in the feature quantity comparison unit 142, each of the plurality of straight pipe sections 1518_1, 1518_2, 1518_3, 1518_4, and 1518_5 is checked against the comparison feature quantity database 141, and the matching pipe candidates are listed to display equipment information. Displayed on the part 135.

As for the first feature amount, when the pipe connected to the valve 1511 includes one or a plurality of curved pipe parts, the straight pipe part and the curved pipe part are used by using the CAD data in the CAD data storage unit 21. And to separate. As in the case of the first embodiment, the feature amount creation unit uses the CAD data in the CAD data storage unit 21 for the first feature amount related to the plurality of straight pipe portions 1518_1, 1518_2, 1518_3, 1518_4, and 1518_5. And is stored in the comparison feature database 141.

As described above, in the second embodiment, when a pipe to be specified is a pipe including one or a plurality of curved pipe portions, each straight pipe portion and the curved pipe portion are set as matching targets, and the pipe is specified. It is characterized by performing processing. As described above, not only the straight pipe portion 1518_1 directly connected to the valve 1511 but also the straight pipe portion and the curved pipe portion on the downstream side thereof are set as matching targets, so that only the straight pipe portion 1518_1 is set as a matching target. The recognition rate of the pipe 1518 can be increased as compared with the case.

Hereinafter, in the plant equipment recognition system according to the second embodiment, a process of recognizing a pipe part by a matching process of each straight pipe part and recognizing a valve part from the pipe part will be described with reference to FIGS. 7 to 10. explain.

FIG. 7 is an example of a flowchart showing the flow of the pipe recognition process in the plant equipment recognition system according to the second embodiment. FIG. 8A and FIG. 8B are examples of diagrams showing display examples of confirmation screens in the piping unit recognition processing. FIG. 9 is an example of a flowchart showing a flow of valve unit recognition processing in the plant equipment recognition system according to the second embodiment. FIG. 10 is an example of a diagram illustrating a display example of a confirmation screen in the valve unit recognition process.

The piping unit recognition process and the valve unit recognition process in the plant equipment recognition system according to the second embodiment are the mobile terminals of the plant equipment recognition system 1 according to the first embodiment and the second embodiment shown in FIG. 10 is performed under the control of the processor that constructs the digital system of the mobile terminal 10.

In the following, for example, when the pipe 1518 in FIG. 6 is the target of recognition processing, the straight pipe portion 1518_1 directly connected to the valve 1511 is referred to as pipe # 1, and the straight pipe that follows the pipe # 1. The parts 1518_2, 1518_3,... Will be referred to as sequence piping # 1sq1, # 1sq2,.

(Pipe recognition process)
Initially, the flow of the recognition process of a piping part is demonstrated using the flowchart of FIG. When the operator takes an image of the pipe # 1, the processor captures an image of the pipe # 1 (step S31), and then displays the confirmation screen 1 illustrated in FIG. 8A on the display unit 15 of the mobile terminal 10, for example. (Step S32).

In the confirmation screen 1 shown in FIG. 8A, “check point” indicating whether or not to enter the narrowing condition for recognizing the pipe # 1 with respect to the pipe diameter, pipe length, pipe height, and pipe direction (horizontal direction H / vertical direction V). A check is performed by the operator. In this example, the pipe diameter, the pipe length, and the horizontal direction H to which the “check” check is added are put into the narrowing condition by the operator.

Next, the processor determines whether or not the pipe # 1 has been recognized based on the narrowing-down condition checked by the operator as “Check” (step S33). This recognition process can be executed under the recognition process in the first embodiment. If the pipe # 1 is a characteristic pipe such as a pipe having a particularly large pipe diameter, the pipe # 1 can be recognized by the determination process in step S33. Here, the fact that the pipe # 1 has been recognized means that the pipe 1518 connected to the valve 1511 has been recognized.

If the pipe # 1, i.e., the pipe 1518 cannot be recognized in the process of step S33, the sequence pipes # 1sq1, # 1sq2,... Connected to the pipe # 1 are sequentially photographed by the operator. become. If the pipe # 1 cannot be recognized (No in S33), the processor determines whether or not the sequence piping has been shot (step S34). If the sequence piping has been shot (S34). Yes), an image of sequence piping # 1sq1 is captured (step S35), and then the confirmation screen 2 shown in FIG. 8B is displayed (step S36).

Next, the processor determines whether or not the imaging of the sequence piping has been completed (step S37), and if it has not been completed (No in S37), the process returns to step S35 and repeats the processes of steps S35 and S36. The confirmation screen 2 shown in FIG. 8B shows a display example in which the pipe diameter, pipe length, pipe height, and pipe direction (horizontal direction H / vertical direction V) up to the sequence pipe # 1 sq2 (straight pipe portion 1518_3) are displayed. ing. As in the case of the confirmation screen 1, the confirmation screen 2 also recognizes the sequence pipes # 1sq1 and # 1sq2 with respect to the pipe diameter, pipe length, pipe height, and pipe direction (horizontal direction H / vertical direction V). The operator performs a “check” check as to whether or not to enter the narrowing-down conditions.

Next, the processor saves the check results on the confirmation screens 1 and 2 in a file with a different file name, and updates the candidate for the valve 1511 (step S38). If it is determined in step S33 that pipe # 1 has been recognized (Yes in S33), or if it is determined in step S34 that sequence piping has not been photographed (No in S34), the process directly proceeds to step S38.

Next, the processor determines whether or not the number of pipes connected to the valve 1511 is one (step S39). If it is not one (No in S39), photographing of all the pipes connected to the valve 1511 is completed. Whether or not (step S40). If the imaging of all the connected pipes has not been completed (No in S40), the processor returns to step S31 and repeatedly executes the above-described series of piping unit recognition processes. In addition, when the number of valve candidates related to the valve 1511 is narrowed down to one (Yes in S39), or when all of the connected pipes have been photographed (Yes in S40), the processor performs the above-described series of piping unit recognition processing. finish.

(Valve recognition process)
Next, the flow of valve unit recognition processing will be described with reference to the flowchart of FIG. The processor takes in the number of pipe connections of the valve 1511 input by the operator (step S41). In this example, as is clear from FIG. 6, the number of pipe connections = 3 is input by the operator.

Next, the processor selects a desired file, for example, the latest file from the result files updated in step S38 (step S42), and then displays the confirmation screen 3 (step S43). As shown in FIG. 10, the confirmation screen 3 displays a plurality of similar candidates for the pipe connected to the valve 1511 as a result file. In this example, two candidates # 1_1 and # 1_2 are displayed for the result file of pipe # 1, and two candidates # 2_1 and # 2_2 are displayed for the result file of pipe # 2. There is one candidate for the result file of pipe # 3.

On the confirmation screen 3, the operator can perform a “check” check as to whether or not to add the contents of the result file to the list output condition. As a result, the worker corrects the data of the result file contents by the “check” check. That is, the worker confirms the narrowing conditions on the confirmation screen 3.

Next, the processor updates the file with respect to the result file contents subjected to the data correction (step S44), and then outputs the narrowing result as a valve list (plant facility equipment candidate list) (step S45).

As described above, when the pipe to be subjected to the specific processing is a pipe including one or a plurality of curved pipe portions, not only the straight pipe portion 1518_1 directly connected to the valve 1511 but also the straight pipe portion on the downstream side thereof. As a matching target, the recognition rate of the pipe 1518 can be increased. As a result, the recognition rate of a valve that is an example of plant equipment can be increased.

In the first embodiment and the second embodiment, pipe information such as pipe diameter, pipe length, pipe height, pipe direction (horizontal direction H / vertical direction V) is used as the second feature quantity that is a comparative feature quantity. However, the present invention is not limited to these pieces of tube information. For example, identification information such as a line number assigned to each pipe (straight pipe portion / curved pipe portion) for specifying the pipe can be used as the second feature amount. In this case, identification information such as a line number may be read using a well-known character recognition technique, and piping may be specified from the read result. In the case of the second embodiment, information such as the number of straight pipe portions and curved pipe portions may be used as the second feature amount.

<Third Embodiment>
The third embodiment is an example in which the plant equipment recognition system according to the first embodiment or the plant equipment recognition system according to the second embodiment is used as a navigation system that does not use radio. A plant facility equipment recognition system according to the third embodiment will be described with reference to FIG. FIG. 11 is an example of a block diagram showing the configuration of a plant equipment recognition system according to the third embodiment of the present invention.

In the plant facility equipment recognition system 1 according to the first embodiment or the second embodiment, the plant facility equipment to be worked is narrowed down by the action of the feature quantity acquisition unit 12 and the feature quantity comparison control unit 14. Then, the facility information display control unit 13 acquires facility information including the work instruction content of the plant facility equipment to be worked from the facility information database 131, and the facility information display unit 135 displays the facility information.

That is, according to the plant facility equipment recognition system 1 according to the first embodiment or the second embodiment, since the installation positions of the piping and plant equipment are known, the current position of the mobile terminal 10 can be specified. Therefore, for example, a worker wears a wearable device that is an example of the mobile terminal 10 and moves while specifying surrounding piping from the time of entering the area (plant facility) where the plant facility device is installed. Thus, information on the movement route from the area entrance to the current position can be acquired.

Thus, the plant equipment recognition system 1 according to the third embodiment includes a function unit of the mobile terminal 10 in addition to the area specifying unit 11, the feature amount acquisition unit 12, the facility information display control unit 13, and the feature amount comparison control unit 14. As one of them, a movement route presentation unit 16 that presents a movement route from the area entrance of the mobile terminal 10 to the current position is provided.

The movement route presentation unit 16 includes a map information database 161, a current position display unit 162, and a movement route display unit 163. The map information database 161 is a database of map information in an area (plant facility) where plant equipment is installed, specifically, information such as installation positions and passages of piping and plant equipment. The current position display unit 162 and the movement route display unit 163 are display units of the mobile terminal 10 and display the current position and movement route of the mobile terminal 10.

As described above, according to the third embodiment, the current position of the mobile terminal 10 can be determined by using the technology of the first embodiment that can identify the current position of the mobile terminal 10 from the installation position of the piping and plant equipment. And a moving route from the area entrance to the current position can be displayed. In addition, the movement route presentation unit 16 can store and hold the movement route from the area entrance to the current position.

Then, by updating and grasping the current position of the mobile terminal 10 in real time, the plant facility equipment recognition system 1 according to the third embodiment can be used as a mobile navigation system that does not use radio. In other words, the plant facility equipment recognition system 1 according to the third embodiment can have a mobile navigation function.

Here, for example, if there is a place that cannot be moved, for example, during construction or because of a material storage place, the place will be moved around that place. Therefore, the movement route presentation unit 16 has a function of storing and holding the movement route of the mobile terminal 10, estimating a place that cannot be passed from the movement route that is stored and holding, and presenting a movement route that avoids the place. be able to. As a result, it is possible to estimate a place that cannot be passed during the next movement and to present a movement route that avoids the place.

[Modification]
The present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to one having all the configurations described. For example, each of the functional units shown in FIGS. 1 and 11 may be realized by hardware, for example, by designing with an integrated circuit, in addition to the case where they are realized by software.

Further, in the above-described embodiment, as a plant facility device, a valve to which piping is connected has been described as an example, but the present invention is not limited to application to a valve, and a motor to which piping is connected, etc. It can be applied to general equipment with pipes connected.

DESCRIPTION OF SYMBOLS 1 Plant equipment apparatus recognition system, 10 Mobile terminal, 11 Area specification part, 12 Feature quantity acquisition part, 13 Equipment information display control part, 14 Feature quantity comparison control part, 15 Mobile terminal display part, 16 Travel route presentation part, 20 Comparison feature database creation unit, 21 CAD data storage unit, 22 imaging data storage unit, 23 feature creation unit, 1511 valve, 1512 to 1518 piping, 1518_1 to 1518_5 straight pipe unit, 1518_6, 1518_7, 1518_8, 1518_9 curved pipe Part

Claims

A plant equipment recognition system for recognizing plant equipment to be worked in plant equipment using a mobile terminal equipped with a camera,
From the image obtained by photographing the plant facility equipment to be worked and the piping existing around the plant equipment with the camera, a feature amount acquisition unit that acquires information about the piping as a first feature amount;
A feature amount comparison unit that compares the first feature amount with a second feature amount relating to the piping acquired from design data;
The plant equipment recognition system characterized by comprising.
A facility information display unit that displays information related to the piping based on the comparison result of the feature amount comparison unit and facility information of the plant facility equipment that is the work target that is narrowed down from the information related to the piping. The plant equipment recognition system according to claim 1.
The plant facility equipment recognition system according to claim 2, wherein the facility information includes a work instruction content related to the plant equipment to be worked.
The plant facility equipment recognition system according to claim 1, wherein the second feature amount includes a database including a pipe diameter, a pipe length, and a distance from a floor or a wall to the pipe in the plant equipment. .
5. The plant facility equipment recognition system according to claim 4, wherein the database includes a positional relationship between the work target plant equipment and the pipe as the second feature amount.
The plant database recognizing system according to claim 4, wherein the database includes, as the second feature amount, a positional relationship between pipes in the plant facility.
The pipe includes a straight pipe part and a curved pipe part,
2. The database according to claim 1, wherein the second feature value includes a database including a pipe diameter, a pipe length, and a distance from a floor or a wall to the straight pipe part. Plant equipment recognition system.
From the position information of the mobile terminal obtained by specifying the piping in the plant equipment and the plant equipment and the orientation information of the mobile terminal by the orientation meter installed in the mobile terminal, an image taken by the camera is taken. The plant facility equipment recognition system according to claim 2, further comprising an AR display unit configured to estimate and superimpose and display a three-dimensional image of the plant equipment on the image.
The facility information display unit displays information on structures existing around the mobile terminal based on position information of the mobile terminal obtained by specifying piping and plant equipment in the plant facility. The plant equipment recognition system according to claim 2 characterized by things.
A moving path presenting unit that presents the moving path of the mobile terminal by updating the position information of the mobile terminal obtained in real time by specifying piping and plant equipment in the plant facility is provided. The plant facility equipment recognition system according to claim 9.
The travel route presenting unit stores and holds a travel route of the mobile terminal, estimates a place where the mobile terminal cannot pass from the stored travel route, and presents a travel route that avoids the place. Item 11. The plant equipment recognition system according to Item 10.
A plant equipment device recognition method for recognizing a plant equipment device to be worked in a plant equipment using a mobile terminal equipped with a camera,
From the image obtained by photographing the plant facility equipment to be worked and the pipe existing around the plant equipment with the camera, obtaining information about the pipe as a first feature amount;
Comparing the first feature quantity with a second feature quantity relating to the piping obtained from design data;
The plant equipment recognition method characterized by performing the process of.