JP2001239983A - Outfit assisting system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an outfit assisting system capable of efficiently mounting equipment. SOLUTION: A position and posture measuring device 2 measures the position and posture data of a worker to indicate the position and posture of the worker. A watch data generating part 31 generates watch data to indicate wherefrom and whereto the worker is looking. A display data segmenting part 33 segments the image data of an equipment mounted in an area within the visibility range of the worker in a body K as display data from three dimensional drawing data memorized in a three dimensional CAD device 1. An image projecting device 4 mounted to the head part of the worker displays the visual reality image of the equipment indicated by the display data, and transmits the reality image of the body K. It looks to the worker as if the equipment as the virtual reality image were attached to the body K as the reality image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an outfitting support system used when various outfittings are attached to a frame or the like.

[0002]

2. Description of the Related Art Generally, a large ship has a structure in which a plurality of blocks are connected to each other. That is, such a ship is constructed by connecting the blocks manufactured in advance in the factory respectively to each other in the dock.

[0003] For example, a cabin block constituting a cabin is
It is manufactured by attaching various fittings such as air ducts, pipes, and cables to a skeleton having a three-dimensional structure. Conventionally, in order to manufacture such a cabin block, a process called marking has been performed in advance.

[0004] This marking is performed by an operator with reference to the design drawing of the cabin. That is, the worker reads the cabin plan view and the cabin side view showing the state in which the outfitted parts are attached, specifies the position of the outfitted parts with respect to the skeleton, and indicates the mounting position of the outfitted parts with chalk or graphite. Write on top. Then, fittings are attached to the marked skeleton.

[0005]

By the way, the cabin block is placed in a factory with its ceiling down. In contrast, the cabin plan is often drawn from above the ceiling. That is, the actual skeleton on which the worker performs marking and the plan view of the cabin are mirror images of each other. The actual skeleton and the cabin side view have an inverted image relationship with each other.

For this reason, the work of reading and marking these drawings is different from the work of reading ordinary drawings.
Special skills are required. Further, the fittings are classified according to types such as air ducts, pipes, cables, and the like, and the drawings are divided according to these types. Then, the worker must correctly read the information described in the plurality of drawings.

Therefore, when an unskilled worker performs this marking work, not only the work efficiency is significantly reduced, but also the marking may be performed at a different position. If this marking is not done correctly, the fittings will not be installed correctly. In particular, if the positions of the outfits interfere with each other to cause a so-called hit, the outfits cannot be mounted.

In order to reduce such marking work, there has been proposed an outfitting support system for projecting a drawing directly on a frame by a projection device such as a projector. In this method, an operator performs marking on the skeleton according to the drawing projected on the skeleton and character information indicating various designations. For this reason, the work of reading drawings for marking is reduced.

However, in such an outfitting support system, since the projection device and the projection surface are separated from each other, if an operator intervenes between the projection device and the projection surface, necessary drawing information and character information are required. There is a problem that is hidden behind.
Further, when the character information protrudes out of the projection area, the worker must shift the drawing or refer to another drawing. For this reason, improvement in working efficiency cannot be expected very much.

Therefore, even workers other than skilled workers,
An object of the present invention is to provide an outfitting support system that enables accurate and efficient installation of outfitted components.

[0011]

The outfitting support system according to the present invention employs the following configuration in order to solve the above problems.

That is, the outfitting support system includes a three-dimensional CAD device that stores three-dimensional drawing data indicating the position and orientation of a skeleton and the outfitting components attached to the skeleton, and the position of a worker attaching the outfitting component to the skeleton. And a position and orientation measurement device for acquiring worker position and orientation data indicating the orientation,
An image projection device that displays a virtual reality image in a state where the worker is superimposed on a real image visually recognized by the worker; and from which position the worker is in which direction based on the worker position and orientation data. A gaze data generation unit that generates gaze data indicating whether the user is facing the camera, and, based on the gaze data, from the three-dimensional drawing data stored in the three-dimensional CAD device, within a visual field range of the worker in the skeleton. A display data cutout unit that cuts out image data of the outfitting device attached to the area as display data, and transmits the display data to the image projection device, and the image projection device sends the display data to the worker. A communication unit for displaying the virtual reality image as the virtual reality image.

With this configuration, the worker visually recognizes the real image transmitted by the image projection device and the virtual reality image displayed by the image projection device in a superimposed state. That is, the worker can visually recognize an image in which the actual frame and the fittings to be attached to the frame are superimposed.

Further, the outfitting support system includes a three-dimensional CAD device for storing three-dimensional drawing data in which the position and posture of a skeleton and an outfit attached to the skeleton are expressed in a predetermined design drawing coordinate system; For a worker who attaches outfitting to an image projection device that displays a virtual reality image in a state of being superimposed on a real image visually recognized by the worker, the worker can photograph the skeleton and a worker located near the skeleton. From a camera and the image of the skeleton acquired by the camera, the position and orientation of the skeleton are acquired as skeleton position and orientation data expressed by the on-site coordinate system, and the skeleton position and orientation data and the three-dimensional drawing are acquired. By comparing the data indicating the position and orientation of the skeleton included in the data, the conversion relationship between the design drawing coordinate system and the site coordinate system is represented. A skeleton position / posture measurement device that acquires as conversion data, and an operation that acquires the position and orientation of the worker from the image of the worker acquired by the camera as worker position / posture data expressed by the on-site coordinate system. Based on the worker position and posture measurement device and the worker position and posture data acquired by the worker position and posture measurement device, the gaze data indicating from which position the worker is facing in which direction; A gaze data generation unit generated according to a system, the coordinate transformation data acquired by the skeleton position and orientation measurement device, and the gaze data generated by the gaze data generation unit, from which position the worker A coordinate conversion unit that generates a design drawing gaze data expressing which direction is directed by the design drawing coordinate system, and the coordinate conversion From the three-dimensional drawing data stored in the three-dimensional CAD apparatus based on the design drawing gaze data generated by the above, image data of the outfitting parts attached to an area within the field of view of the worker in the skeleton A display data cutout unit that cuts out as display data, and the display data acquired by the display data cutout unit is transmitted to the image projection device, and the display data is sent to the worker by the image projection device. And a communication unit for displaying the image as the virtual reality image.

[0015] The image projection device may be capable of switching display or non-display of the virtual reality video for each type of the outfit, and may display the virtual reality video in different colors. You may. Further, the image projection device may be capable of switching display or non-display of a virtual reality image of the outfitting component at each stage of the outfitting component mounting process to completion. The worker position and orientation data is
It may be generated based on a signal detected by a radio wave sensor, a sound wave sensor, a gyro sensor, or the like.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram schematically illustrating a configuration and a processing flow of the outfitting support system according to the present embodiment. As shown in FIG. 1, the outfitting support system includes:
It includes a three-dimensional CAD device 1, a position and orientation measurement device 2, an image acquisition device 3, and an image projection device 4.

The three-dimensional CAD apparatus 1 stores, in a storage unit (not shown), a cabin design drawing (three-dimensional drawing data) expressed in accordance with a predetermined design drawing coordinate system which is three-dimensional coordinates (for example, three-dimensional Cartesian coordinates). Have. This cabin plan is
It may be created by the three-dimensional CAD device 1. Further, the cabin plan may be obtained by converting the two-dimensional drawing data by the three-dimensional CAD device 1 as described below.

FIG. 2 is an explanatory diagram showing the processing of the three-dimensional CAD apparatus 1. The three-dimensional CAD device 1 can read two-dimensional drawing data such as a three-sided drawing, and can represent an object represented by the two-dimensional drawing data as three-dimensional drawing data in a design drawing coordinate system. That is, this three-dimensional C
The AD device 1 reads two-dimensional drawing data and outputs three-dimensional drawing data.

FIG. 3 is an explanatory diagram of the position and orientation measuring device 2. As shown in FIG. 3, the position and orientation measurement device 2
Are a camera 21, an image processing unit 22, a camera driving unit 23,
The camera includes a camera control unit 24, a camera sensor 25, and a position and orientation calculation unit 26.

FIG. 4 is a schematic diagram of an outfitting factory.
In this outfitting factory, a frame K is arranged with its ceiling down. Each of the cameras 21 is provided at three or more positions where the frame K in the outfitting factory can be seen.

Each of these cameras 21 has, for example, a lens and a CCD, and as shown in FIG.
Each of the image processing units 22 is connected to an image processing unit 22 provided for each of them, and acquires image data and transmits the image data to the image processing unit 22.

Further, the camera driving unit 23
It is provided for each camera and is connected to the camera control unit 24. Then, the camera driving unit 23 is controlled by the camera control unit 24 to rotate the corresponding camera 21 in a predetermined two-axis direction. Further, the camera driving unit 23
Is controlled by the camera control unit 24 and the corresponding camera 2
The lens of the camera can be focused on an object existing at a predetermined position within one shooting range.

The camera sensor 25 is provided for each camera 21 and is connected to the camera control unit 24. Then, each camera sensor 25 acquires the attitude (rotational position in two axial directions) of the corresponding camera 21 and the distance to the object from the camera 21 and transmits the camera attitude distance data to the camera control unit 24. .

Each image processing section 22 has a corresponding camera 21
Performs image processing on the image data obtained from the image data, and extracts data on the position and orientation of the object in the image (position and orientation data in the image).

The position / orientation calculation unit 26 is, for example, a computer, and includes the image processing units 22 and the camera control units 2.
4, respectively. This position and orientation calculation unit 26
The camera posture distance data is acquired from each camera control unit 24, and the position and orientation data in the image is acquired from each image processing unit 22.

The position / posture calculation unit 26 generates frame position / posture data and worker position / posture data as described below based on the acquired camera posture distance data and the in-image position / posture data. Note that these body position / posture data and worker position / posture data are expressed in accordance with a site coordinate system which is a predetermined three-dimensional coordinate system. This site coordinate system is, for example, a three-dimensional Cartesian coordinate system, has an x-axis and a y-axis in a predetermined plane horizontal to the floor, and is orthogonal to the x-axis and the y-axis at a predetermined origin. It has a z-axis.

First, generation of body position / posture data will be described with reference to FIGS. FIG.
Is a perspective view of the frame K and the camera 21. FIG. FIG.
7 is a side view of the skeleton K and the camera 21, and FIG. 7 is a plan view of the skeleton K and the camera 21. In these figures, only one of the three cameras 21 is shown, but the same processing is performed for the other cameras 21.

As shown in FIG. 5, the camera 21 can photograph an object within the predetermined photographing range P.
Further, as shown in FIG. 6, the camera 21 can measure a distance L to an object (body K) located at the center (on the optical axis of the lens) within the photographing range P.
Assuming that the angle between the vertical line lowered from the camera 21 toward the floor and the optical axis of the lens is the first camera rotation angle ψ, the angle from the intersection S between the optical axis of the lens and the surface of the body K to the vertical line The horizontal distance d is d = L · sinψ.

The position of the camera 21 is represented by (x, y, z) = (x _cam , y _cam , z _cam ) in the on-site coordinate system. The position of the camera 21 is stored in advance in a storage area (not shown) of the position and orientation calculation unit 26.

Using the position of the camera 21 as the origin, the x-axis,
When a predetermined three-dimensional Cartesian coordinate system (local coordinate system) including an X axis, a Y axis, and a Z axis parallel to the y axis and the z axis is set, as shown in FIGS. S is represented by this local coordinate system as [X, Y, Z] = [L _X , L _Y , L _Z ].

Then, as shown in FIG.
Assuming that the angle between the X axis and the straight line obtained by projecting the straight line connecting 1 and the intersection point S on the XY plane is the second camera rotation angle φ, L _X and L _Y are L _X = d · cosφ = L · sinψ · cosφ ... L Y = d · sinφ = L · sinψ · sinφ ... is.

Also, as shown in FIG. 6, L _Z is L _Z = L · cos….

The camera attitude distance data acquired by the camera sensor 25 includes the above distance L, the first camera rotation angle ψ, and the second camera rotation angle φ. Then, the position and orientation calculation unit 26 acquires the camera orientation distance data from the camera sensor 25 via the camera control unit 24, and substitutes the values of L, ψ, and φ into the above equations, and The coordinates [L _X , L _Y , L _Z ] of the intersection S in the local coordinate system are calculated.

The position / orientation calculation unit 26 is a camera 2 based on a site coordinate system stored in advance in a storage area (not shown).
1 position _{(x cam, y cam, z} cam) and the coordinates of the intersection point S by the local coordinate system [L _X, L _Y, L _Z] from the coordinates of the intersection point S by the site coordinate system (x _s, y _s , Z _s ) can be calculated. Note that these x _s, y _s and z _s is the following expression, the _{x s = x cam + L X} y s = y cam + L Y z s = z cam + L Z, respectively are calculated.

On the other hand, the image processing section 22 performs image processing on the image obtained from the camera 21. That is, the image processing unit 22 performs a process of detecting a boundary line (contour line) in the image. Then, the image processing unit 22 acquires a line image including the outline of the skeleton K within the photographing range P of the camera 21 as position data in the image, and transmits the line image to the position / posture calculation unit 26.

The position / posture calculation unit 26 is provided for each camera 21.
Of the image output from the image processing unit 22 corresponding to
Creates a three-dimensional outline drawing of the skeleton K based on the placement data
I do. The position and orientation calculation unit 26 uses a site coordinate system.
Coordinates of the intersection S (x_s, Y_s, Z _s), And this skeleton K
The three-dimensional three-dimensional outline drawing in the field coordinate system
Thus, frame position / posture data is generated.

Further, the position / posture calculating section 26 compares the skeleton position / posture data with the three-dimensional drawing data in the three-dimensional CAD apparatus 1 to obtain the skeleton position / posture data based on the site coordinate system and the design drawing coordinate system. The conversion relationship with the three-dimensional drawing data is acquired.

That is, the position / posture calculation unit 26 rotates the three-dimensional drawing data in the predetermined three-axis directions and translates the three-dimensional drawing data in the predetermined three-axis directions, respectively, and compares the data with the body position / posture data. Then, the position and orientation calculation unit 26 rotates and translates the three-dimensional drawing data until the skeleton position and orientation data matches the three-dimensional drawing data. By this processing, the conversion relationship between the frame position / posture data in the site coordinate system and the three-dimensional drawing data in the design drawing coordinate system is obtained.

This conversion relationship is a conversion relationship between the site coordinate system and the design drawing coordinate system. That is, the position and orientation calculation unit 2
6 acquires an operator such as a matrix for converting the data into position and orientation data in the design drawing coordinate system by acting on arbitrary position and orientation data in the site coordinate system as coordinate conversion data, (Coordinate conversion unit 32).

As described above, the position and orientation measurement device 2 generates the worker's position and orientation data on the worker who attaches the outfit to the frame K after generating the body position and orientation data and the coordinate conversion data. Hereinafter, generation of the worker position and orientation data will be described.

The worker, as shown in FIG.
The image projection device 4 is mounted. This image projection device 4
Is a transmission type three-dimensional image projection device formed in a goggle shape, and can display a virtual reality video obtained by the image acquisition device 3 as described later. Then, the worker can visually recognize the displayed virtual reality video and the displayed real video in a superimposed state.

The position and orientation measuring device 2 acquires worker position and orientation data indicating the position and orientation of the image projection device 4 worn by the worker. The worker position / posture data is obtained in substantially the same manner as the above-described skeleton position / posture data.

However, since the worker always moves, the camera 21 automatically tracks the worker so as to put the worker in the center of the photographing range P. That is, the camera control unit 2
4 controls the camera driving unit 23 based on the latest worker position and orientation data obtained from the position and orientation calculation unit 26 so that the camera 21 puts the worker in the center of the shooting range P. The camera driving unit 23 changes the posture (ψ, φ) of the camera 21 according to the control of the camera control unit 24. Then, the position and orientation measurement device 2 acquires the worker's position and orientation data in real time from the start of the outfitting work by the worker until it is completed, and continuously transmits the data to the image acquisition device 3 (the gaze data generation unit 31).

In the above description, the body position / posture data and the worker position / posture data are acquired by the same position / posture measuring device 2. In this case, the position and orientation measurement device 2 also serves as a skeleton position and orientation measurement device and a worker position and orientation measurement device. However, these skeleton position and orientation measurement devices and worker position and orientation measurement devices are:
The devices may be independent of each other.

Next, the image acquisition device 3 will be described.
As shown in FIG. 1, the image acquisition device 3 includes a gaze data generation unit 31, a coordinate conversion unit 32, and a display data cutout unit 3.
3 and a communication unit 34. The gaze data generation unit 31 and the coordinate conversion unit 3 in the image acquisition device 3
2, and the display data extracting unit 33 may be configured by a computer or may be configured by an electronic circuit designed specifically. Further, the communication unit 34 has a wireless communication function, and can transmit and receive data to and from a wireless communication unit (not shown) of the image projection device 4.

The gaze data generator 31 obtains worker position / posture data from the position / posture measuring device 2. The gaze data generation unit 31 recognizes from which position the worker is facing in which direction based on the worker position and orientation data, and converts the position and direction as gaze data based on the site coordinate system. To the unit 32.

The coordinate conversion unit 32 acquires coordinate conversion data from the position and orientation measurement device 2 in advance. Further, the coordinate conversion unit 32 acquires the gaze data in the on-site coordinate system obtained from the gaze data generation unit 31, and applies the coordinate conversion data to the gaze data, so that the worker can determine which direction from which position. It is converted into the design drawing gaze data expressing the orientation of the drawing in the design drawing coordinate system, and transmitted to the display data cutout unit 33.

The display data extracting section 33 extracts display data from the three-dimensional drawing data stored in the three-dimensional CAD apparatus 1 based on the design drawing gaze data transmitted from the coordinate converting section 32. That is, the display data extracting unit 33
The operator recognizes from which position in the blueprint coordinate system the heading is to be directed to the cabin block expressed as three-dimensional drawing data based on the blueprint coordinate system, and the visual field range of this worker The image of the fittings that should be inside is extracted as display data.

It should be noted that the three-dimensional CAD data includes both the data of the frame K and the data of the installed fittings.
The display data is generated by removing the data of the skeleton K from the data and extracting only the data of the fittings. In addition, the display data includes an image projected on the left eye of the worker, and
This is stereo image data including an image projected to the right eye.

The communication section 34 acquires the display data cut out by the display data cutting section 33 and
Wirelessly. The image projection device 4 receives the display data and performs stereo display corresponding to each of the left and right eyes of the worker. Then, the worker visually recognizes the display data as a stereoscopic image and visually recognizes the actual frame K. That is, to the worker, it looks as if the fittings as the virtual reality images are attached to the frame K which is the real images at the predetermined design positions. The display data may include various types of character information described in the design drawing. In this case, the worker visually recognizes the skeleton K, which is a real image, and the fittings and character information, which are virtual reality images.

Then, the worker attaches the outfit to the actual frame K while watching a virtual reality image of the outfit correctly arranged at a predetermined design position. For this reason, the worker does not need to perform the work of reading the drawings, the work of marking, and the like, and only needs to attach the actual outfitting parts as seen. Therefore, even workers other than skilled workers,
Outfitting work can be performed accurately and quickly.

<Modification> In the above embodiment, the image projection device 4 may be capable of displaying or not displaying an image for each type of fitting. In this case, the three-dimensional drawing data in the three-dimensional CAD apparatus 1 previously includes attribute data indicating the type of each of the outfits, and the display data cutout unit 33 applies the attribute data to the image of each outfit. The display data is created by adding attribute data corresponding to the fittings, and the image projection device 4 switches display or non-display for each type of fitting according to the instruction of the worker.

Therefore, the operator can display only necessary types of fittings by operating an operation unit (not shown) of the image projection device 4. For example, a worker
Only one or two types of fittings selected from an air duct, a pipe, and a cable can be selected and displayed on the image projection device 4. Therefore, while displaying images of all the outfits, the worker confirms that there is no so-called hit between the outfits, and then displays only the outfits of the type currently being installed. You can also do the work.

Further, the display data extracting section 33 may create display data so that the image projection device 4 can display virtual reality images in different colors for each type of fitting. Further, each outfit may be displayed in a different pattern (line type, hatching, etc.) for each type.

The image projection device 4 may have a visual line direction sensor (not shown) for detecting the visual line direction of the worker as visual line data.

[0056]

According to the outfitting support system configured as described above, a worker can view a virtual reality image of an outfitted component correctly placed at a predetermined design position while watching the outfitting component with respect to the actual skeleton. Can be attached. Therefore, there is no need for the worker to read the drawings or perform the marking work, etc., and it is only necessary to attach the actual outfitting parts as seen, so even workers other than skilled workers can accurately and quickly Outfitting work can be performed.

[Brief description of the drawings]

FIG. 1 is a diagram schematically showing a configuration and a processing flow of an outfitting support system according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing processing of a three-dimensional CAD device.

FIG. 3 is an explanatory diagram of a position and orientation measurement device.

Fig. 4 Schematic diagram of an outfitting factory

FIG. 5 is a perspective view of a frame and a camera.

FIG. 6 is a side view of a frame and a camera.

FIG. 7 is a plan view of a frame and a camera.

FIG. 8 is an image of a worker's head

[Explanation of symbols]

 Reference Signs List 1 3D CAD device 2 Position and orientation measurement device 21 Camera 3 Image acquisition device 31 Gaze data generation unit 32 Coordinate conversion unit 33 Display data extraction unit 34 Communication unit 4 Image projection device K Body

Claims (4)

[Claims] 1. A three-dimensional CAD device for storing three-dimensional drawing data indicating the position and posture of a skeleton and fittings attached to the skeleton, and a worker indicating the position and posture of a worker attaching the fitting to the skeleton. A position and orientation measurement device that acquires position and orientation data, an image projection device that displays a virtual reality image in a state of being superimposed on a real image visually recognized by the worker, and A gaze data generation unit that generates gaze data indicating from which position the worker is facing in which direction, based on the gaze data, and the three-dimensional drawing data stored in the three-dimensional CAD device. From, a display data cutout unit that cuts out image data of the outfitting equipment attached to an area within the field of view of the worker in the skeleton as display data, A communication unit for transmitting the display data to the image projection device and displaying the display data to the worker as the virtual reality image by the image projection device. . 2. A three-dimensional CAD apparatus for storing three-dimensional drawing data in which the position and orientation of a skeleton and fittings attached to the skeleton are represented by a predetermined design drawing coordinate system, and an operation of attaching fittings to the skeleton. An image projection device that displays a virtual reality image in a state of being superimposed on a real image visually recognized by the worker, a camera capable of photographing the skeleton and a worker located near the skeleton, and the camera From the acquired image of the skeleton, the position and orientation of the skeleton are acquired as skeleton position and orientation data represented by the site coordinate system, and the skeleton position and orientation data and the three-dimensional drawing data By comparing the data indicating the position and orientation of the frame with the frame, the conversion relationship between the design drawing coordinate system and the site coordinate system is acquired as coordinate conversion data. A position and orientation measurement device, and a worker position and orientation measurement device that acquires the position and orientation of the worker from the image of the worker acquired by the camera as worker position and orientation data expressed by the site coordinate system. Based on the worker position and posture data acquired by the worker position and posture measurement device, gaze data indicating which direction the worker is facing from which position, and gaze data that is generated in accordance with the on-site coordinate system. A generation unit, based on the coordinate transformation data acquired by the skeleton position and orientation measurement device, and the gaze data generated by the gaze data generation unit, from which position the worker is facing and in which direction A coordinate conversion unit that generates design drawing gaze data expressing the design drawing coordinate system in the design drawing coordinate system; and the design generated by the coordinate conversion unit. Based on the gaze data, stored in the three-dimensional CAD in the apparatus the 3
A display data cutout unit that cuts out, as display data, image data of the outfitting equipment attached to an area of the skeleton in the field of view of the worker from the three-dimensional drawing data, and the display acquired by the display data cutout unit A communication unit that transmits data to the image projection device and causes the image projection device to display the display data to the worker as the virtual reality video. 3. The outfitting support system according to claim 1, wherein the display data cutout section cuts out image data of each type of the outfitting component. 4. The display data cutout unit cuts out image data of each type of fitting as display data to be displayed in different colors from each other.
The outfitting support system according to any one of the above.