JP2003270719A - Projection method, projector, and method and system for supporting work - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a system for supporting work, which enable not only a worker to accurately and easily recognize indications but also an instructor to accurately and easily perform instructing operations though having a simple constitution and do not burden the worker and cope with an object having any depth and are not hindered even in the case of a plurality workers. <P>SOLUTION: Two sets of PCs 1a and 1b, cameras 2a and 2b, and projectors 3a and 3b are installed in a work space wherein the worker performs work of an object M. The three-dimensional shape of the object M is measured by cameras 2a and 2b and projectors 3a and 3b, and a CG image of the object M is displayed on a PC 4 in a remote place wherein the instructor is present. When the instructor uses an input tool to give an instruction related to work of the object M, instruction data is transmitted to PCs 1a and 1b, and an indication image corresponding to instruction data is projected on the object M by the projectors 3a and 3b. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a projection method and apparatus for projecting an image on a three-dimensional object, and a work support method and system for giving instructions such as a work procedure to the object.

[0002]

2. Description of the Related Art With the development of image technology and wide area communication technology, various methods have been developed as a method of giving a working instruction to a worker who works on an object. .

As a system for giving an instruction to a worker from a remote place using an image, there is a system which directly uses a normal television conference. With this system, the state of the work space, which is the place of work, can be easily observed at a remote location. However, if the same object as the work target does not exist in the remote location on the instructing side, the instruction content cannot be directly explained using the object, so the instruction content must rely on voice, gesture / hand gesture, or drawing. There is a problem that it will disappear and it will not be possible to give accurate instructions.

In addition, even though it is a dialogue system using images, it is possible to easily understand the mutual positional relationship with each other (Hyper Mi
rror) system is known. In this system, images taken in the work space and images taken in remote areas are combined,
A mirror image is created by inverting the image horizontally and projecting it on the screen. It is displayed as if the instructor at the remote place is in the background of the work space, and the instructor can give an instruction to the actual object to be worked by a physical action such as pointing. However, the operator needs to compare the display on the screen with the actual object, and it is not easy to understand the instruction. Moreover, the operator cannot give the instruction directly to the object and see the image. Since it is the instruction of, the fine adjustment of the instruction direction cannot be avoided.

In order for the operator to grasp the instruction more intuitively, it seems that the instruction is directly given to the actual object, that is, the instruction to the object is geometrically aligned. preferable. As a mixed reality technology for satisfying this, a system using an HMD (Head Mount Display) and a system using a half mirror have been proposed.

In the system using the HMD, there is no limitation on the instruction range depending on the position / posture of the head, and a relatively large work space can be targeted. However, in order to geometrically match the object and the pointing image, it is necessary to attach a position sensor to the HMD or provide a position marker in space. Also,
The worker has to equip the HMD, which places a heavy burden on the worker.

In a system using a half mirror, an operator visually observes a pointing image displayed on a display and an actual object so that the operator can observe the object. The burden is also small. However, the position of the viewpoint is restricted by the size of the half mirror, which makes it difficult for a plurality of workers to perform the work. Since the planar images on the display are superimposed, there is a problem that it is not possible to deal with an object having a large depth.

As described above, each of the conventional systems has its own drawbacks, and none of them are satisfactory systems, and it is desired to develop a new work support system capable of solving these drawbacks.

The present invention has been made in view of the above circumstances, and eliminates the drawbacks of the conventional system, and the operator recognizes an instruction and the instruction operation by the instructor is accurate and accurate even with a simple structure. An object of the present invention is to provide a projection method, a projection device, a work support method, and a work support system that are easy, do not impose a burden on workers, can handle objects of any depth, and that do not hinder a plurality of workers. .

[0010]

A projection method according to the first invention measures a three-dimensional shape of an object, generates image data for projection based on the measured three-dimensional shape data, and generates the generated image data. Is projected onto the object.

A projection apparatus according to a second aspect of the present invention, means for measuring the three-dimensional shape of an object, means for generating image data for projection based on the measured three-dimensional shape data, and a means for generating the image data. Means for projecting the captured image onto the object.

In the first and second inventions, the three-dimensional shape of the object is measured, and the image is projected on the object based on the three-dimensional shape data. By using the three-dimensional shape information of the object in this way, an accurate image display can be performed corresponding to the shape of the three-dimensional object.

A work support method according to a third aspect of the present invention is a method for supporting work on an object, wherein a three-dimensional shape of the object is measured, and instruction data representing an instruction regarding the work is generated based on the measured shape data. An image according to the generated instruction data is projected on the object.

A work support system according to a sixth aspect of the present invention is a system for supporting work on an object, wherein means for measuring a three-dimensional shape of the object and instruction data representing an instruction for the work are generated based on the measured shape data. And a means for projecting an image corresponding to the generated instruction data onto the object.

In the third and sixth inventions, when a worker is working on an object, an image required for the work is directly projected and displayed on the object to be worked. That is, first, the three-dimensional shape of the work target object is measured, instruction data representing an instruction regarding work is generated based on the measured shape data, and an image corresponding to the generated instruction data is projected on the object. Therefore, since the work instruction is directly displayed on the object,
The operator can easily and accurately confirm the instruction content. Also,
No special equipment is required for the worker, and there is no burden on the worker.

A work support method according to a fourth aspect of the present invention is a method for supporting work on an object using one or a plurality of sets of cameras and projectors provided in a work space of the object, wherein light is emitted from the projector to the object. Is projected and the image of the object is acquired by the camera, the three-dimensional shape of the object is measured from the acquired image of the object, and instruction data indicating an instruction regarding the work based on the measured shape data. Is generated, and the projector is controlled to project an image corresponding to the generated instruction data on the object.

A work support system according to a seventh aspect of the present invention is a system for supporting work on an object, wherein one or a plurality of sets of cameras and projectors, and light projected from the projector onto the object are photographed by the camera. Measuring means for measuring the three-dimensional shape of the object from the image of the object, generating means for generating instruction data indicating an instruction regarding the work based on the measured shape data, and an image corresponding to the generated instruction data for the object. Control means for controlling the projector to project.

In the fourth and seventh inventions, the three-dimensional shape of the object is measured by using the camera and the projector installed in the place (work space) where the worker works on the object, and the measured shape is measured. Instruction data representing an instruction regarding work is generated based on the data, and an image corresponding to the generated instruction data is projected on the object by the projector. Therefore, since the work instruction is directly projected from the projector onto the object, the operator can easily and accurately confirm the instruction content. In addition, no special equipment is required for the worker, and there is no burden on the worker.

In the work support method according to the fifth aspect of the present invention, in the fourth aspect, the instruction is input in two-dimensional coordinates, the input instruction is converted into three-dimensional coordinates, and the instruction data is generated. The instruction data in three-dimensional coordinates is converted into two-dimensional coordinates for the projector.

A work support system according to an eighth aspect of the present invention is the seventh aspect.
In the invention, the generating means has means for receiving an input of the instruction in two-dimensional coordinates, and means for converting the instruction in the input two-dimensional coordinates into three-dimensional coordinates to generate the instruction data. However, the control means has means for converting the generated instruction data in three-dimensional coordinates into two-dimensional coordinates for the projector.

In the fifth and eighth inventions, an instruction is input in two-dimensional coordinates, the instruction in the two-dimensional coordinates is once converted into three-dimensional coordinates to generate instruction data, and the generated three-dimensional coordinates are generated. The instruction data is converted into two-dimensional coordinates for the projector. Therefore, it is possible to easily input an instruction using the mouse, and it is possible to realize accurate projection using the projector.

A work support system according to a ninth aspect of the present invention is the seventh aspect.
Alternatively, the eighth invention is characterized by further comprising a data communication line connecting the measuring means and the generating means, and the generating means and the control means.

According to the ninth aspect of the invention, when the place where the instructor who gives the work instruction is present is far away from the work space, the processing is performed as follows. First, a camera and a projector installed in a work space are used to measure a three-dimensional shape of an object, and the acquired image of the object and distance data are transmitted to a remote place where an instructor exists through a data communication line. To do.
At a remote location, an image of an object is displayed based on these transmission data, and instruction data representing an instruction regarding work is generated for the displayed image. The generated instruction data is sent to the work space via the data communication line, and the image corresponding to the instruction data is projected on the object by the projector. Therefore, it is possible to easily give an appropriate work instruction even from a remote place.

A work support system according to a tenth invention is the work support system according to any one of the seventh to ninth inventions, wherein a plurality of sets of the camera and the projector are present, and when the image is projected onto the object, the projectors respectively project from the plurality of projectors. It is characterized in that projection lights of different colors are emitted.

According to the tenth aspect of the invention, by projecting light of different colors from each of the plurality of projectors, it is possible to easily and clearly designate the portion where the projection lights are mixed as the designated position.

A work support method according to an eleventh aspect of the present invention is a method for supporting positioning work of an object having a known shape, wherein image data for projection is generated based on the shape of the object, and the generated image data is generated according to the generated image data. The projected image is projected at a position in the real space where the object is to be positioned.

The work support system according to the twelfth invention is a system for supporting the positioning work of an object whose shape is already known, and means for generating image data for projection based on the shape of the object, and the generated image data. And a means for projecting an image corresponding to the above into a position in the real space where the object is to be positioned.

In the eleventh and twelfth aspects of the invention, when the operator performs the positioning work of the object whose shape is known, the image of the object is projected at the position of the real space where the object should be positioned. Therefore, the operator can easily and accurately confirm the instructed position, and the correct alignment of the object can be easily performed.

[0029]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be specifically described below with reference to the drawings showing the embodiments thereof. FIG. 1 is a diagram showing a configuration of a work support system of the present invention. Two personal computers (hereinafter referred to as PCs) 1a, 1 are installed in a place (work space) where the worker works on the object M.
b is provided. For each PC 1a, 1b,
The camera 2a and the projector 3a, and the camera 2b and the projector 3b are connected so that the respective photographing surfaces and the respective projection surfaces face the object M. Since the camera 2a and the projector 3a (camera 2b and projector 3b) connected to the same PC 1a (1b) perform shape measurement based on the triangulation method using the one set, the camera 2a (2b) has an image capturing range. The projectors 3a (3b) are installed so as to be overlapped with the projection range of the projector 3a (3b) by an appropriate baseline length. The cameras 2a and 2b and the projector 3a,
The installation position of 3b is fixed.

On the other hand, a PC 4 is provided in a place (remote location) where an instructor who gives an instruction regarding the work of the object M exists, and PCs 1a and 1b in the work space (worker side) and a remote location (instructor). Various data can be transmitted / received to / from the PC 4 (on the side) via the data communication line 5.

FIG. 2 is a functional block diagram of the PCs 1a and 1b. Each of the PCs 1a and 1b includes a CPU 11, a camera control unit 12, a projector control unit 13, a communication interface unit 14, a ROM 15, a RAM 16 and the like. C
The PU 11 is connected to each of these hardware units as described above via a bus 17, and controls them according to a program stored in the ROM 15 and executes various software functions.

The camera control section 12 acquires image data of the object M photographed by the cameras 2a and 2b. The projector control unit 13 controls the light projected on the object M from the projectors 3a and 3b. The communication interface unit 14 is
It is connected to the PC 4 via the data communication line 5,
Data (photographed image data and distance data of the object M, instruction data for work, etc.) is exchanged with C4.
The ROM 15 stores in advance various software programs necessary for operations such as calibration processing, three-dimensional shape measurement processing, and instruction data projection processing, which will be described later. The RAM 16 is configured by SRAM, flash memory, or the like, and stores temporary data generated when software is executed.

FIG. 3 is a functional block diagram of the PC 4.
The PC 4 has a CPU 21, a display 22, and a mouse 2.
3, a keyboard 24, an instruction data generation unit 25, a communication interface unit 26, etc. are provided. The CPU 21 is connected to each of these hardware units as described above via the bus 26 and controls them.

The display 22 displays the object M based on the data sent from the PCs 1a and 1b. The mouse 23 and the keyboard 24 receive instructions regarding work from the instructor. The instruction data generation unit 25 generates work instruction data based on the input instruction from the instructor. The communication interface unit 26 uses the data communication line 5
Is connected to the PCs 1a and 1b via
Data is exchanged with 1b.

Next, the operation of the work support system of the present invention will be described. FIG. 4 is a diagram showing an outline of this operation.

First, in the working space, the cameras 2a and 2b and the projectors 3a and 3b are calibrated using a reference object. Then, the cameras 2a and 2b and the projector 3 that have completed the calibration
The three-dimensional shape of the work target object M is measured using a and 3b. The still color image and the distance data obtained by this three-dimensional shape measurement are transmitted to the PC 4 at a remote place.

The PC 4 generates a CG (computer graphics) of the object M on the basis of the still color image and the distance data and displays it on the display 22. In addition, in order to grasp the state of work in real time, the camera 2
The moving image of the object M captured in a and 2b is also displayed on the display 2
It is displayed in 2. The instructor is a mouse 23, a keyboard 2
4 is used to give an instruction regarding work by a GUI (graphical user interface) operation on the display 22. The instruction data is generated by the instruction data generation unit 25 according to this input instruction, and the generated instruction data is transmitted to the PCs 1a and 1b in the work space.

The PCs 1a and 1b respectively generate image data for projection from the instruction data, and project images corresponding to the image data onto the object M by the projectors 3a and 3b. The worker performs the work according to this projected image.

When the worker works on the object M according to the instruction and the shape and position of the object M change, the three-dimensional shape of the object M is measured again and the display image on the PC 4 is updated. The above process is repeated.

Next, details of each processing (calibration processing, three-dimensional shape measurement processing, display processing of the object M on the PC 4, input processing of instruction, projection processing of instruction data) in the above-described operation will be described.

(Calibration processing) In order to measure the three-dimensional shape of the object M, the projectors 3a, 3
It is necessary to calibrate the camera 2a (2b) and the projector 3a (3b) in order to geometrically align the instruction projected by b with the target object M. This calibration process uses the cubic reference object D shown in FIG. A reference point for calibration is drawn on each surface of the reference object D.

First, the reference object D is photographed by the camera 2a (2b), and the arrangement of the reference point is recognized from the obtained image, whereby the world coordinates (X, Y, Z) of the reference point and the camera 2a (2b) are detected. ), A large number of pairs with the coordinates (x _c , y _c ) are obtained, and the camera parameter C shown in (1) below is obtained. Here, C is a perspective transformation matrix of 3 rows and 4 columns.

[0043]

[Equation 1]

FIG. 5B shows how the reference points on the reference object D are correctly recognized (points facing each other on each surface are connected by a straight line). The intersection of these straight lines is used as the set of corresponding points used for calibration.

Next, the projector 3a (3
Vertical / horizontal gray code pattern light is projected from b), and an image of the reference object D is acquired by the camera 2a (2b).
By analyzing this image, the projector 3a (3
b) each pixel (x _p , y _p ) and camera coordinates (x _c ,
y _c ) is obtained. Since the relationship between the camera coordinates (x _c , y _c ) and the world coordinates (X, Y, Z) has already been obtained, by using this, as shown in (2) below, as in (1) above. The projector parameter P of 3 rows and 4 columns is obtained. The functions of the focal plane of the lens are different between the camera and the projector: "receive light and output image" and "emit light according to input image", but the arrangement of optical system and image plane Are the same, and both can be treated with the same mathematical model.

[0046]

[Equation 2]

If only distance data is obtained as in the case of general three-dimensional measurement, depending on the direction of the baseline length,
It is sufficient to project one of the horizontal gray code lights and obtain the degenerate projector parameters of 2 rows and 4 columns.
However, in this case, when the instruction image is projected, the projector coordinates (x _p , y _p ) corresponding to the coordinates (X, Y, Z) of the point to be instructed cannot be uniquely determined. Therefore, in the present invention, as in (2) above, the projector 3a (3b) is also calibrated in the same format as the camera 2a (2b).

Since two sets of cameras / projectors are installed in this example, it is necessary to calibrate the positional relationship between them. Therefore, for the same reference object D, each camera 2a
(2b) and the projector 3a (3b) are calibrated.

(Three-dimensional shape measurement process) Calibration
After the work is completed, the object M to be worked is placed in the work space.
Then, the object M is moved vertically and horizontally from the projector 3a (3b).
Ray code pattern light is projected to the camera 2a (2b)
The image of the object M is acquired. And the parameters mentioned above
Using the data C and P, the shape (X,
Y, Z) is the camera coordinate (x _c, Y_c) And projector
Coordinates (x_p, Y_p) From. Calibration
After the end, the camera 2a (2b) and the projector 3
Since the position of a (3b) is fixed, an arbitrary object M
The shape of can be measured.

Then, the distance data obtained as described above, together with the color image of the object M photographed by the camera 2a (2b), together with the PC 4 at the remote place (instructor side) via the data communication line 5. Sent to.

(Display Processing of Object M on PC4) The PC4 generates a polygon model based on the distance data and the color image transmitted from the PCs 1a and 1b, and the image of the object M is displayed on the display 22. Since the world coordinates corresponding to each pixel of the color image are obtained, a model with texture is formed by drawing each one of the polygons with the points having those colors and coordinates as vertices. The models obtained by two sets of cameras / projectors are simultaneously displayed, and the two measured surfaces are displayed in an overlapping manner.

The instructor can freely rotate the displayed CG model by operating the mouse 23.
The CG model can be observed from any direction, and the working state of the object M can be easily and accurately grasped. In the case of a work with a lot of movement, it is possible to grasp the work in real time by continuously transmitting the images captured in the work space and displaying the moving image on the display 22.

(Instruction Input Processing) The instructor gives an instruction by dragging a desired portion on the CG of the operation screen with the mouse 23. An example in which the instructor draws an instruction is shown in FIG. The user's instructions (arrows, circles) are displayed on the CG of the work target object M. In order to project the instruction at an accurate position on the object M, the three-dimensional drawing in the world coordinate system should be drawn. Data of coordinates (X, Y, Z) is sent to PC1
It is necessary to send to a and 1b. Since mouse coordinates are two-dimensional coordinates, it is necessary to convert them into three-dimensional coordinates in the world coordinate system. This conversion process is performed as follows, for example.

A depth buffer, which is a memory area used for displaying the context of each primitive of CG without contradiction, is used. The depth buffer is the CG to be drawn
Has the same resolution as, and each pixel has a value corresponding to the depth to the primitive. First, two-dimensional coordinates (mouse coordinates) on the operation screen at the position where the mouse 23 is clicked are acquired. Next, the depth buffer value at this coordinate is read from the memory area. Similarly, the perspective transformation matrix / model view matrix and the viewport transformation matrix are acquired. Finally, the set of the mouse coordinates and the depth buffer value is subjected to the inverse conversion of the conversion by these matrices, whereby the world coordinates corresponding to the position clicked with the mouse can be obtained.

The instruction data (three-dimensional coordinates in the world coordinate system) dragged with the mouse 23 is transmitted to the PCs 1a and 1b in the work space (worker side) via the data communication line 5.

(Projection processing of instruction data) PC 1a, 1b
Projects the transmitted instruction on the object M. FIG. 7 shows an example of projection onto the object M. The instruction drawn by the instructor (arrow, circle: see FIG. 6) is correctly projected on the object M. PC
The images 1a and 1b first convert the instruction data (three-dimensional coordinates in the world coordinate system) transmitted from the PC4 into projector coordinates (two-dimensional coordinates) according to the above (2), and display the instruction according to the converted projector coordinates. Is projected onto the object M. In this way, it is guaranteed that the projector coordinates (two-dimensional coordinates) obtained by converting the instruction data (three-dimensional coordinates) are located on the surface of the object M. As a result, the instruction is projected at the position exactly as drawn by the instructor on the operation screen. Since this projection processing is separately performed by the PCs 1a and 1b, the processing time does not depend on the number of projectors. .

By the way, the two projectors 3a and 3b
Since it is installed, it is necessary to determine whether or not the instructed three-dimensional coordinates in the world coordinate system are positions that can be projected by the respective projectors 3a and 3b. Because
As shown in FIG. 8, when the point a on the instruction A to be originally projected can be projected only from one of the projectors 3a, the three-dimensional coordinates of the point a are calculated using the projector parameter P of the other projector 3b. This is because, when the conversion is performed according to (2), the projected light reaches the point b of the object M, so that the false instruction B is projected on the object M. Therefore, for a projector that cannot accurately project an instruction, the projection process is prohibited to prevent incorrect instruction projection.

If accurate projection from both projectors 3a and 3b is possible, then both projectors 3a and 3b
The instruction is projected onto the object M using a and 3b. This makes it possible to increase the brightness of the instruction and prevent the instruction image from disappearing due to the projection light being blocked by the operator's body or hand.

The concrete method of judging whether or not the projection is possible is as follows. First, the camera 2 in which the world coordinates (X, Y, Z) in the work space transmitted from the PC 4 at the remote location (instructor side) are paired with the projector 3a (3b) to be projected.
Using the camera parameter C of a (2b), it is converted into camera coordinates (x _c , y _c ) according to the above (1). Next, from the distance data measured by the camera 2a (2b) / projector 3a (3b), the camera coordinates (x _c , y _c )
The world coordinates (X ', Y', Z ') corresponding to is acquired.
Then, the distance between this world coordinate (X ′, Y ′, Z ′) and the world coordinate (X, Y, Z) at which the instruction should be projected is calculated, and if the calculated distance is equal to or greater than the threshold value, projection should be performed. Since the point is blocked, projection of the point is prohibited. In addition, when a valid distance value is not stored in the corresponding coordinate in the distance data, that is, it is visible from the camera 2a (2b) but invisible from the projector 3a (3b), that is, the projection light reaches it. Projection is also prohibited when there is a portion that is not used or a portion whose reflectance is extremely low. Since such a determination process is separately performed by the PCs 1a and 1b, the processing time does not depend on the number of projectors.

The distance data used here is a set of cameras 2
a (2b) / projector 3a (3b), all valid coordinates in the distance data are measured by this camera 2a (2b) and projector 3a (3).
visible from b). That is, the projector 3a (3
Since the coordinates of the point invisible from b) are not included in this distance data, normal determination can be performed. However, the projector 3a
Although the projection of a point that is visible from (3b) but invisible from the camera 2a (2b) is prohibited, such an area is rare and a point of view for preventing the occurrence of false instructions. From there is no problem.

The threshold value used for the determination is set according to the measurement accuracy of the distance data, especially the difference in the shape data measured from a plurality of directions. Actually, the world coordinates (X ', Y', Z ') are calculated from the integer coordinates of four surrounding points obtained by rounding up / down the real camera coordinate values x _c , y _c , respectively.
Is determined to reduce the influence of the quantization error.

Next, a specific example of the work support system of the present invention (work for moving the object M placed on the table N) will be described. FIG. 9 shows a display example on the display 22 of the PC 4 at a remote place (instructor side), and FIG. 10 shows a projection example of the object M in the work space. In the working space, the cameras 2a, 2b / projectors 3a, 3
It is assumed that the calibration process of b is completed.

In the working space, the cameras 2a, 2b / projectors 3a, 3b measure the three-dimensional shapes of the object M and the table N that actually exist in the working space, and the acquired distance data and color image are displayed on the PC 1a, It is transmitted from 1b to PC4. A CG image of the object M based on the data is displayed on the display 22 of the PC 4 (FIG. 9).
(A)).

The instructor designates a portion to be instructed to move. Specifically, the instructor uses the mouse 23 to drag a portion of the CG image where the boundaries of the object M, such as the edges and the texture, are clear. The dragged portion is displayed in a color different from that of the object M, as indicated by the thick line portion Q in FIG. 9B. The specified drag locus (mouse coordinates) is converted into three-dimensional coordinates in the world coordinate system,
The instruction data is transmitted from the PC 4 to the PCs 1a and 1b. In the work space, this drag locus is the projector 3
It is projected on the object M at a and 3b (FIG. 10A).

Next, the instructor sets the amount of movement by operating the keyboard 24 and specifies the movement destination of the object M. The designated destination is the thick line portion R in FIG. 9 (c).
As shown in (1), the drag trajectory is displayed in a moved form. At this time, the object M and the movement destination can be rotated in any direction and displayed (FIG. 9D), and the instructor can confirm the position after movement from various directions. Therefore, it becomes possible for the instructor to specify the destination accurately and easily.

The data of this moving destination is also subjected to coordinate conversion and transmitted as instruction data to the PCs 1a and 1b, and the moving destination is projected on the object M and the table N (FIG. 10 (b)).
At this time, the aerial image cannot be presented as in the case where the binocular parallax image is given to the HMD, and the projection image is formed on the object M and the table N, so that the movement destination instruction (the thick line in FIG. 10B) is displayed. The portion S) is displayed with a slight distortion, but the worker can move the object M by referring to the destination instruction (FIG. 10).
(C)) can be easily performed.

In this example, both projectors 3
None of the projection lights from a and 3b are blocked, but when the instructions are projected from both of the projectors 3a and 3b, the operator may confuse the instructions. , 3b, the instruction is projected.

Next, another concrete example of the work support system of the present invention (work for positioning the object T, which is already prepared in the work space but not added to the work target, on the table N) will be described. . The CAD (Computer Aided Design) of the object T to be positioned is known.

The instructor uses the mouse 23 and the keyboard 24 to instruct the position and shape of the object T on the operation screen of the display 22 on which the CG image of the table N is displayed (FIG. 11A). ). The instructed position information and shape information are coordinate-converted into PC 1a as instruction data,
Sent to 1b. Then, in accordance with this instruction data, the projection lights of different colors are emitted from the projectors 3a and 3b (FIG. 11 (b)). By using the projector parameter P as a parameter for coordinate conversion, the projected image is geometrically aligned with the actual table N.

Similar to the above-mentioned example, the projectors 3a, 3a
Although it is possible to project from only one of b, the projection lights from the two projectors 3a and 3b are not confused with each other, so that the colors of the respective projection lights are changed and projected. In FIG. 11B, an area U represents a projection area from one projector 3a, an area V represents a projection area from the other projector 3b, and an overlapping area W thereof is an area on which the object T is placed. Becomes Therefore, the worker can easily position and place the object T at the correct position instructed (FIG. 11C).

Furthermore, the work support system of the present invention can be applied to other than the specific examples described above. For example, if this system is applied to surgery in a hospital, a doctor who installs a camera / projector in the operating room,
The abdominal position is instructed or the incision is exposed to the affected area on his / her own PC, an image based on the instruction is projected on the patient, and the surgeon in charge performs surgery with reference to this instruction.

In addition, for example, a camera / projector is installed in a work space where an operator is assembling a machine body such as an automobile using a large number of parts, and the assembly procedure is instructed from a distant place. Can adapt the work support system.

In the above example, the number of camera / projector sets is two, but the number of sets may be one set or three or more sets. Even if the number of sets is increased, the processing can be performed in parallel by one PC for each set, so that the time required for the processing does not increase and it is easy to increase the number of sets.

Further, in the above-mentioned example, the case has been described where the instructor located at a remote place far from the work space gives an instruction to the worker in the work space. Needless to say, it may be close to the place where is present. In addition, there is no instructor, instruction data is prepared in advance, the instruction data is sequentially sent to the work space, and an image corresponding to the instruction data is projected on the object to be worked. It is also possible to do so.

Although the position of the camera is fixed in the above-mentioned example, the camera can be rotated if the parameters of the camera when the camera is rotated are calculated in advance using the reference object. You can also

Further, in the above-mentioned example, the position is specified by projecting it onto the surface of the object, but the position can be specified in the air as follows. When the projection lights are emitted from the two projectors, the position where the projection lights overlap becomes the correct pointing position. Therefore, when the projection lights of different colors are radiated into the air from two projectors and an object is carried in the air, the operator can visually confirm the position where the projection lights of these two colors overlap each other. You can specify the position even if there is.

[0077]

As described above in detail, in the present invention, the three-dimensional shape of the object is measured and the image is projected on the object based on the three-dimensional shape data. Since information is used, even a three-dimensional object can be displayed accurately in accordance with its shape.

Further, in the present invention, the image necessary for the work is directly projected and displayed on the object to be worked, so that the structure is simple and the drawbacks in the conventional work support system (worker It is possible to solve the problems such as high load on the body, difficulty in dealing with an object having a large depth, and unsuitability for work by a plurality of people, and the operator's recognition of the instruction and the instruction operation of the instructor can be performed accurately and easily.

Further, the image data of the work space is transmitted from the work space to a remote place where the instructor is present via the data communication line, and the instruction data by the instructor is transmitted to the work space via the data communication line. Therefore, even when the instructor is present at a remote place far away from the work space, it is possible to easily perform an accurate instruction and a correct recognition of the instruction.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a work support system of the present invention.

FIG. 2 is a functional block diagram of a PC installed in a work space (worker side).

FIG. 3 is a functional block diagram of a PC installed at a remote place (instructor side).

FIG. 4 is a schematic diagram of the operation of the work support system of the present invention.

FIG. 5 is a diagram showing a reference object used in a calibration process.

FIG. 6 is a diagram showing an example in which an instructor draws an instruction on a CG image of an object.

FIG. 7 is a diagram showing an example in which an instruction is projected on an object.

FIG. 8 is a diagram showing generation of a false instruction by projecting an object on the back surface.

FIG. 9 is a diagram showing a display example on a PC installed at a remote place (instructor side) in a specific example of the present invention.

FIG. 10 is a diagram showing an example of projection onto an object in a specific example of the present invention.

FIG. 11 is a diagram showing a display example on a PC installed at a remote place (instructor side) and a projection example on an object in another specific example of the present invention.

[Explanation of symbols]

1a, 1b PC 2a, 2b cameras 3a, 3b projector 4 PC 5 data communication lines

Continued front page    F term (reference) 2F065 AA04 AA53 FF05 JJ03 JJ05                       JJ26                 2K103 AA22 AA26 AB10 BB05                 5B050 AA03 BA04 BA09 BA13 CA07                       CA08 DA04 DA07 EA07 EA19                       EA24 EA27 EA28 EA30 FA02                       FA05 FA09 FA14                 5B057 AA04 BA02 CA01 CA08 CA13                       CA16 CB01 CB13 CB17 CC04                       CD03 CD14 DA07 DA17 DB03                       DB06 DB09 DC09 DC22 DC36

Claims (12)

[Claims] 1. The three-dimensional shape of an object is measured, and the measured three
A projection method comprising: generating image data for projection based on three-dimensional shape data, and projecting an image corresponding to the generated image data onto the object. 2. A means for measuring a three-dimensional shape of an object, a means for generating image data for projection based on the measured three-dimensional shape data, and an image corresponding to the generated image data is projected on the object. A projection device comprising: 3. A method for supporting work on an object, wherein a three-dimensional shape of the object is measured, instruction data representing an instruction regarding the work is generated based on the measured shape data, and an image corresponding to the generated instruction data is generated. A method for supporting work, characterized in that the object is projected onto the object. 4. A method for assisting work on an object using one or a plurality of sets of cameras and projectors provided in a work space for an object, the method comprising: projecting light from the projector onto the object, An image of an object is acquired by the camera, a three-dimensional shape of the object is measured from the acquired image of the object, instruction data representing an instruction regarding the work is generated based on the measured shape data, and the generated instruction data A method for supporting work, wherein the projector is controlled so as to project an image corresponding to the object onto the object. 5. The instruction is input in two-dimensional coordinates, the input instruction is converted into three-dimensional coordinates to generate the instruction data, and the instruction data in the generated three-dimensional coordinates is used for the projector. The work support method according to claim 4, wherein the work support method comprises converting into two-dimensional coordinates. 6. A system for supporting work on an object, means for measuring a three-dimensional shape of the object, means for generating instruction data indicating an instruction for the work based on the measured shape data, and the generated instruction data. And a means for projecting an image corresponding to the object onto the object. 7. A system for supporting work on an object, comprising one or more sets of cameras and projectors,
Measuring means for projecting light from the projector onto the object to measure the three-dimensional shape of the object from the image of the object captured by the camera; and instruction data representing instructions related to the work based on the measured shape data. A work support system comprising: a generation unit that generates the image, and a control unit that controls the projector to project an image corresponding to the generated instruction data onto the object. 8. The generating means includes means for receiving input of the instruction in two-dimensional coordinates, and means for converting the input instruction in two-dimensional coordinates to three-dimensional coordinates to generate the instruction data. 8. The work support system according to claim 7, further comprising: means for converting the generated instruction data in three-dimensional coordinates into two-dimensional coordinates for the projector. 9. The work support system according to claim 7, further comprising a data communication line connecting the measuring unit and the generating unit, and the generating unit and the control unit. 10. A plurality of sets of the camera and the projector are present, and when projecting the image on the object, projection lights of different colors are emitted from each of the plurality of projectors. Work support system described in Crab. 11. A method for assisting a positioning operation of an object having a known shape, wherein image data for projection is generated based on the shape of the object, and the object is positioned by an image corresponding to the generated image data. A work support method characterized by projecting to a position in a real space to be processed. 12. A system for supporting a positioning operation of an object having a known shape, a means for generating image data for projection based on the shape of the object, and an image corresponding to the generated image data, And a means for projecting to the position of the real space to be positioned.