JP2015149011A - Display system, display apparatus and display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display system, a display apparatus and a display method allowing an operator to easily and properly process an object.SOLUTION: The display system includes: an imaging unit 12 that takes an image of a material 50 to acquire image data representing a material 50; a shape data generation unit that generates shape data representing a shape of the material based on the image data of the material acquired by the imaging unit; an operation determination unit that determines a processing operation to be executed on the material based on the shape data of the material generated by the shape data generation unit and component data representing a component to be formed by the processing operation; an image generation unit that generates a guide image for processing operation determined by the operation determination unit; and an image display apparatus 15 that displays the guide image of the processing operation generated by the image generation unit on the material.

Description

  The present invention relates to a display system, a display device, and a display method for displaying a guidance image of processing work performed on an object.

  2. Description of the Related Art Conventionally, an operation content to be performed is displayed as an image that can be visually recognized by an operator so that an operation is appropriately performed on an object. As an example, there is a display device that shoots an object to be worked, and displays a manual (guide image) that guides the work contents and work procedures related to the image of the taken object on a head-mounted display (for example, , See Patent Document 1).

JP 2011-114781 A

  However, such a work procedure is to select and display an image supplied or stored in advance on a display device in accordance with the work situation for an object that is not deformed or moved by the work. In addition, when the worker performs a work that deviates from the work procedure, the work procedure according to the deviated work is not displayed.

  For this reason, in a conventional display device, when performing a processing operation (for example, a cutting operation) involving deformation or movement of an object, it is difficult to guide the operator on the processing operation to be performed on the deformed object. . In addition, it is difficult to guide a worker to a repairable work to be performed on an object deformed by a wrong work out of the work procedure. Alternatively, it has been difficult to guide the operator about the machining operation to be performed on the moved object. Thus, it has been difficult for an operator to perform an appropriate processing operation on an object that involves deformation or movement.

  Such a situation is almost common in at least a display system, a display device, and a display method for displaying a guidance image of a processing operation involving deformation or movement of the object on the object.

  The present invention has been made in view of the above circumstances, and a main object of the present invention is to provide a display system, a display device, and a display method that allow an operator to easily process an object appropriately.

Hereinafter, means for solving the above-described problems and the effects thereof will be described.
A display system that solves the above-described problem is a display system that displays a guide image of a processing operation involving deformation or movement of an object on the object, and image data indicating the object by imaging the object An imaging unit to be acquired, a shape data generation unit that generates shape data indicating the shape of the target object from image data of the target object acquired by the imaging unit, and a shape of the target object generated by the shape data generation unit Based on the data and part data indicating the part to be formed by the machining operation, a work determination unit for determining a machining operation to be performed on the target object, and the guidance of the machining operation determined by the work determination unit An image generation unit that generates an image, and an image display unit that displays a guide image of the processing work generated by the image generation unit on the object.

  According to this configuration, since it is possible to display and guide a processing operation according to the shape of the target object after deformation or movement on the target object, it is easy for the operator to process the target object appropriately. It is.

  In the display system, a processed shape data storage unit that stores processed shape data of the object when processed according to the processing operation determined by the operation determination unit, and the shape data generation unit generates the processed data A shape determination unit that determines whether or not the shape data matches the processed shape data, and the work determination unit is configured so that the shape determination unit does not match the shape data and the processed shape data. When it is determined, it is preferable to determine, as a processing operation to be performed on the object, a correction processing operation obtained by correcting the processing operation determined when the processing is performed according to the guide image.

  According to this configuration, when the machining operation performed is wrong, the next machining operation can be corrected and guided, so that the operator can easily perform an appropriate machining operation to repair the machining error. Can do.

  In the display system, the imaging unit acquires image data indicating a position and a posture of a work tool used for the processing work, and starts a processing work on the object using the acquired image data of the work tool. A start detection unit that detects time, and a work determination unit that determines whether the processing operation detected by the start detection unit is the processing operation according to the guide image displayed by the image display unit. And when the work determination unit determines that the non-working work is not a processing work according to the guide image, the image generation unit generates a warning image corresponding to the non-working work, and displays the image The unit preferably displays the warning image on the object.

According to this configuration, an operator can easily process an object appropriately by a warning displayed when starting a processing different from the guided processing operation.
In the display system, the imaging unit captures an object at predetermined time intervals to sequentially acquire image data of the object, and the image generation unit stores the image data of the object acquired this time. When the image data of the object acquired last time is different, the guide image is corrected based on the image data of the object acquired this time, and the image display unit displays the corrected guide image as the image data. It is preferable to display it superimposed on the object.

  According to this configuration, since the guide image indicating the position to be actually processed with respect to the moved object is displayed superimposed on the object, the operator moves the object during the machining operation. However, it is easy to process the object appropriately.

  In the display system, the processing operation is a cutting operation in which the object is cut into a plurality of pieces, and the state data generation unit generates shape data of each of the plurality of objects separated by the cutting. It is preferable that the work determination unit determines a work to be performed on the plurality of separated objects based on the generated shape data.

  According to this configuration, even if the object is separated into a plurality of parts by the machining operation, it is possible to display a guidance image of the machining operation for each separated object. Therefore, the worker can easily perform an appropriate processing operation.

  A display device that solves the above problem is a display device that displays a guidance image of a processing operation involving deformation or movement of an object on the object, and obtains shape data of the object from image data indicating the object. Based on the shape data generation unit to be generated, the shape data of the object generated by the shape data generation unit, a work determination unit that determines a machining operation to be performed on the target object, and the work determination unit determined An image generation unit that generates a processing work guidance image, and an image display unit that displays the processing work guidance image generated by the image generation unit on the object.

According to this configuration, it is possible to guide an appropriate processing operation according to the shape of the object with an image, so that the operator can easily process the object appropriately.
A display method for solving the above problem is a display method for displaying a guidance image of a processing operation involving deformation or movement of an object on the object, and image data indicating the object by imaging the object. A step of acquiring, a step of generating shape data indicating the shape of the object from the acquired image data of the object, a shape data of the generated object, and a component indicating a part to be formed by the processing operation A step of determining a machining operation to be performed on the object based on the data, a step of generating a guidance image of the determined machining operation, and displaying the generated guidance image of the machining operation on the object And a step of.

  According to this method, an appropriate processing operation according to the shape of the object can be guided by an image, so that the operator can easily process the object appropriately.

The block diagram which shows the display system of embodiment. The functional block diagram of the display system of embodiment. The flowchart which shows the preparatory work process performed in a display system. (A) is a perspective view showing a finished product, (b) is a perspective view showing parts constituting the finished product, and (c) is a table showing part data input to the display system. (A) is a plan view of a material to be cut and a guide image of the cutting operation, (b) is an explanatory diagram showing a state in which the guide image is superimposed on the material, and (c) is cut according to the guide image. The top view which shows the external shape of the material after having left. The flowchart which shows the main work process of the cutting | disconnection performed by a display system. The flowchart which shows the confirmation process of the correctness of the cutting operation | work at the time of a cutting | disconnection start. The perspective view which shows an example of the warning image displayed when a cutting position differs from the position which a guidance image shows. The flowchart which shows the projection process of the guidance image after a cutting | disconnection start. The flowchart which shows the projection process of the guidance image after completion | finish of cutting | disconnection. (A) is a top view which shows an example in which material is cut | disconnected in the wrong position different from a guide image, (b) is a top view which shows the guide image projected on each material cut | disconnected by mistake. (A) is a block diagram which shows a part of display system which displays the guidance image of a cutting operation on a head mounted display, (b) is a schematic diagram which shows the image of the material which the camera with which a head mounted display is equipped, (c). FIG. 3 is a schematic diagram showing a visual field region of a head mounted display.

Hereinafter, an embodiment of a display system will be described with reference to the drawings.
As shown in FIG. 1, the display system 11 of the present embodiment includes four imaging units 12 capable of imaging a material 50 as a processing target installed in a work place, and a data input device 14 such as a keyboard and a mouse. And a control device 13 to which a display monitor is connected, and an image display unit 15 such as a projector for projecting and displaying the input image. In the present embodiment, the material 50 is a plate made of wood, resin, or metal having a predetermined thickness, and the display system 11 performs a cutting operation as a processing operation using the saw 16 as a work tool. At this time, a display image such as a guide image GP (see FIG. 5A) showing the cutting line CL is displayed on the main surface (upper surface) of the material 50.

  Each imaging unit 12 receives an infrared ray or laser light emitted from the emitting unit 12a by the light receiving unit 12b, and can measure a distance (depth) from the light receiving unit 12b (imaging unit 12) to each part of the material 50. And a digital camera 12c that projects a color image. Then, by imaging the material 50, R, G, B image data including depth data of each part of the material 50 is acquired. The image data indicating the material 50 is three-dimensional data indicated by the coordinate values of the X, Y, and Z axes, which are three-dimensional orthogonal coordinate axes in the workplace, and data (images) indicating the position and orientation of the material 50 in the workplace. Data). In the present embodiment, the imaging units 12 are installed at the four corners of the work place, respectively, and by capturing images of the material 50 from four different positions, it is possible to suppress missing image data caused by the material 50 being hidden by the operator.

  The control device 13 uses the image data indicating the material 50 output from each imaging unit 12 (also referred to as “image data of the material 50”), and the spatial region indicated by the X, Y, and Z coordinate values of the work place. The shape data of the material 50 placed at a predetermined position is generated. That is, the control device 13 calculates using the image data of the material 50 that is captured and acquired by each imaging unit 12, and the projection surface (main surface) on which the display image is projected with respect to the material 50 located in the space area of the work place ) Is generated. And the control apparatus 13 produces | generates and outputs the display image displayed on the material 50 by making it project on the image display part 15 according to the shape data of the produced | generated material 50. FIG. The control device 13 is a so-called computer including a central processing element (CPU) and a storage element (RAM, ROM).

  The image display unit 15 is a projector (projector) in the present embodiment, and projects the display image output from the control device 13. The display image projected at this time is an image corresponding to the position and orientation of the material 50 viewed from the installation position of the image display unit 15, and the image display unit 15 is fixed without changing the installation position. The display image is projected from the set position and displayed on the material 50. Therefore, the control device 13 and the image display unit 15 function as a display device.

  Further, in the display system 11 of the present embodiment, the imaging unit 12 images the saw 16 as a work tool used by the worker in the cutting work, and acquires image data of the saw 16 indicating its position and posture. Then, the control device 13 detects a contact position where the saw 16 and the material 50 are in contact using the acquired image data of the saw 16 and outputs a predetermined display image to the image display unit 15 according to the detected contact position. To do. The image display unit 15 displays the display image output from the control device 13 on the material 50.

Next, the functional configuration of the control device 13 in the display system 11 will be described.
As shown in FIG. 2, the control device 13 includes a shape data generation unit 31, a work determination unit 32, an image generation unit 33, a shape determination unit 34, a start detection unit 35, a work determination unit 36, and a component data storage unit 37. A post-processing shape data storage unit 38 is provided. Each unit functions when the control device 13 operates according to a predetermined program stored in the storage element.

  The shape data generation unit 31 calculates and generates shape data of the material 50 from image data indicating the position and orientation of the material 50 acquired by the imaging unit 12. The work determination unit 32 is based on the shape data of the material 50 generated by the shape data generation unit, and the part data including the shape data and quantity data of the part to be formed input to the part data storage unit 37 by the input device 14. Then, by laying out the parts on the main surface (projection surface) of the material 50, the cutting operation to be performed on the material 50 is determined. Then, the shape of the main surface after cutting of the material 50 to be cut according to the determined cutting operation is stored in the processed shape data storage unit 38. The image generation unit 33 generates a guide image GP that guides the cutting work determined by the work determination unit 32, or corrects the generated guide image GP. Then, the generated or corrected guide image GP is output to the image display unit 15 as a display image (display image data).

  The shape determination unit 34 compares the shape data generated by the shape data generation unit 31 with the processed shape data stored in the processed shape data storage unit 38, and compares the comparison result with the work determination unit 32 or the image generation unit. To 33. In addition, the shape determination unit 34 compares the shape data generated by the shape data generation unit 31 with the shape data of the component stored in the component data storage unit 37, and stores the shape data in the component data storage unit 37 according to the comparison result. Rewrite the stored component data.

  In addition, the start detection unit 35 uses the image data of the material 50 acquired by the imaging unit 12 and the position data of the saw 16 to detect the state in which the saw 16 is in contact with the material 50 as the start of work. The work determination unit 36 compares the contact position of the saw 16 with the position of the cutting line CL shown in the guide image GP, that is, the cutting position when starting work, and outputs the comparison result to the image generation unit 33. Then, the image generation unit 33 generates a predetermined warning image according to the comparison result, and outputs the generated warning image to the image display unit 15 as a display image (display image data).

  Next, the operation of the display system 11 of the present embodiment configured as described above, that is, the display operation in the cutting operation of the material 50 will be described. In the present embodiment, a predetermined preparatory work process is performed before the actual work for actually cutting is started. Hereinafter, the preparation work process and the main work process will be described in this order.

(Preparation work processing)
As shown in FIG. 3, in the preparatory work process, first, in step S1, shape data and quantity data of all parts are input. In this process, for example, the operator inputs the part data of all the parts to be completed by the cutting work, that is, the shape data and the quantity data into the input table displayed on the display monitor using the input device 14. . The input component data is stored in the component data storage unit 37 in the control device 13.

  As shown in FIG. 4A, in the present embodiment, a wood deck that is a finished product is created as an example, and the part data is input for all the parts constituting the wood deck. Here, this wood deck is composed of three large-sized parts A (three), two medium-sized parts B (two), and three small-sized parts C (three). Is done.

  As shown in FIG. 4B, the part A, the part B, and the part C are rectangular plates having the same plate thickness and having a longitudinal direction. Therefore, in the present embodiment, for the parts A, B, and C, the dimensions of the long side L and the short side W and the value of the number N are stored in the part data storage unit 37 as part data.

  FIG. 4C shows an example of component data stored in the component data storage unit 37. As illustrated, in this embodiment, the long side L of the component A and the component B is the same length (190 centimeters), and the short side W of the component A and the long side L of the component C are the same length. (60 centimeters). Further, the parts B and C have the same short side W (10 centimeters). The number N of parts A is 3 (sheets), the number N of parts B is 2 (sheets), the number N of parts C is 3 (sheets), and the number N is incomplete when it is input to the input table. Is a number.

  Returning to FIG. 3, in the next step S2, the position and orientation of the material to be cut are acquired. Here, the image capturing unit 12 captures an image of the material 50 (see FIG. 1) to be cut placed on the work site, and acquires image data indicating the position and orientation of the material. Next, in step S3, shape data of the material 50 is generated. Here, the material 50 is placed so that the plate surface (main surface) direction is substantially horizontal, and the shape data generation unit 31 determines from the image data of the material 50 the upper surface (reverse surface) that becomes the main surface (projection surface). The shape data indicating the shape of the surface in the direction of gravity) is calculated and generated.

  Next, in step S4, layout processing is performed for all parts on the material using the shape data of the material and the part data of all parts. Here, the work determination unit 32 can lay out all the parts on the material 50 using one of the well-known methods disclosed as, for example, the “rectangular filling problem”, and therefore, the specifics of the layout processing are described. The detailed explanation is omitted.

  Next, in step S5, a guide image GP (see FIG. 5A) showing the cutting position between the laid out parts is generated, and in step S6, the guide image is matched with the position and orientation of the material. Is corrected. Here, the image generation unit 33 generates a guide image GP indicating the cutting position of the material 50 based on the position of each laid-out component, and further displays the generated guide image GP superimposed on the material 50. Correct as follows. That is, the image generation unit 33 calculates the position and orientation of the material 50 viewed from the position of the image display unit 15 from the image data of the material 50, and corrects the guide image GP according to the calculated position and orientation of the material 50. To do. In step S7, the corrected guide image GP is projected onto the material 50. The image generation unit 33 outputs the corrected guide image GP as a display image to the image display unit 15, and the image display unit 15 projects the corrected guide image.

  FIG. 5A shows an example of the generated guide image GP. The guide image GP is an image having a plurality of cutting lines CL between the laid out components. In the present embodiment, the material 50 to be processed is a rectangular plate having a length of one side H1 of 200 centimeters and a length of the other side H2 of 250 centimeters. The guide image GP is an image having a total of eight cutting lines CL including the cutting lines CL1 to CL8, and each cutting line CL is an image indicated by a broken straight line. Then, in a state where the guide image GP is projected so as to be superimposed on the entire upper surface of the material 50, each cutting line CL is displayed in a state where the component A, the component B, and the component C are laid out on the material 50.

  Then, as shown in FIG. 5B, for example, if the shape of the material 50 viewed from the image display unit 15 is a trapezoidal posture, the image generation unit 33 displays the image indicating each cutting line CL as The guide image GP is deformed and corrected so as to be similar to a trapezoid and is output to the image display unit 15. Further, in the present embodiment, the image display unit 15 is configured to project an image within a projection range determined from the installation position, and the image generation unit 33 includes, in addition to the deformation correction of the guide image GP, within the projection range. Movement correction for moving the guide image GP to a position corresponding to the position of the material 50 is performed. As a result, the guide image GP projected from the image display unit 15 is projected such that each cutting line CL is displayed so as to be superimposed on the entire upper surface of the trapezoidal material 50 (see FIG. 1).

  Returning to FIG. 3, in the next step S8, the shape of the material when cut at the cutting position is stored as processed shape data, and the pre-operation preparation processing ends. Here, the shape data after processing of the material 50 formed when the work determination unit 32 is cut according to the cutting line CL of the guide image GP in a state where the parts are laid out, is processed shape data storage unit 38.

  FIG. 5C shows post-processing shape data K1 to post-processing shape data K10 stored in the post-processing shape data storage unit 38. As shown in the drawing, the processed shape data K1 to the processed shape data K10 are shapes that are separately formed when cut along a plurality of cutting lines CL of the guide image GP, and are at least one cutting line. All shapes with CL.

(This work process)
Next, with reference to FIG. 6, the main work process for cutting the material 50 will be described. This process is started in the display system 11 when the saw 16 is imaged by the imaging unit 12 and the image data is output to the control device 13. Alternatively, the operation is started when the operator inputs processing start command data from the input device 14 to the control device 13.

  As shown in FIG. 6, when this process is started, first, in step S10, a process for confirming the correctness of the work at the start of cutting is performed. This process is a process for suppressing an operator's work mistake (cutting mistake).

  That is, as shown in FIG. 7, in step S10, first, in step S11, processing for obtaining the position and orientation of the saw is performed. Here, similarly to the process of acquiring the position and orientation of the material 50 (FIG. 3, step S2), the imaging unit 12 images the saw 16 and acquires image data indicating the position and orientation of the saw 16.

  In subsequent step S12, it is determined whether or not the saw and the material have contacted each other. Here, the start detection unit 35 calculates the position of the tooth portion of the saw 16 using the image data of the material 50 and the image data of the saw 16 acquired in step S2, and the tooth 16 of the calculated saw 16 is calculated. It is determined whether the portion has contacted the material 50.

  As a result of the determination process in step S12, when the saw 16 and the material 50 are not in contact (step S12: NO), the processes in steps S11 and S12 are repeated at predetermined time intervals. When the saw 16 and the material 50 are in contact (step S12: YES), in the subsequent step S13, it is determined whether or not the position where the saw has contacted the material is a cutting position of the guide image. Here, the work determination unit 36 determines whether or not the position data of the contact position between the saw 16 and the material 50 matches the position data of the cutting line CL of the guide image GP projected on the material 50 in a superimposed manner. Determine.

  If the result of determination is that the saw 16 has contacted at the cutting position of the guide image GP (step S13: YES), the processing here is terminated and the routine proceeds to step S20 (FIG. 6). On the other hand, when the saw 16 contacts at a position different from the cutting position of the guide image GP (step S13: NO), the warning image is projected onto the material 50 in step S14, and the process is terminated. Proceed to step S20.

  FIG. 8 shows an example of a projected warning image. As shown in FIG. 8, when the contact position P where the saw 16 contacts the material 50 is a position different from the position of the cutting line CL, the image display unit 15 displays the main surface of the material 50 in the vicinity of the saw 16. The warning image is projected on the (upper surface) so as to be superimposed on the guide image GP. In the present embodiment, the warning image is an image including the word “caution”. In the case where the wording is displayed in this way, an image in which the wording is upright when viewed from the operator is preferable. Note that the position of the operator can be estimated from, for example, the posture (handle position) of the saw 16.

  Returning to FIG. 6, next, in step S <b> 20, a guide image is projected according to the position and posture of the material after the start of cutting. In this process, when the operator moves the material 50 during the cutting operation, a display is made so that the cutting line CL indicating the cutting position does not move relative to the material 50, thereby causing an operation error (cutting error). Is a process of suppressing

  That is, as shown in FIG. 9, in the process in step S20, first, in step S21, the position and orientation of the material are acquired. Here, similarly to the preparatory work process (FIG. 3, step S2), the imaging unit 12 images the material 50 being cut by the cutting work, and acquires image data indicating the position and orientation of the material 50. Next, in step S22, material shape data is generated. Here, the shape data generation unit 31 calculates and generates shape data indicating the shape of the main surface as the upper surface from the image data of the material 50.

  Next, in step S23, it is determined whether or not the shape of the material is the same. Here, the work determination unit 32 performs the process of step S3 in the shape data of the material 50 generated in the current process in step S22 and the shape data of the material 50 generated in the previous process, that is, in the first process. It is determined whether or not the shape data generated in this way matches.

  If the shape of the material 50 is not the same as a result of the determination (step S23: NO), the material 50 has been cut by the cutting operation, so the processing here ends (returns) and the process proceeds to step S30. To do. On the other hand, as a result of the determination, if the shape of the material 50 is the same (step S23: YES), the material 50 is in an uncut state, so that the process proceeds to step S24, and is the same as the position and posture of the material acquired last time. It is determined whether or not. Here, the image generation unit 33 and the image data acquired in the current process in step S21 and the image data of the material 50 acquired in the previous process, that is, the image acquired in step S2 in the first process. It is determined whether or not the data matches.

  As a result of the determination, if the image data of the material 50 is the same (step S24: YES), the material 50 is not moved in the cutting operation, so the process returns to step S21 as it is, and the processing from step S21 to step S23 is repeated. . On the other hand, if the image data of the material 50 is different as a result of the determination (step S23: NO), since the material 50 is in a state of being moved during the cutting operation, the process proceeds to step S25 to match the acquired position and posture of the material. To correct the guidance image. Then, the guide image corrected in the subsequent step S26 is projected onto the material, and the process returns to step S21.

  In step S25 and step S26, the image generation unit 33 performs the same processing as in step S6 and step S7. That is, as shown in FIG. 5B, the guide image GP corresponding to the position and orientation of the material 50 within the projection range of the image display unit 15 so that the guide image GP is displayed superimposed on the moved material 50. Movement correction and deformation correction. Therefore, the guide image GP is superimposed and displayed following the material 50 that moves during the cutting operation, and the state in which the cutting line CL is appropriately displayed on the material 50 is maintained.

  If the material shape data generated in the previous process in step S22 is different from the material shape data generated in the current process in the repetition process from step S21, the determination is performed in the determination process in step S23. Is set to “NO”, and the process proceeds to step S30.

  Returning to FIG. 6, next, in step S <b> 30, a guide image projection process is performed in accordance with the position and orientation of the material after the end of cutting. This process is a process for displaying a guide image GP for guiding the next cutting operation, that is, the cutting position, for the material 50 that has been cut and deformed when the operator cuts the material 50 by the cutting operation. .

  That is, as shown in FIG. 10, in the processing in step S30, first, in step S31, the shape of the cut material is compared with the shape of the unfinished part, and then in step S32, the cutting is performed. It is determined whether or not the shape of the formed material is the same as the shape of the unfinished part. Here, the shape determination unit 34 compares the shape data of the material 50 generated in the process of step S22 with the component data (size data) stored in the component data storage unit 37, and the components A, B, It is determined whether or not it has the same shape as any one of the parts C. Note that the material 50 is deformed by the cutting operation, that is, separated into a plurality (usually two), and the processing of step S31 and step S32 is performed for each material 50 separated by cutting.

  As a result of the determination, when the shape of the material 50 separated (deformed) by cutting is the same shape as any part (step S32: YES), the part data of the part having the same shape is not completed in step S33. The change process is completed, and the process proceeds to step S34. Here, the shape determination unit 34 performs a process of reducing the value of the number N indicating the incomplete number among the component data stored in the component data storage unit 37 by one. For example, if the shape of the cut material is the same as the shape of the part A, the number N of the parts A is rewritten from the value “3” to the value “2”.

  On the other hand, as a result of the determination, if the shape of the material 50 separated (deformed) by cutting is different from the shape of any part (step S32: NO), the process proceeds to step S34, and the position where the material is cut is the cutting of the guide image. Judgment processing is performed to determine whether the position is correct. Here, the shape determination unit 34 compares the shape data of the material 50 generated in step S22 with each of the processed shape data K1 to the processed shape data K10 stored in the processed shape data storage unit 38. Depending on whether or not the same shape data exists, it is determined whether or not the position where the material 50 is cut is the same as the cutting position of the guide image GP.

  As a result of the determination, if the cutting position is the same as the cutting position of the guide image GP (step S34: YES), the process proceeds to step S35, the guide image is divided at the cut position, and the process proceeds to step S38. Here, the image generation unit 33 divides the guide image GP projected onto the material 50 before cutting by the cutting line CL at the position cut by the cutting work, and the divided images are separated into the respective materials. 50 is generated as a guide image GP to be projected onto 50.

  On the other hand, as a result of the determination, if the cutting position is not the same as the cutting position of the guide image GP (step S34: NO), the process proceeds to step S36, and layout processing is performed on the unfinished parts in accordance with the shape of the cut material. Here, the work determination unit 32 lays out (re-lays out) the incomplete parts using the same method as the processing in step S4. In subsequent step S37, a guide image indicating a cutting position between the laid out incomplete parts is generated, and the process proceeds to step S38. Processing in step S36 and step S37 will be described with reference to the drawings.

  As shown in FIG. 11A, for example, the cutting operation by the operator is the first time, and the saw 16 contacts the cutting line CL2 displayed on the material 50 from the contact position P along the cutting line CL2. Cutting shall be performed. At this time, the cutting line CL2 is a line that ends in the middle of the material 50 up to the contact point EP with the cutting line CL1, but the operator cuts the material 50 by mistake as indicated by the solid line arrow T, so that the material 50 becomes two. When separated, the cut position is not the same as the cut position indicated by the guide image GP.

  As shown in FIG. 11B, in such a case, the work determination unit 32 distributes all unfinished parts to the material 51 and the material 52 separated by cutting, and re-lays out. That is, two parts A and two parts B are laid out in the material 51, and one part A and three parts C are laid out in the material 52. Then, the image generation unit 33 newly generates a guide image GP1 having a cutting line CL11 to a cutting line CL14 and a guide image GP2 having a cutting line CL21 to a cutting line CL24 as new guide images GP. In this example, the cutting line CL11 and the cutting line CL21 are at the same cutting position as the cutting line CL1, and the cutting line CL12 is at the same cutting position as the cutting line CL3.

  Returning to FIG. 10, in the subsequent step S38, the position and orientation of the material to be cut next are acquired. Here, it is assumed that at least one of the material 51 and the material 52 is placed in the work place. Therefore, the material 51 or the material 52 placed in the work place becomes a new material 50 to be cut, and the image pickup unit 12 images the material 50 and obtains image data indicating the position and orientation as in the process of step S2. get. Next, in step S39, the guide image GP is corrected in accordance with the position and orientation of the material 50 to be cut, and in step S40, the corrected guide image is projected on the material to be cut. These processes are the same as steps S6 and S7 in FIG. The process here is completed (return), and the process proceeds to step S45.

  Returning to FIG. 6, in step S45, it is determined whether or not all the parts are completed. Here, the shape determination unit 34 determines whether or not the value N of the number of component data stored in the component data storage unit 37 is “zero” for all components. If the value of the number N is not “zero” for all parts (step S45: NO), the process returns to step S10 because there is an unfinished part, and the processes of steps S10 to S45 are repeated.

  In this iterative process, if the value of the number N is “zero” for all parts (step S45: YES), all parts have been completed, so the main work in the display system 11 of this embodiment is performed. finish. In addition, although description is abbreviate | omitted here, the process of step S10-step S45 is performed about all the materials which were isolate | separated by the cutting operation and one or more cutting lines CL are displayed.

According to the above embodiment, the following effects can be obtained.
(1) Since the cutting line CL corresponding to the shape after deformation or movement of the material 50 can be displayed and guided on the material 50 as an image, the operator can easily cut the material 50 appropriately. is there.

  (2) If the cut position is wrong, the next cutting position can be corrected and guided, so that the operator can easily perform an appropriate cutting operation to repair the cutting error.

(3) The operator can easily cut the material 50 appropriately by a warning displayed when starting a cutting operation at a position different from the guided cutting position (the position of the cutting line CL).
(4) Since the guide image GP indicating the position (the position of the cutting line CL) to be actually cut with respect to the moved material 50 is displayed superimposed on the material 50, the operator can cut the material 50 Even if it moves in, it is easy to cut the material 50 appropriately.

  (5) Even if the material 50 is separated into a plurality by the cutting operation, it is possible to display the guide image GP indicating the cutting line CL for each separated material 50 (materials 51 and 52). Therefore, the worker can easily perform an appropriate cutting operation.

In addition, you may change the said embodiment as follows.
In the above embodiment, the image display unit 15 is not necessarily limited to a projector. For example, a head mounted display may be adopted. This modification will be described with reference to the drawings.

  As shown in FIG. 12A, in this modification, the worker wears a head mounted display 18 including an imaging camera (digital camera) 17 capable of capturing a color image as the image display unit 15. The head mounted display 18 has a so-called transflective display body in which the material 50 can be directly visually recognized and an image is displayed in a state where the material 50 is overlapped with the material 50. In FIG. 12A, some components different from the display system 11 shown in FIG. 1 are shown, and the other components are not shown.

  In the present modification, the control device 13 corrects the generated guide image GP according to the position and shape of the image of the material 50 captured by the imaging camera 17 and outputs the corrected guide image GP to the head mounted display 18. The head mounted display 18 displays the guide image GP so that the cutting line CL is superimposed on the material 50 when the operator looks at the material 50.

  That is, as shown in FIG. 12B, the position and shape of the material 50 in the camera image 17G are specified by the camera image 17G obtained by imaging the material 50 by the imaging camera 17. The control device 13 corrects the guide image GP by moving and deforming it so as to match the position and shape of the specified material.

  Then, as shown in FIG. 12C, the corrected guide image GP is displayed so as to overlap the visually recognized trapezoidal material 50 in the visual field region 18G of the operator in the head mounted display 18. A cutting line CL is displayed on the upper surface of 50. In the present modification, the image area of the camera image 17G and the visual field area 18G are preferably the same area. By doing so, the processing load accompanying the correction of the guide image GP for the position and shape of the material 50 is reduced.

  In the embodiment described above, the image display unit 15 that is a projector may include the imaging camera 17 according to the modification. In this case, the image generation unit 33 may correct the guide image GP based on the image of the material 50 captured by the imaging camera 17 instead of the image data of the material 50 acquired by the imaging unit 12.

  -In the said embodiment, the material 50 as a target object may not necessarily be a board | plate material but a block-shaped material. In that case, the surface on which the cutting line CL is displayed may be a plurality of surfaces instead of a single surface.

  In the above embodiment, when the control device 13 lays out the component, for example, when the material 50 is wood, the direction along the plate is set as the longitudinal direction of the component, and the wood grain is taken into consideration. You may make it lay out. Further, when the material 50 is a stretched material, the layout may be made in consideration of the stretching direction, for example, the stretching direction is the longitudinal direction of the component.

  In the above embodiment, the number of imaging units 12 is not necessarily limited to four. The number may be less than four or more than four as long as the image data of the material 50 can be acquired without missing.

  In the above embodiment, a plurality of projectors as the image display unit 15 may be provided. Moreover, when providing with two or more, it is preferable to install in a mutually different position. By doing so, the cutting line CL is displayed on the material 50 without being hidden by the saw 16 or the operator.

  In the above embodiment, the processing operation is not necessarily limited to the cutting operation in which the material 50 is cut and separated into a plurality of parts. For example, it may be a processing operation in which the original shape is deformed by cutting the material 50, making a hole, or melting the material 50. In such a case, since the material 50 is not separated into a plurality of parts by the processing operation, the image display unit 15 displays the guide image GP for one material.

  In the above embodiment, the imaging unit 12 may not necessarily acquire the image data of the material 50 sequentially by imaging the material 50 at a predetermined time interval. For example, when the material 50 to be cut is placed in a fixed state in the work place and does not move during cutting, it is not necessary to correct the projected guide image GP, and thus the material 50 is photographed at a predetermined time interval. You don't have to. In this case, the operator inputs predetermined data to the control device 13 by using the input device 14 at each end point of the cutting, and the control device 13 is based on the input of the predetermined data. The material 50 may be imaged to obtain image data of the material 50.

  In the above embodiment, the image display unit 15 does not necessarily have to display the warning image including the wording on the material 50. For example, the warning image may be obtained by changing the display color of the cutting line CL or blinking the display. Alternatively, the image display unit 15 may not project the warning image. For example, when an operator performs a cutting operation without making a mistake in the cutting position, it is not necessary to display a warning image. In this case, the start detection unit 35 and the work determination unit 36 are unnecessary, and the control device 13 may be omitted.

  In the above-described embodiment, the work determination unit 32 determines the corrected cutting work, which is a modification of the cutting work determined when cutting is performed according to the cutting line CL indicated by the guide image GP, as the next cutting work to be performed on the material 50. It is not always necessary to perform the processing. For example, when the operator performs the cutting operation without making a mistake in the cutting position, the correction cutting operation is not necessary. In this case, the guide image GP generated to project onto the separated material is an image corresponding to one of the materials of each processed shape stored in the processed shape data storage unit 38. Thus, an image having at least one cutting line CL is obtained.

  DESCRIPTION OF SYMBOLS 11 ... Display system, 12 ... Imaging part, 13 ... Control apparatus, 15 ... Image display part, 31 ... Shape data generation part, 32 ... Work determination part, 33 ... Image generation part, 34 ... Shape determination part, 35 ... Start detection Part, 36 ... work determination part, 38 ... processed shape data storage part, A, B, C ... parts, GP, GP1, GP2 ... guide image, K1 to K10 ... processed shape data.

Claims (7)

A display system for displaying a guidance image of a processing operation involving deformation or movement of an object on the object,
An imaging unit that captures the object and acquires image data indicating the object;
A shape data generation unit that generates shape data indicating the shape of the object from the image data of the object acquired by the imaging unit;
A work determination unit that determines a machining operation to be performed on the target object based on the shape data of the object generated by the shape data generation unit and component data indicating a component to be formed by the machining operation; ,
An image generation unit for generating a guide image of the processing work determined by the work determination unit;
An image display unit for displaying a guide image of the processing work generated by the image generation unit on the object;
A display system comprising: A post-processing shape data storage unit that stores post-processing shape data of the object when processed according to the processing operation determined by the work determination unit;
A shape determination unit that determines whether the shape data generated by the shape data generation unit matches the processed shape data;
With
The work determination unit
When the shape determination unit determines that the shape data and the processed shape data do not match, the object is subjected to a correction processing operation that corrects a processing operation determined when the shape data is processed according to the guide image. The display system according to claim 1, wherein the display system is determined as a power machining operation. The imaging unit obtains image data indicating the position and posture of a work tool used for the processing work,
A start detection unit for detecting the start of the processing operation on the object using the acquired image data of the work tool;
A work determination unit that determines whether the processing operation detected by the start detection unit is the processing operation according to the guide image displayed by the image display unit;
With
When it is determined that the work determination unit is a non-processing work that is not a processing work according to the guide image,
The image generation unit generates a warning image according to the non-processing work,
The display system according to claim 1, wherein the image display unit displays the warning image on the object. The imaging unit captures an object at predetermined time intervals to sequentially acquire image data of the object,
When the image data of the target object acquired this time is different from the image data of the target object acquired last time, the image generation unit displays the guide image based on the image data of the target object acquired this time. Correct,
The display system according to claim 1, wherein the image display unit displays the corrected guide image superimposed on the object. The processing operation is a cutting operation for cutting the object and separating it into a plurality of pieces,
The shape data generation unit generates shape data of each of a plurality of objects separated by cutting,
The display system according to any one of claims 1 to 4, wherein the work determination unit determines a processing work to be performed on the plurality of separated objects based on each of the generated shape data. A display device that displays a guidance image of a processing operation involving deformation or movement of an object on the object,
A shape data generation unit that generates shape data of the object from image data indicating the object;
Based on the shape data of the object generated by the shape data generation unit, a work determination unit that determines a machining operation to be performed on the object;
An image generation unit for generating a guide image of the processing work determined by the work determination unit;
An image display unit for displaying a guide image of the processing work generated by the image generation unit on the object;
A display device comprising: A display method for displaying a guidance image of a processing operation involving deformation or movement of an object on the object,
Capturing the image of the object and obtaining image data indicating the object;
Generating shape data indicating the shape of the object from the acquired image data of the object;
Determining a processing operation to be performed on the target object based on the generated shape data of the target object and part data indicating a part to be formed by the processing operation;
Generating a guide image of the determined machining operation;
Displaying the generated guidance image of the processing work on the object;
A display method comprising: