JP2014065060A - Supporting system, and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a supporting system using an AR (augmented reality) technique thereby making bending work preparation efficient and to improve the safety of works.SOLUTION: A supporting system supports the preparation of bending work or a bending operation. The supporting system includes a device made wearable on a human body for recognizing an AR marker and for displaying an image responding to the recognized AR marker on a display. The image relates to a preparation for the bending work or a bending operation, or displays of an image visibly at the preparation or the bending work.

Description

  The present invention relates to a support system and a method thereof, and relates to a support system and a method for improving the safety of work while improving the efficiency of bending setup using AR (augmented reality) technology.

  [1] relates to setup, and [2] relates to bending work.

  [1] In the conventional bending process setup, the operator determines the mold to be installed from the mold number, work instruction sheet, screen display of the processing machine, etc. and removes it from the mold rack, and displays the position by screen display or abutment. It was attached to the processing machine while receiving.

  [2] In the conventional bending process, the bending process is performed by confirming the abutting position / abutting position of the part from the screen display of the bending machine or the abutting position.

  See Patent Documents 1 and 2.

JP2010-200894 JP 2010-259497 A

  [1] relates to setup, and [2] relates to bending work.

  [1] Since it is difficult to work while viewing information on the screen display or work instruction sheet, the work is temporarily stopped while the information is being confirmed. In addition, when a die different from the instruction is attached due to an incorrect reading of the number, or when the direction of the die (front and back) is wrong, a processing failure, a bending machine failure, or a serious accident occurs. There is a fear.

  [2] It is difficult to perform work while viewing information on the work content. Due to factors such as the fact that the information on the work contents is simple, there is a problem that the front and back of the material to be processed and the processing location are mistaken, resulting in processing failures and accidents.

  It is an object of the present invention to improve the safety of work while improving the efficiency of the bending setup by using the AR technology.

The present invention is for solving the above-mentioned problems, and the invention according to claim 1 is a support system for supporting the setup of bending work.
A device that is configured to be wearable on a human body, recognizes an AR marker, and displays an image corresponding to the recognized AR marker on a display (sometimes simply referred to as a head-mounted display),
The image relates to a setup for bending, and the image is displayed on the display so as to be visible at the time of setup.

The invention according to claim 2 is a support system for supporting bending work of bending work.
A device that is configured to be wearable on a human body, recognizes an AR marker, and displays an image corresponding to the recognized AR marker on a display (sometimes simply referred to as a head-mounted display),
The image relates to a bending operation, and the image is displayed on the display so as to be visible during the bending operation.

  The invention according to claim 3 is characterized in that the AR marker is arranged at a plurality of positions of a bending machine.

  The invention according to claim 4 is characterized in that images corresponding to a plurality of bending processes are displayed for each process.

  The invention according to claim 5 is characterized in that the image is information in which polygon data and annotation data are associated with each other.

The invention according to claim 6 is the support method for supporting the setup of the bending process.
A device configured to be wearable on a human body, including an apparatus that recognizes an AR marker and displays an image corresponding to the recognized AR marker on a display (sometimes simply referred to as a head-mounted display);
The image relates to a setup for bending, and the image is displayed on the display so as to be visible at the time of setup.

The invention according to claim 7 is a support method for supporting bending work of bending work.
A device configured to be wearable on a human body, including an apparatus that recognizes an AR marker and displays an image corresponding to the recognized AR marker on a display (sometimes simply referred to as a head-mounted display);
The image relates to a bending operation, and the image is displayed on the display so as to be visible during the bending operation.

  [1] relates to setup, and [2] relates to bending work.

  [1] According to the present invention, it is possible to smoothly perform work setup in bending work, and in addition, there is an effect that mistakes and accidents can be prevented.

  [2] The bending work can be performed smoothly, and in addition, there is an effect that mistakes and accidents can be prevented in the work.

It is explanatory drawing explaining a support system. It is explanatory drawing explaining the AR marker for bending machines. It is explanatory drawing explaining the AR marker for bending machines. It is explanatory drawing explaining the AR marker for bending machines. (A), (b) is explanatory drawing explaining the AR marker for bending machines. (A), (b) is explanatory drawing explaining the AR marker for bending machines. (A), (b) is explanatory drawing explaining the AR marker for bending machines. It is explanatory drawing explaining the AR marker for bending machines. It is explanatory drawing explaining the AR marker for bending machines. It is explanatory drawing explaining the AR marker for bending machines. It is explanatory drawing explaining the arrangement position of AR marker of a bending machine. It is explanatory drawing explaining the position of the AR marker for bending machines, and the display of an image. It is explanatory drawing explaining the display of the image of a punch. It is explanatory drawing explaining the display of the image of a punch. It is explanatory drawing explaining the display of the image of a punch. It is explanatory drawing explaining another assistance system. It is explanatory drawing explaining the display of the image of processing work efficiency improvement. It is explanatory drawing explaining the display of the image of processing work efficiency improvement. It is explanatory drawing explaining the display of the image of processing work efficiency improvement. It is explanatory drawing explaining the display of the image of processing work efficiency improvement.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The two embodiments will be described separately as [Embodiment 1] and [Embodiment 2]. [Embodiment 1] relates to setup, and [Embodiment 2] relates to bending work.

[Embodiment 1]
Please refer to FIG. The component of the support system s is shown.

  The support system s includes an AR marker d1 for a bending machine, a mold database d2, a machining operation content d3, and a head mounted display h1 equipped with an AR marker recognition camera. The support system s includes a computer and executes computer control.

  The AR marker d1 for the bending machine is an AR marker for causing the support system s to recognize position information necessary for the bending setup (the mounting position of the AR marker is fixed with the origin of the bending machine m as a reference position). It shall be attached based on the rules).

  In this example, an AR (augmented reality) marker used for a bending machine (also called a bending machine) m is specifically defined as an AR marker d1 for a bending machine.

  In addition, the AR marker (for example, described in claims) is a broad concept including an AR marker for a bending machine in addition to a normal AR marker.

  The mold database d2 accumulates mold shape data and outputs mold shape data in response to input of mold information (such as mold identification information).

  The machining work content d3 is information for reading the created work setup and outputting necessary mold information and the mounting position / orientation of the mold.

  The head-mounted display h1 equipped with the AR marker recognition camera is an apparatus for grasping the worker's visual recognition state by attaching it to the worker and teaching the worker the work contents each time.

  More specifically, the head mounted display h1 equipped with the AR marker recognition camera includes an AR marker recognition camera km, an image generation unit pr, a display (including all having a display function) dp, and a frame ( For example, it can be worn on the head of a human body.

  In this example, an image relates to a setup for bending, and includes polygon data, annotation data, and the like.

  The operation of the support system s will be described.

  In the process p1, at the start of machining setup, the AR marker d1 for the bending machine is recognized using the AR marker recognition camera km of the head mounted display h1 on which the AR marker recognition camera worn by the operator is mounted.

  In the process p2, the content of the bending machine AR marker d1 acquired in the process p1 is compared with the data received from the mold database d2 and the machining work content d3, and an image corresponding to the work content is displayed on the AR marker recognition camera. Is displayed on the display dp of the head mounted display h1.

  In process p3, the worker attaches the mold according to the display on the head mounted display h1 equipped with the AR marker recognition camera (by acquiring and displaying the outline data of the mold from the mold database d2). Prevent work mistakes).

In the process p4, every time the mold is attached, the support system s reads the next work and displays it on the display dp. In the process p5, the processes p1 to p3 are repeated, and when all work is completed, This is notified to the display dp of the head mounted display h1 equipped with the marker recognition camera (“The mold installation is completed. Next, the bending process is performed”).

  In this embodiment, the recognition of the AR marker d1 for the bending machine and the processing of the information are concentrated in the head mounted display h1 equipped with the AR marker recognition camera (the processing work content d3 is the head equipped with the AR marker recognition camera. The mount display h1 receives and performs processing internally), but the processing performed by the head mounted display h1 equipped with the AR marker recognition camera is limited to image acquisition and teaching information display, and data processing is performed by the operator. It may be performed at a distance (information is transferred from the head mounted display h1 equipped with the AR marker recognition camera, and necessary information is received and displayed). Conversely, the mold database d2 and the machining work content d3 can be integrated into the head mounted display h1 equipped with the AR marker recognition camera.

  A commonly used AR (Augmented Reality) technology is shown.

  For example, a projector projection type monitor is mounted on a glasses-like frame. The monitor can display information transparently. Projection information is obtained from a connected small PC. In this support system s, a camera is added to recognize the AR marker.

  For example, if a specified AR marker is placed at a position where furniture is to be placed and photographed with a camera, an image of the furniture is displayed overlapping the AR marker position. When the AR marker is rotated and moved, the display on the screen is also changed. The display changes in the same way even if the camera is moved. It is possible to visually understand the size of furniture.

  Furthermore, the position, size, and angle of the AR marker are calculated. For example, a wristwatch type image in which an AR marker is recorded is used. When the web camera is passed through the printout and wound around the arm, the clock image is displayed superimposed. By applying this technology, the work content is displayed in the field of view of the worker.

  Although the above is a general usage method, the application to a bending process is demonstrated.

  Please refer to FIG. 2, FIG. 3 and FIG. Two directional components (XY axes) perpendicular to the inner and outer frame edges of the bending machine AR marker armk-a are extracted. The processing side holds the data of the AR marker armk-a for the bending machine.

  Based on the extraction results of the internal feature points pt1 to pt10, the axis allocation and the origin position are detected. The internal pattern should not be rotationally symmetric. Note that “AR”, which is an internal pattern in this example, is not rotationally symmetric.

  The third axis (Z axis) is calculated from the origin and the axis system.

  Please refer to FIG. 3 and FIG. As shown in the AR marker armk-b for the bending machine and the AR marker armk-c for the bending machine, even if the AR markers armk-b and armk-c for the bending machine are tilted depending on the camera position, the feature points pt1 to pt1 If pt10 can be extracted, the axis can be calculated.

  Then, the rotation amount of the shaft system is calculated from the way the three axes are reflected.

  Thereby, data projection linked to the camera position is possible by applying the rotation amount of the coordinate system with the AR marker for the bending machine as a reference to the polygon data.

  Note that the AR markers for the bending machine of this example are all black (regions indicated by hatching).

  Reference is made to FIGS. 5A, 5B, 6A, 6B, 7A, and 7B. The size of the polygon data is changed according to the size of the AR marker for the bending machine that is shown.

  As shown in FIG. 5A, polygon data pg-a viewed from the front side according to the size of the AR marker armk-a for the bending machine is displayed. As shown in FIG. 5B, when the AR marker armk-as for the bending machine is small, the polygon data pg-as is also displayed small.

  Similarly, as shown in FIG. 6A, polygon data pg-b viewed from the side surface corresponding to the size of the AR marker armk-b for the bending machine is displayed. As shown in FIG. 6B, when the AR marker armk-bs for the bending machine is small, the polygon data pg-bs is also displayed small.

  Similarly, as shown in FIG. 7A, polygon data pg-c viewed from the bottom side corresponding to the size of the AR marker armk-c for the bending machine is displayed. As shown in FIG. 7B, when the AR marker armk-cs for the bending machine is small, the polygon data pg-cs is also displayed small.

  Please refer to FIG. 8, FIG. 9 and FIG. When the AR marker position for the bending machine and the projection position of the polygon data are deviated, a parallel movement process is applied to the polygon data in addition to the rotation process so that the display corresponds to an appropriate one.

  Please refer to FIG. The projection position of the AR marker armk-a for the bending machine viewed from the front side and the polygon data pg-am corresponding thereto are shifted. The polygon data pg-am can be overlapped by performing parallel processing.

  Please refer to FIG. The projection positions of the AR marker armk-b for the bending machine viewed from the side and the polygon data pg-bm corresponding thereto are shifted. The polygon data pg-bm can be overlapped by performing parallel processing.

  Please refer to FIG. The projection position of the AR marker armk-c for the bending machine viewed from the bottom side and the polygon data pg-cm corresponding thereto are shifted. The polygon data pg-cm can be overlapped by performing parallel processing.

  Please refer to FIG. The AR marker for the bending machine is installed at the specified position of the processing machine m. A plurality of bending machine AR markers are installed in accordance with the visual field range of the AR marker recognition camera km (see FIG. 1) and the size of the bending machine m. The AR marker for the processing machine having a different pattern is used so that the current position of the worker can be identified. It is assumed that which processing machine AR marker is in which position of the machine. In addition, the bending machine m abuts the work against the abutment T, and processes it with a mold to which the intermediate plate CP is attached.

  Here, the bending machine AR marker armk1 (internal pattern is “AR1”), the bending machine AR marker armk2 (internal pattern is “AR2”), and the bending machine AR marker armk3. (The inner pattern is “AR3”), the bending machine AR marker armk4 (the inner pattern is “AR4”), and the bending machine AR marker armk5 (the inner pattern is “AR5”). ) And the AR marker armk6 for bending machine (the inner pattern is “AR6”) are arranged in the bending machine m at equal intervals.

  It is assumed that the processing machine AR markers armk1 to armk6 are attached based on a certain rule with the origin of the bending machine m as a reference position.

  Please refer to FIG. The machining operation content d3 (see FIG. 1) is read, and the display content corresponding to the bending machine AR marker to be recognized is determined. For example, when the bending machine AR marker armk1 or the bending machine AR marker armk2 is reflected in the AR marker recognition camera km, the polygon data pg of the punch die corresponding to the AR marker is displayed at the corresponding mounting position (2). The punch die indicated by the dotted line is displayed on the dill play dp).

  On the other hand, when only the other bending machine AR markers armk3 to armk6 are shown in the AR marker recognition camera km, polygon data, annotation data, etc. for instructing the operator to move left and right are displayed.

  Thus, for example, a plurality of bending machine AR markers armk1 to armk6 are installed according to the visual field range of the AR marker recognition camera km and the size of the bending machine m. The AR marker for the bending machine having a different pattern is used so that the current position of the operator can be identified. It is assumed that which bending machine AR marker is located at which position of the bending machine m.

  Please refer to FIG. The support system s reads information on the mold database d2, machining content d3, and the AR marker d1 (specifically armk1 to armk6, etc.) for the bending machine obtained from the AR marker recognition camera km, and the necessary metal for machining. Displays the shape and position of the mold. The operator attaches the mold while looking at the display.

  For example, polygon data pg1 (indicated by a two-dot chain line) of a punch die is displayed, and an operator mounts an actual punch pnc1 (indicated by a solid line) on the intermediate plate cp according to this display.

  Refer to FIG. Depending on the type of mold, alert the front and back. This is to prevent processing defects and accidents due to mistakes in the front and back.

  For example, punch die polygon data pg2 (indicated by a two-dot chain line) is displayed. In accordance with this display, the operator mounts the actual punch pnc2 (shown by a solid line) on the intermediate plate cp.

  Further, related note data nt1 (for example, “Please pay attention to the direction of the mold”) is displayed in association.

  Refer to FIG. By displaying the shape of the mold, mold errors and mounting errors are prevented. By recognizing the operator's position using the AR marker for the bending machine and overlaying display of polygon data, it is possible to teach work more easily.

  For example, since the punch die polygon data pg3 (indicated by a two-dot chain line) and the actual punch pnc3 (indicated by a solid line) are superimposed and displayed, it is possible to prevent a die error and a mounting error.

[Embodiment 2]
Here, an embodiment related to work will be described. Refer to FIG. The components of the support system S are shown. Note that functions different from those in [Embodiment 1] are mainly described, and other functions are the same as those in [Embodiment 1].

  The support system S includes a bending machine AR marker D1, a machining operation content D2, and a head mounted display H1 on which an AR marker recognition camera KM is mounted. The support system S includes a computer and executes computer control.

  The AR marker D1 for the bending machine is an AR marker for causing the support system S to recognize position information necessary for bending work, work setup, etc. (the mounting position of the AR marker is the origin of the bending machine M as a reference position) It shall be attached based on a certain rule).

  In this example, an AR (augmented reality) marker used for a bending machine (also called a bending machine) M is specifically defined as an AR marker D1 for a bending machine.

  In addition, the AR marker (for example, described in claims) is a broad concept including an AR marker for a bending machine in addition to a normal AR marker.

  The machining work content D2 is information for reading the created bending work, work setup, and the like and outputting necessary mold information and the mounting position / orientation of the mold.

  The head mounted display H1 on which the AR marker recognition camera KM is mounted is a device for grasping the worker's visual recognition state by wearing the AR marker recognition camera KM, and teaching the worker the work contents each time.

  More specifically, the head-mounted display H1 equipped with the AR marker recognition camera includes an AR marker recognition camera KM, an image generation means PR, a display (including all having a display function) DP, a frame ( For example, it can be worn on the head of a human body.

  In this example, the image is related to the bending work, and includes polygon data, annotation data, and the like.

  The operation of the support system S will be described.

  In the process P1, at the start of processing setup, the AR marker D1 for the bending machine is recognized using the camera of the head mounted display H1 on which the AR marker recognition camera worn by the operator is mounted.

  In the process P2, the AR marker position for the bending machine acquired in the process P1 is compared with the data received from the machining operation content D2, and an image corresponding to the operation content is displayed on the head mounted display H1 equipped with the AR marker recognition camera. Display on the display DP.

  In the process P3, the operator hits a component according to the display on the head mounted display H1 equipped with the AR marker recognition camera (the component at the time of work on the display DP of the head mounted display H1 equipped with the AR marker recognition camera). Displaying the outline of the work prevents mistakes in the work).

  In process P4, the worker performs bending according to the display on the head mounted display H1 equipped with the AR marker recognition camera (when there is a risk that the worker may pinch his / her hand, the head mount equipped with the AR marker recognition camera). This is notified to the display DP of the display H1 to urge attention).

  In process P5, every time one bending operation is completed, the support system S reads the next operation and displays it on the display of the head mounted display H1 equipped with the AR marker recognition camera. (Use “←”, “→”, etc. to instruct the change of the mold to be used, and display the outline of the material to be processed if you are at the position of the mold to use.)

  In the process P6, the processes P1 to P4 are repeated, and when all the operations are completed, the fact is notified to the display DP of the head mounted display H1 equipped with the AR marker recognition camera (“the bending process is completed”). etc).

  In the present invention, the recognition of the AR marker D1 for the bending machine and the processing of the information are integrated in the head mounted display H1 equipped with the AR marker recognition camera (the head mounted display equipped with the AR marker recognition camera. H1 is received and processed internally), but the processing performed on the head mounted display H1 equipped with the AR marker recognition camera is limited to image acquisition and teaching information display, and data processing is separated from the operator. However, it may be performed (information is transferred from the head mounted display H1 equipped with the AR marker recognition camera, and necessary information is received and displayed). Conversely, it is also possible to integrate the processing work content D2 into the head mounted display H1 equipped with the AR marker recognition camera.

  Refer to FIG. The support system S reads the work content from the machining work content D2, and takes the work shape (for example, polygon data) at the position to be processed in consideration of the information of the bending machine AR marker D1 obtained from the AR marker recognition camera KM. indicate. The operator confirms the display and performs bending at the instructed position. The shape of the workpiece w to be displayed is changed in conjunction with the process transition (read from the bending data).

  For example, polygon data pg-w is displayed (area indicated by a plurality of dots). In accordance with this, the operator superimposes the actual workpiece and performs bending at the correct position using the punch die pnc and the die die die. Actually, there is a work w held by the operator's hand hd below the polygon data pg-w.

  Please refer to FIG. In the case of work for which there is a possibility that a hand may be caught in the processing setup data, the display is changed to alert (the alert is added when creating the work setup). The operator checks the display and performs the work with caution.

  For example, polygon data pg-w (a region indicated by a plurality of dots) is displayed. In accordance with this, the operator superimposes the actual parts and performs bending at the correct position. Actually, there is a work w held by the operator's hand hd below the polygon data pg-w.

  Furthermore, the annotation data nt2 (for example, “There is a possibility that a hand may be caught.”) Is displayed in association with the polygon data pg-w. The dangerous area may be indicated by a color line such as a red line.

  Refer to FIG. Each time one bending operation is completed, the support system S switches the display (displays the next processing instruction). The worker moves according to the instruction, and the support system S reads the AR marker for the bending machine and switches the display as shown in FIG. 17 when it comes to the mold to be used.

  Annotation data nt3 (for example, “use the right mold”) is displayed.

  Refer to FIG. By displaying the workpiece shape (predicted view), it is possible to prevent mistakes in front / back determination.

  For example, polygon data pg-w (area indicated by a plurality of dots) is displayed. In accordance with this, the operator determines the front and back by superimposing the actual work w.

  The present invention is not limited to the embodiments of the invention described above, and can be implemented in other modes by making appropriate modifications.

s Processing support system h1 Head mounted display km AR marker recognition camera pr Projector dp Display m Bending machine d1 AR marker d2 Mold database d3 Processing work content w Workpiece

Claims (7)

In the support system that supports the setup of bending process,
A device configured to be wearable on a human body, recognizing an AR marker and displaying an image corresponding to the recognized AR marker on a display;
The image relates to a setup for bending process, and the image is displayed on the display so as to be visible at the time of setup. In the support system that supports the bending work of bending work,
A device configured to be wearable on a human body, recognizing an AR marker and displaying an image corresponding to the recognized AR marker on a display;
The support system, wherein the image relates to a bending operation, and the image is displayed on the display so as to be visible during the bending operation.   The support system according to claim 1, wherein the AR marker is arranged at a plurality of positions of a bending machine.   The support system according to claim 3, wherein images corresponding to a plurality of bending processes are displayed for each process.   The support system according to claim 3 or 4, wherein the image is information in which polygon data and annotation data are associated with each other. In the support method that supports the setup of bending process,
A device configured to be wearable on a human body, recognizing an AR marker and displaying an image corresponding to the recognized AR marker on a display;
The support method according to claim 1, wherein the image relates to a setup for bending, and the image is displayed on the display so as to be visible at the time of setup. In the support method to support the bending work of bending work,
A device configured to be wearable on a human body, recognizing an AR marker and displaying an image corresponding to the recognized AR marker on a display;
The image is related to a bending operation, and the image is displayed on the display so as to be visible during the bending operation.

Images