JP2018045669A - Production navigation system, support system therefor, and data management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To present information favorable in producing an apparatus to an assembly operator.SOLUTION: A production navigation system 10 comprises: a shelf 12 that is provided with a plurality of component placement parts 21 on which components are placed; a display part (display mechanism 15 including projector 31, mirror 32, and projection parts 33, 34) that presents display information to an operator; a camera 14 that photographs the operator; and a control device 11 that controls the operation of the display part. The control device determines the display information to be displayed on the display part according to the position of a specific portion of the operator photographed by the camera.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a production navigation system, a support system thereof, and a data management method.

In recent years, factories producing various devices are becoming the following trends.
・ Many parts are handled per person.
・ New products are launched one after another in a short period of time.
・ Production lines are frequently changed for each product.
・ The production system by temporary workers is becoming a daily routine.
Such a tendency becomes more prominent in factories that produce small quantities of a wide variety of devices.
However, as this tendency progresses, the possibility of assembly failure, assembly error, and the like increases. For this reason, in factories, there is a demand for the realization of a mechanism for preventing assembly leakage, assembly errors, work leakage, and the like.

  As a mechanism for preventing assembly leakage, there is a mechanism for performing kit work for parts. The parts kit operation is an operation in which a parts distributor supplies parts to an assembly operator without omission by arranging the parts in a dedicated kit box in which a place for each part is determined from the parts storage warehouse. The parts storage warehouse is configured to notify the operator of the location of the parts to be collected by turning on an LED provided at the sampling port. With this mechanism, the assembly operator can prevent assembly leakage by confirming that no parts remain in the kit box after assembly is completed.

  As an apparatus related to the kit operation, for example, there is an apparatus described in Patent Document 1. The apparatus described in Patent Document 1 improves workability by transporting a part to the vicinity of a workbench equipped with a sensor and moving the next part to the vicinity of the workbench when the removal of the part is detected by the sensor. (For example, refer to Patent Document 1).

  Also, as a mechanism to prevent assembly errors, work instructions that instruct the assembly method etc. are placed around the assembly workers until the assembly workers are familiar with the assembly work, and the assembly workers confirm the work instructions. There is a mechanism to assemble while.

  In addition, as a mechanism to prevent work leakage, a work history is recorded by filling the work history for each worker in the work item list list called check sheet according to the results of work, or by performing stamping or signing. There is.

JP-A-10-29139

However, the above-described conventional mechanism for preventing assembly leakage requires a parts distributor, which increases the number of personnel.
Further, the parts storage warehouse is configured to notify the operator of the location of the parts to be collected by turning on an LED provided at the sampling port. However, the storage warehouse of parts has a structure that makes it difficult to understand the lighting state of the LED depending on the environment of the work place. Also, by turning on the LED, how many parts are collected, where and how There was a problem that it was impossible to inform the worker of information such as whether or not to attach.
Furthermore, the conventional mechanism for preventing assembly leaks takes a lot of man-hours to prepare parts for the start-up of a new product and to prepare kit boxes used to prevent assembly leaks. There was a problem.
In addition, the conventional mechanism for preventing assembly leaks often involves design changes such as component changes during the start-up period. There was a problem.

  In addition, the above-described mechanism for preventing assembly errors is because work instructions are stored in paper files, which requires man-hours to search for desired pages and maintain work instructions, preventing assembly errors. There was a problem that the provision of a support system for this purpose was desired.

  In addition, the mechanism for preventing the above-mentioned work leakage is that the check sheet is a paper medium, and it takes time and effort to write the work history on the paper medium, and to perform stamping and signing, and to maintain the check sheet. Therefore, there is a problem that it is desired to provide a support system for preventing work leakage.

  The present invention has been made in order to solve the above-described problems, and provides a production navigation system, a support system for the production navigation system, and a data management method for presenting information suitable for assembly to the assembly operator. The main purpose.

To achieve the above object, the first invention is a production navigation system, a shelf provided with a plurality of component placement units for placing components, a display unit for presenting display information to an operator,
An imaging unit that images the worker; and a control device that controls the operation of the display unit, wherein the control unit is configured to display the display unit according to a position of a specific part of the operator imaged by the imaging unit. The display information to be displayed is determined.

  The second invention is a production navigation system, wherein a shelf provided with a plurality of component placement units for placing components, a projector for projecting display information, and the display information projected from the projector. And a mirror that reflects the projection unit provided on the shelf.

  The third invention is a support system, and provides the production navigation system with a storage unit for saving a photographed image taken by the production navigation system and navigation information displayed as display information in the production navigation system. And a providing unit.

  The fourth invention is a data management method, wherein work data acquired at the time of product assembly work is stored in a storage unit, work results are managed by the management unit based on the stored work data, and displayed. The information creation unit creates display information representing the work performance at an arbitrary timing.

  ADVANTAGE OF THE INVENTION According to this invention, when manufacturing an apparatus, suitable information can be shown to an assembly operator.

1 is a diagram illustrating an overall configuration of a system according to an embodiment. It is a figure (1) which shows the composition of the production navigation system concerning an embodiment. It is a figure (2) which shows the composition of the production navigation system concerning an embodiment. It is a figure (1) which shows the composition of the principal part of the production navigation system concerning an embodiment. It is FIG. (2) which shows the structure of the principal part of the production navigation system which concerns on embodiment. It is a flowchart which shows the whole operation | movement of the production navigation system which concerns on embodiment. It is a flowchart which shows the operation | movement at the time of component navigation of the production navigation system which concerns on embodiment. It is a figure (1) which shows an example of a display screen. It is FIG. (2) which shows an example of a display screen. It is a figure (3) which shows an example of a display screen. It is FIG. (4) which shows an example of a display screen. It is a figure (5) which shows an example of a display screen. It is a figure (6) which shows an example of a display screen. It is a figure (1) which shows the example of a display at the time of parts navigation of the production navigation system concerning an embodiment. It is a figure (2) which shows the example of a display at the time of the parts navigation of the production navigation system concerning an embodiment. It is a figure which shows the outline | summary of the position detection operation | movement of the hand in the production navigation system which concerns on embodiment. It is a figure which shows the image analysis example of the production navigation system which concerns on embodiment. It is a figure which shows the fixed procedure at the time of the position detection of the hand in the production navigation system which concerns on embodiment. It is a figure (1) which shows the example of a setting for image analysis of the production navigation system concerning an embodiment. It is FIG. (2) which shows the example of a setting for image analysis of the production navigation system which concerns on embodiment. It is a flowchart which shows the operation | movement at the time of the image analysis of the production navigation system which concerns on embodiment. It is a flowchart which shows the data management operation | movement of the production navigation system which concerns on embodiment. It is a figure which shows an example of a management information file. It is a figure which shows an example of a main screen. It is a figure which shows an example of a production status screen. It is a figure which shows an example of a production number performance screen. It is a figure which shows an example of a work comparison screen. It is a figure which shows an example of a work performance (daily) screen. It is a figure which shows an example of a work performance (by process) screen. It is a figure which shows an example of an operation | movement analysis screen. It is a figure which shows an example of an end screen.

  Hereinafter, an embodiment of the present invention (hereinafter referred to as “the present embodiment”) will be described in detail with reference to the drawings. Each figure is only schematically shown so that the present invention can be fully understood. Therefore, the present invention is not limited to the illustrated example. Moreover, in each figure, the same code | symbol is attached | subjected about the common component and the same component, and those overlapping description is abbreviate | omitted.

[Embodiment]
<Overall system configuration>
Hereinafter, the overall configuration of the system according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an overall configuration of a system according to the present embodiment.

As shown in FIG. 1, a production navigation system 10 and a support system 60 are installed in a factory that produces some device.
The production navigation system 10 is a system that presents various information to an assembly worker (hereinafter, may be simply referred to as “worker”) to support product assembly work. The production navigation system 10 can output, to the support system 60, a captured image (still image or moving image) that captures the work status of the worker, voice information that records the worker's voice, and the like. 1 shows the control device 11, the management unit 71, the display information creation unit 72, the storage unit 73, and the like, which will be described later.
The support system 60 is a system that supports the production navigation system 10.

  The production navigation system 10 and the support system 60 are connected to be communicable by wire or wireless. The support system 60 may be installed outside the factory.

  The support system 60 includes a data collection device 61, a data processing device 62, and a support device 63.

The data collection device 61 stores shooting information, audio information, and the like output from the production navigation system 10 in a storage unit. In addition, the data collection device 61 includes various information (for example, work instruction information, work process information, guidance information, projection position information such as marks) and various programs (for example, display mechanism control programs, image analysis). And the like can be stored in the storage unit and provided to the production navigation system 10.
The data processing device 62 analyzes work contents and the like based on shooting information, audio information, and the like stored in the data collection device 61.
The support device 63 is used for analyzing the work contents of the worker, creating work instructions, and the like. The support device 63 is configured by a personal computer (PC). The support device 63 is operated by an administrator, support personnel, or the like (hereinafter referred to as “manager etc.”), and is used to create analysis data (for example, see FIGS. 19B to 19E described later).

  The data collection device 61 includes functional means such as an acquisition unit 81, a storage unit 82, an analysis unit 83, and a provision unit 84, for example. The acquisition unit 81 is a functional unit that acquires work data at the time of product assembly work from the production navigation system 10. The work data includes a photographed image photographed by the production navigation system 10 during product assembly work. The storage unit 82 is a functional unit that stores work data at the time of product assembly work. The analysis unit 83 is a functional unit that analyzes the work progress, progress, and the like based on work data during product assembly work. The providing unit 84 is a functional unit that provides the production navigation system 10 with navigation information displayed as display information in the production navigation system 10. One or more of these functional means may be provided in the data processing device 62 of the support system 60, the support device 63, the control device 11 of the production navigation system 10, and the like.

<Configuration of production navigation system>
Hereinafter, the configuration of the production navigation system 10 will be described with reference to FIGS. 2 and 3. 2 and 3 are diagrams showing the configuration of the production navigation system 10.

  As shown in FIG. 2, the production navigation system 10 includes a control device 11, a shelf 12, a workbench 13, a photographing unit 14 (14a, 14b), a display mechanism 15, and a display 16 (16a, 16b). , Microphone 17, speaker 18, code input unit 19, and motion capture camera 20.

  The control device 11 controls operations of the photographing unit 14, the display mechanism (display unit) 15, the display 16, the microphone 17, the speaker 18, and the like. The control device 11 is communicably connected to these devices by wire or wireless. The control device 11 is connected to the data collection device 61 of the support system 60 so as to be communicable by wire or wireless. The control device 11 is configured by a personal computer (PC) and operates based on a control program stored in advance. The control device 11 determines display information to be displayed on the display mechanism 15 according to the position of the specific part of the worker imaged by the imaging unit 14.

  As illustrated in FIG. 1, the control device 11 includes functional units such as a management unit 71, a display information creation unit 72, and a storage unit 73. The management unit 71 is a functional unit that manages work results based on work data stored in the storage unit 73. The display information creation unit 72 is a functional unit that creates display information representing work results at an arbitrary timing. The accumulation unit 73 is a functional unit that accumulates work data acquired at the time of product assembly work. One or more of these functional means may be provided in the data collection device 61, the data processing device 62, the support device 63, and the like of the support system 60.

  Returning to FIG. 2, the shelf 12 includes a large number of component placement units 21 (four in the vertical direction and six rows in the horizontal direction, 24 in total), and functions as a component warehouse. The shelf 12 is provided with a plurality of partition plates 22. Each of the partition plates 22 is arranged in a substantially vertical direction, divides the individual component mounting portions 21, and is arranged so that the tip protrudes forward from the front surface of the shelf 12. A tray 40 is mounted on each component mounting portion 21, and a single type of component (for example, a screw, a bracket, a roller, or the like) is stored in the tray 40. The control device 11 manages component placement information indicating which type of component 41 is placed on the component placement portion 21 at which position, and an operator can remove the component 41 from the component placement portion 21. When the sample is collected, it is possible to identify which type of component 41 is collected. For example, an assembly tool such as an electric screwdriver may be placed on the component placement unit 21.

  A command input position can be arbitrarily set in advance on the shelf 12. In the illustrated example, for example, the position of the first component placement unit 21 from the bottom and the first from the right is set as a special command input unit 23 used for inputting a special command. The contents of the command include, for example, reset, return (decision), menu selection, assembly confirmation, and the like. When the operator's hand is held over a command input position (special command input unit 23 in the illustrated example) (that is, the control device 11 puts the hand over the object). Display information is determined according to the command corresponding to the input position.

  The work table 13 is a table used by an operator for parts assembly and the like. The work table 13 includes a flat table surface (upper surface). The base surface is arranged so as to extend in the horizontal direction over the entire width of the shelf 12.

  The photographing unit 14 (14a, 14b) photographs a worker's work situation and the like. The imaging unit 14 is positioned above the height of the standard height operator's head (for example, above the uppermost component mounting unit) by the support frame 44 that supports the mirror 32, and It is arranged at a position ahead of the front surface of the shelf 12. The photographing unit 14 is composed of a USB camera. The USB camera 14 as a photographing unit outputs a photographed image to the control device 11. The number of USB cameras 14 is two or more. Two or more USB cameras 14 are spaced apart in the left-right direction. In the illustrated example, two USB cameras 14 are attached to the left and right of the mirror 32. However, the number of USB cameras 14 may be three or more. In addition, the USB camera 14 is not limited to the support frame 44, and may be disposed at various places (for example, outside the shelf 12 or inside each component placement unit 21).

  The display mechanism 15 is a display unit that displays various information, marks, and the like. In the present embodiment, the display mechanism 15 includes a projector 31, a mirror 32, and plate members 33 and 34. However, the display mechanism 15 can be configured by a device such as an EL display or a liquid crystal display disposed at the position of the plate-like members 33 and 34 instead of these devices.

The display 16 (16a, 16b) displays various information to the worker. The display 16 is preferably configured by a touch panel display. The display 16a displays, for example, a screen IM20 shown in FIG. On the other hand, the display 16b displays, for example, main screens IM21 to IM25 shown in FIGS. 19A to 19E to be described later, a screen IM31 shown in FIG. In addition, for example, one of the display 16a and the display 16b is used as a motion capture display that displays a moving image of the worker taken by the motion capture camera 20, and the other is used based on the moving image. You may make it use as a data analysis display which displays the result of having analyzed work contents. In the present embodiment, the control device 11 will be described as controlling the display operations of the two displays 16a and 16b. However, the production navigation system 10 may be provided with a separate control device for each of the two displays 16a and 16b.
The microphone 17 records the worker's voice and the like.
The speaker 18 reports an alarm sound and voice guidance information.

  A code is input to the code input unit 19. In the illustrated example, the code input unit 19 is configured by a barcode scanner and is attached to the work table 13. The code input unit 19 is not limited to a barcode scanner, and can be configured by various input means such as an IC card reader and a keyboard.

  The motion capture camera 20 captures almost the entire image of the worker including at least the upper body of the worker. The motion capture camera 20 is arranged on the side of the work table 13, for example. The control device 11 can analyze the flow line of the worker, the face direction, and the like based on the moving image captured by the motion capture camera 20. In the present embodiment, the production navigation system 10 is described as including a display 16 and a motion capture camera 20. However, the production navigation system 10 can be realized without these. One display 16 and one motion capture camera 20 are not necessarily provided for each shelf 12, and may be provided for each of the plurality of shelves 12.

  Note that a rail 45 that functions as a hanging portion for suspending an assembly tool 46 such as an electric screwdriver is provided at a position in front of the arrangement position of the USB camera 14 of the support frame 44 that supports the mirror 32. The operator can easily perform the assembly work by using the assembly tool 46 suspended from the rail 45. Further, for example, when the component 41 is a screw and an electric driver as the assembly tool 46 is used, the control device 11 can perform a process of changing the torque of the electric driver according to the type of the component 41. . Thereby, the production navigation system 10 can support the assembly work. Further, the control device 11 displays a display so as not to proceed to the next process when a plurality of electric drivers are provided in the production navigation system 10 and when an operator makes a mistake in selecting an electric driver to be used. An error message can be displayed as information to alert the operator. Also by this, the production navigation system 10 can support the assembly work. Note that the type of the component 41 is, for example, the position information and display information of the component placement unit 21 (collection port) which is the collection position of the component 41 analyzed from the moving image captured by the USB camera 14, and the component is displayed to the worker. Based on the positional information of the component placement unit 21 (collection port) that has instructed the collection of 41, it can be specified.

  As shown in FIG. 3, in the present embodiment, the display mechanism 15 projects various display images from the projector 31 onto the plate-like members 33 and 34 and the work surface of the work table 13 via the mirror 32. That is, the display mechanism 15 uses the surface of the plate-like members 33 and 34 and the work table 13 as a projection unit, and uses projection mapping technology to display various display images on the surface of the plate-like members 33 and 34 and the work table 13. Projecting. The projector 31 corrects the display information and projects it on the surface of the plate members 33 and 34 and the work table 13 in a shape approximate to a rectangle.

  The projector 31 is disposed on the shelf 12. Further, the mirror 32 is positioned above the height of the head of a standard height worker (for example, a position above the uppermost component placement section) by the support frame 44 and the front surface of the shelf 12. It is arrange | positioned in the front position rather than. Specifically, the mirror 32 is disposed above the work table 13 and obliquely above the shelf 12 so as to extend in the left-right direction (lateral direction) over the entire width of the shelf 12. The mirror 32 is disposed so that the reflection surface faces the projector 31. The mirror 32 is rotatably supported by a support mechanism so that the reflection angle can be changed in the vertical direction. The mirror 32 can be fixedly installed at an arbitrary reflection angle.

  The plate-like members 33 and 34 are parts that function as a projection unit on which an image (display information) projected from the projector 31 is formed. The plate-like members 33 and 34 are arranged at one or both of the positions above and below the component placement portion 21. However, the plate-like members 33 and 34 are preferably such that the operator's hand does not block the image (display information) projected from the projector 31 (that is, the projected image is reflected on the back of the operator's hand). In order to prevent the shadow of the operator's hand from appearing at the position where the image is to be projected), it may be arranged at a position above the component placement unit 21. Thereby, the projector 31 can project an image (display information) at a position where the operator's hand does not interfere. Particularly when, for example, an arrow mark Mk (see FIG. 8) or the like is projected, an image (display information) may be displayed (projected) at a position above the component placement unit 21. At least the upper surfaces (exposed surfaces) of the plate-like members 33 and 34 are flat surfaces. The plate member 33 has a larger configuration than the plate member 34. The plate-like member 33 is suitable for presenting information requiring a relatively large area (for example, assembly guidance information Gu (see FIG. 8), work instruction information, graphic information, process flow information, etc.). The plate-like member 34 is suitable for presenting information (for example, the mark Mk (see FIG. 8)) that requires a relatively small area.

  In the illustrated example, the plate-like members 33 and 34 are disposed obliquely on the front surface of the shelf 12 so as to extend in the left-right direction (lateral direction) over the entire width of the shelf 12. However, the plate-like member 33 can be configured with a width shorter than the width of the shelf 12.

  The plate-like members 33 and 34 can project the display image clearly, and the USB camera 14 (see FIG. 2) can improve the operator's hand (specifically, it can be extracted from the background). ) It should have a white color so that it can be photographed. The workbench 13 preferably also has a white color.

  The plate-like members 33 and 34 are arranged so as to protrude forward (working space side) as they are arranged below according to the arranged height (see FIG. 4A). In addition, the plate-like members 33 and 34 are disposed so as to be inclined so that the front portion thereof is lower than the rear portion thereof. In particular, the plate-like member 34 is arranged so as to be inclined according to the arranged height so that the lower the plate-like member 34 is laid down (the collapsed state) (see FIG. 4B). ). Thereby, the display mechanism 15 can favorably project the display information reflected by the mirror 32 onto the plate-like members 33 and 34.

  In such a configuration, the display mechanism 15 reflects the display image by the mirror 32 so that a sufficient distance for projecting the display image is provided between the projector 31 and the plate-like members 33 and 34 and between the projector 31 and the work table. It is secured between 13 base surfaces. Such a display mechanism 15 can project a display image over a wide range from the projector 31 at a narrow projection angle. As a result, the production navigation system 10 can secure a projection distance for satisfactorily projecting the display information on the projection unit (the plate-like members 33 and 34 and the surface of the work table 13). It is possible to avoid disposing the projector 31 on the screen. Therefore, the production navigation system 10 can be used even in a place where the ceiling is not high, for example.

  In addition, since the production navigation system 10 can project the display information on the surface of the work table 13 by the projector 31, it can present a relatively large image at a position that is easy for the operator to see. Can be improved.

  Further, the display mechanism 15 can place the projector 31 on the shelf 12 by reflecting the display image with the mirror 32. Since the place on the shelf 12 is relatively unused, the display mechanism 15 can effectively use the place. Moreover, since the display mechanism 15 can arrange the projector 31 that is a relatively heavy object at a place away from the work place of the worker, the safety of the worker can be ensured. The mirror 32 is made of a material that is difficult to break, such as reinforced plastic. Also by this, the production navigation system 10 can ensure the safety of the worker.

  Further, the production navigation system 10 can display the display information simultaneously at a plurality of locations by increasing the number of the plate-like members 34. That is, the production navigation system 10 can easily and inexpensively increase the number of parts for displaying the display information simply by increasing the number of plate-like members 34.

  4A and 4B are diagrams each showing a configuration of a main part of the production navigation system 10.

  As shown in FIG. 4A, in the present embodiment, the shelf 12 is provided with a four-stage component placement portion 21. Hereinafter, when distinguishing each of the four-stage component placement portions 21, they are referred to as “component placement portions 21 a, 21 b, 21 c, 21 d” in order from the upper one (see FIG. 4B). The front surface portion of the shelf 12 is inclined so that the lower side protrudes forward rather than the upper side. Moreover, each component mounting part 21 is inclined and provided so that a front part may fall rather than a rear part. A space below the lowermost component mounting portion 21 is a dead space that is difficult to access from the front surface portion of the shelf 12, and the control device 11 is disposed in that space. Thereby, the production navigation system 10 uses the space in the shelf 12 without waste.

  In front of the shelf 12, a work space SP1 for performing assembly work is provided. On the other hand, a passage space SP2 for arranging the tray 40 in which the components 41 are stored is provided behind the shelf 12. The distributor or worker arranges (inserts) the tray 40 in which the component 41 is stored from the passage space SP2 into the component mounting portion 21, thereby sliding the tray 40 in the component mounting portion 21. The tray 40 can be arranged at a position near the front surface of the shelf 12.

  The projector 31 is disposed in the rear part on the shelf 12. Thereby, the production navigation system 10 can suppress the shelf 12 from falling forward.

  As shown in FIG. 4B, the component placement unit 21 is preferably configured so that a plurality of trays 40 can be placed on each stage. That is, the length in the depth direction of the component placement portion 21 is preferably longer than the length in the depth direction of two or more trays 40. As a result, the production navigation system 10 removes the empty front tray 40 and places the new tray 40 placed behind the empty tray 40 at a position near the front surface of the shelf 12. Can do. In addition, the length of the depth direction of the component mounting part 21 is shorter as the upper one. Therefore, the length in the depth direction of the uppermost component placement portion 21a is the shortest.

  Each of the partition plates 22a, 22b, 22c, and 22d that divides the component placing portion 21 has a shape including inclined surfaces 22aa, 22ab, 22ac, and 22ad whose upper portions are cut obliquely. The shape of each partition plate 22 is different between the one arranged on the upper stage side of the shelf 12 and the one arranged on the lower stage side of the shelf 12. For example, in the illustrated example, the length of each partition plate 22 in the depth direction is longer as it is arranged on the lower side. Further, the vertical lengths of the inclined surfaces 22aa, 22ab, 22ac, and 22ad are longer as they are arranged on the lower side. In addition, each partition plate 22a, 22b, 22c, 22d may be a shape provided with the R surface formed in circular arc shape instead of inclined surface 22aa, 22ab, 22ac, 22ad.

  FIG. 5 is a flowchart showing the overall operation of the production navigation system 10. FIG. 6 is a flowchart showing the operation of the production navigation system 10 during component navigation. 7A to 7F are diagrams each illustrating an example of a display screen. 7A to 7F, the uppermost display screen is displayed (projected) on the plate-shaped member 33, and the middle display screen is displayed on the plate-shaped member 34 (any of the four-stage plate-shaped members 34). Is displayed (projected), and the lowermost display screen is displayed (projected) on the surface of the work table 13. Here, the display (projection) operation of each display screen will be described mainly as being performed by the display information creation unit 72 (see FIG. 1) or the management unit 71 (see FIG. 1) of the control device 11. The flow shown in FIGS. 5 and 6 and the display screens shown in FIGS. 7A to 7F can be appropriately changed according to the operation.

  As shown in FIG. 5, the production navigation system 10 first converts, for example, the uppermost display screen IM11a, the middle display screen IM11b, and the lowermost display screen IM11c shown in FIG. A mode selection is received by displaying (projecting) the plate-like member 34 and the table surface of the work table 13 (S105). However, in the example shown in FIG. 7A, the uppermost display screen IM11a and the middle display screen IM11b are in a blank state.

  In the example shown in FIG. 7A, three modes of “work acquisition mode”, “skilled mode”, and “demo mode” are displayed on the display screen IM11c in the lowermost stage so as to be selectable. The “work acquisition mode” is a mode in which navigation information including a work moving image as display information and instruction information with detailed explanations regarding various items is displayed. The “skilled mode” is a mode in which the navigation information is displayed as the display information, and there is no work moving image and there is instruction information with only a commentary on a relatively important matter. “Demo mode” is a mode in which navigation of a temporary screen is displayed as display information. In S105, the worker selects a mode by placing his / her hand in any mode among the three modes displayed on the lowermost display screen IM11c. Here, it is assumed that the worker has selected the “work acquisition mode”.

  Next, when the mode is selected, the production navigation system 10 uses, for example, the uppermost display screen IM12a, the middle display screen IM12b, and the lowermost display screen IM12c shown in FIG. It displays (projects) on the plate-shaped member 33, the plate-shaped member 34, and the surface of the work bench 13 (S110).

  In the example shown in FIG. 7B, the display information “End” is displayed in the uppermost display screen IM12a. Further, display information of “work ID”, “order ID”, and “progress% (progress rate)” is displayed in the lowermost display screen IM12c.

  Next, the production navigation system 10 receives input of the worker ID and the order ID (S115, S120). The input is accepted, for example, when the operator inputs a code from the code input unit 19 (see FIG. 2). For example, the code is entered by attaching a label with barcode information indicating the personal ID assigned to each worker, the product ID assigned to each product to be manufactured, etc. to a portable portable medium. The barcode information is read from the label by the code input unit 19 (see FIG. 2). The label can be created at an arbitrary timing. Further, the barcode information can be of any content depending on the operation.

  Next, the production navigation system 10 determines whether or not it is the final process (S125). When it is determined in S125 that the process is not the final process (in the case of “No”), the production navigation system 10 performs component navigation (S130).

The component navigation is performed, for example, along the flow shown in FIG.
As shown in FIG. 6, the production navigation system 10 first converts the uppermost display screen IM13a, the middle display screen IM13b, and the lowermost display screen IM13c shown in FIG. It is displayed (projected) on the plate-like member 34 and the surface of the work table 13, and the position and quantity of the parts to be collected are displayed by the arrow-shaped mark Mk (S205).

  As shown in FIG. 7C, a mark Mk as navigation information is included in the middle display screen IM13b displayed on the plate member 34 at this time. In the example shown in FIG. 7C, the mark Mk is information (for example, arrow information) indicating the position of the component placement unit 21 (collection port) on which the component 41 to be collected is placed, and the number information of the component 41 to be collected. (For example, “× n (pieces)”). The operator collects as many parts 41 as instructed by the mark Mk from the part placement unit 21 (collecting port) instructed by the mark Mk. At this time, it is preferable that the worker collects the component 41 by repeatedly putting his / her hand in the designated component placement unit 21 (collection port) as many times as the number designated by the mark Mk. The shape of the “information indicating the position of the component mounting portion 21 (collecting port) on which the component 41 to be collected is placed” is not limited to “arrow information”, and may be a shape indicating an arbitrary direction. For example, it may include a shape such as a triangle.

  Next, the production navigation system 10 detects the removal of the component 41 based on the moving image photographed by the USB camera 14 (S210), and the removed component 41 is mounted with the component at the correct address indicated by the mark Mk. It is determined whether or not the sample is collected from the placement unit 21 (collection port) (S215). Here, “address” means, for example, an arrangement position of each component placement unit 21 on the shelf 12 shown in FIG.

  If it is determined in S215 that the extracted component 41 is not collected from the component placement unit 21 (collection port) with the correct address (in the case of “No”), the production navigation system 10 returns an error. Display information is presented (S220).

  At this time, the production navigation system 10 operates, for example, the uppermost display screen IM14a, the middle display screen IM14b, and the lowermost display screen IM14c shown in FIG. 7D with the plate member 33 and the plate member 34, respectively. It is displayed (projected) on the base surface of the base 13. In the example shown in FIG. 7D, each display screen IM14a, IM14b, IM14c has a content in which a red alarm image is displayed on the entire surface as error display information. The operator performs a redo operation for collecting the component 41 in accordance with the error display information. The operation of re-collecting the component 41 returns the component 41 collected in S210 to the original component placement unit 21, and uses the mark Mk from the component placement unit 21 (collection port) of the correct address indicated by the mark Mk. This is done by collecting the number of parts 41 instructed.

  Next, the production navigation system 10 detects the re-collection operation of the component 41 based on the moving image captured by the USB camera 14 (S225). Thereafter, the process returns to S215.

  On the other hand, if it is determined in S215 that the taken-out component 41 is collected from the component placement unit 21 (collection port) with the correct address (in the case of “Yes”), the production navigation system 10 Writes the code information, the number information, etc. of the collected component 41 into the database (storage unit) (S230).

  Next, the production navigation system 10 determines whether or not the number of the collected parts 41 has reached the number indicated by the mark Mk (S235). If it is determined in S235 that the number of collected parts 41 has not reached the number of parts indicated by the mark Mk (in the case of “No”), the process returns to S210. On the other hand, if it is determined in S235 that the number of the collected parts 41 has reached the number indicated by the mark Mk (in the case of “Yes”), the production navigation system 10 displays the navigation information. The corresponding moving image, instruction sheet, etc. are displayed (S240).

  At this time, the production navigation system 10 operates, for example, the uppermost display screen IM15a, the middle display screen IM15b, and the lowermost display screen IM15c shown in FIG. 7E with the plate member 33 and the plate member 34, respectively. It is displayed (projected) on the base surface of the base 13. In the example shown in FIG. 7E, assembly guidance information Gu composed of moving images, instructions, etc. is included as navigation information in the lowermost display screen IM15c displayed on the work table 13. The worker performs assembly work according to the assembly guidance information Gu. Thereafter, the process returns to S125 of FIG.

  Returning to FIG. 5, when it is determined in S125 that the process is the final process (in the case of “Yes”), the production navigation system 10 displays the uppermost display screen IM16a shown in FIG. The display screen IM16b and the lowermost display screen IM16c are displayed (projected) on the plate-like member 33, the plate-like member 34, and the table surface of the work table 13, respectively, and the process is terminated.

  In such a production navigation system 10, the control device 11 is configured to determine display information in accordance with detection information of the operator's hand. Further, the control device 11 is configured to present error display information when the component placement unit 21 (collection port) to be put in is wrong.

  FIG. 8 shows an example in which the display information is projected onto the plate-like members 33 and 34. In the illustrated example, assembly guidance information Gu explaining the assembly operation is projected onto the plate-like member 33, and a mark Mk indicating the collection port of the part to be collected is projected onto the plate-like member 34. The mark Mk can also be projected onto the plate-like member 33.

  The mark Mk can indicate not only the location of the sampling port but also the order of components to be sampled. The mark Mk can also indicate the name, code, etc. of the component to be collected. Further, the mark Mk can be pointed not only to the parts but also to the sampling ports of various assembly tools.

FIG. 9 shows an example in which the display information is projected onto the work table 13. In the illustrated example, assembly guidance information Gu is projected on the work table 13.
As the display information, the projector 31 can project, for example, at least one of a moving image representing a product assembly operation, a manual image, information on the number of parts 41, and product assembly process information.

  FIG. 10 is a diagram showing an outline of the hand position detection operation in the production navigation system 10. For example, the control device 11 detects the position of the operator's hand using the principle of trigonometry. Specifically, as shown in FIG. 10, the control device 11 is based on the AND condition between the left captured image captured by the left USB camera 14a and the right captured image captured by the right USB camera 14b. The position of the operator's hand is detected. In that case, the control apparatus 11 analyzes the position of an operator's hand by the process shown, for example in FIG.

  FIG. 11 is a diagram illustrating an image analysis example of the production navigation system 10. As shown in FIG. 11, the control device 11 (see FIG. 2) has a function of analyzing an image captured by the USB camera 14. The control device 11 can identify, for example, the movement of the operator's hand by performing image analysis. As a result, the control device 11 can also identify information on whether or not the worker has collected parts, information on the number of parts collected, and the like.

  Hereinafter, an example of image analysis will be described with reference to FIG. In addition, the production navigation system 10 distinguishes the background of the worker's hand from the background color (for example, the color of the plate-like members 33 and 34, the surface of the work table 13, the partition plate 22 and the like). Is a white color. Thereby, the control apparatus 11 can detect suitably the position of an operator's hand.

First, as illustrated in FIG. 11A, the control device 11 acquires a master image taken by the USB camera 14 in advance. The master image is an image in which no worker is shown. A plurality of master images are prepared in advance corresponding to arbitrary parts of the shelf 12.
Next, as illustrated in FIG. 11B, when the assembly work is started, the control device 11 acquires an image being worked from the USB camera 14. In FIG. 11B, a broken line area represents an area from which the master image in FIG.
Next, as shown in FIG. 11C, an image of the operator's hand is extracted from the image during work (see FIG. 11B).

Next, as illustrated in FIG. 11D, the control device 11 performs binarization processing on the extracted worker's hand image (see FIG. 11C).
Next, as shown in FIG. 11E, the control device 11 performs a matching process on the image after binarization (see FIG. 11D) and the master image (see FIG. 11A). Then, noise (portion overlapping the plate-like member 34) is deleted from the obtained image, and then the center of gravity of the operator's hand is detected from the obtained image.
Thereafter, as shown in FIG. 11 (f), the control device 11 mounts the component on which the operator's hand is contained based on the detected center of gravity of the operator's hand (see FIG. 11 (e)). The position of the placement unit 21 (collection port) is detected.

  FIG. 12 is a diagram showing a determination procedure when the hand position is detected in the production navigation system 10. The control device 11 sets squares in the left-right direction (horizontal direction) and the vertical direction (vertical direction) corresponding to the individual component placement units 21 provided on the shelf 12, and positions of the operator's hands with respect to each square. Depending on, the position of the hand in each cell is determined. For example, in the present embodiment, the shelf 12 includes six stages of component placement portions 21 in the left-right direction (horizontal direction) and four stages in the vertical direction (vertical direction) (see FIG. 2). Therefore, the control device 11 sets six columns in the left-right direction (horizontal direction) and four columns in the vertical direction (vertical direction).

  Then, like the image Ph14a in FIG. 12, the control device 11 performs the processing shown in FIG. 11 on the square in which the position of the operator's hand is detected from the left captured image captured by the left USB camera 14a. Then, a mark (black dot in the illustrated example) is added. Similarly, like the image Ph <b> 14 b in FIG. 12, the control device 11 performs a process shown in FIG. 11 to display a square in which the position of the operator's hand is detected from the left captured image captured by the right USB camera 14 b. On the other hand, a mark (black dot in the illustrated example) is added. Then, the control device 11 determines the position of the operator's hand (that is, the component placement unit 21 (collecting port) containing the operator's hand) based on the AND condition of the images Ph14a and Ph14b. .

The control device 11 performs image analysis using a plurality of types (for example, two types) of filters in order to realize good image analysis.
FIG. 13 shows a filter setting example for the skin color. FIG. 13A shows a photographed image of an operator's hand that is about to be inserted into the sampling port and a filter setting example for the photographed image. FIG. 13B shows a photographed image example of a female hand or a male hand. For example, the control device 11 uses a blue filter to identify the skin color. The filter is set to a desired gradation value so that the hand shown in FIG. 13B can be identified.

  FIG. 14 shows a filter setting example for the color of work clothes. FIG. 14A shows a photographed image of a worker's hand that is about to be inserted into the sampling port and a filter setting example for the photographed image. FIG. 14B shows a photographed image example of work clothes for women and work clothes for men. Here, the color of the work clothes for women is yellow and light green, or yellow and dark green, while the color of work clothes for men is black and dark green. . In this case, the control device 11 uses a red filter in order to identify the color of the work clothes. The filter is set to a desired gradation value so that the work clothes shown in FIG. 14B can be identified.

  The filter to be used may be automatically adjusted (set) in accordance with the skin color of the worker or the color of the work clothes.

FIG. 15 is a flowchart showing the operation of the production navigation system 10 during image analysis.
As illustrated in FIG. 15, the control device 11 performs the processes of S305 to S350 on the left captured image captured by the left USB camera 14a. Further, in parallel with the processes of S305 to S350, the control device 11 performs the processes of S405 to S450 on the right photographed image photographed by the right USB camera 14b.

  Specifically, the control device 11 acquires a left captured image (S305), and deletes an unnecessary image (an image of a portion that is not a worker's hand) from the left captured image (S310). The control device 11 masks the left camera image with a red filter (S322), binarizes the obtained image (S324), and extracts a difference from the master image (S326). Then, noise (portion overlapping with the plate member 34) is deleted from the obtained image (S328). Similarly, the control device 11 masks the left captured image with a blue filter (S332), binarizes the obtained image (S334), and extracts a difference from the master image (S336). ), Noise (a portion overlapping the plate-like member 34) is deleted from the obtained image (S338). Thereafter, the control device 11 combines the image obtained by applying the red filter and the image obtained by applying the blue filter (S340), and detects the position of the center of gravity of the operator's hand from the obtained image. (S345), the position of the sampling port containing the hand is determined (S350).

  In addition, the control device 11 acquires the right captured image (S405), and deletes an unnecessary image (image of a portion that is not the operator's hand) from the right captured image (S410). The control device 11 masks the right camera image with a red filter (S422), binarizes the obtained image (S424), and extracts a difference from the master image (S426). Then, noise (portion overlapping the plate-like member 34) is deleted from the obtained image (S428). Similarly, the control device 11 masks the left captured image with a blue filter (S432), binarizes the obtained image (S434), and extracts a difference from the master image (S436). ), Noise (portion overlapping the plate-like member 34) is deleted from the obtained image (S438). Thereafter, the control device 11 combines the image obtained by applying the red filter and the image obtained by applying the blue filter (S440), and detects the position of the center of gravity of the operator's hand from the obtained image. (S445), the position of the sampling port containing the hand is determined (S450).

  After that, if the determination result is the same sampling port, the control device 11 writes the sampling port position in the database (S500). In this way, the control device 11 performs image analysis.

  The control device 11 of the production navigation system 10 accumulates work data acquired at the time of product assembly work in the accumulating unit 73, manages work results in the managing unit 71 based on the accumulated work data, and displays a display information creating unit. In 72, a data management operation can be performed in which display information representing work performance is created at an arbitrary timing. At that time, the management unit 71 can manage work data and work results based on the code input from the code input unit 19. In addition, the management unit 71 assigns a semi-finished product code to the semi-finished product that is an intermediate product at an arbitrary timing, and completes the work product and the finished product code and semi-finished product that are given to the finished product that is the final product. The code can be associated and registered in the management information file of the storage unit 73.

  Hereinafter, a data management operation of the production navigation system 10 for realizing such processing will be described with reference to FIG. FIG. 16 is a flowchart showing the data management operation of the production navigation system 10.

  As shown in FIG. 16, the control device 11 of the production navigation system 10 determines whether or not the work is started (S605), and when it is determined that the work is started (in the case of “Yes”), the work data is accumulated. Start (S610).

  Next, the control device 11 determines whether or not there is an instruction to give a semi-finished product code (S615). If it is determined in S615 that there is no instruction to give a semi-finished product code (in the case of “No”), the process proceeds to S625. On the other hand, when it is determined in S615 that there is an instruction to give a semi-finished product code (in the case of “Yes”), the control device 11 assigns an arbitrary semi-finished product code to the semi-finished product, and the database The information is registered in the management information file D11 (see FIG. 17) stored in the (accumulating unit) 11a (see FIG. 17) (S620).

  FIG. 17 is a diagram illustrating an example of the management information file D11. In the example shown in FIG. 17, the management information file D11 is registered with information such as semi-finished product code, worker ID, order ID, station ID, work date, start time, end time, elapsed time, and finished product code. It has become.

  Returning to FIG. 16, in the case of “No” in the determination of S615 or after S620, the control device 11 determines whether or not there is an instruction to display the work performance (S625). If it is determined in S625 that there is no instruction to display the work record (in the case of “No”), the process proceeds to S635. On the other hand, when it is determined in S625 that the operation result display instruction is given (in the case of “Yes”), the control device 11 analyzes the operation result and displays the analysis result of the operation result on the display 16. (S630).

  When the determination in S625 is “No” or after S630, the control device 11 determines whether or not there is an instruction to end the work (S635). If it is determined in S635 that there is no instruction to end the work (in the case of “No”), the process returns to S615. On the other hand, when it is determined in S635 that there is an instruction to end the work (in the case of “Yes”), the control device 11 ends the accumulation of the work data (S640), and the work performance analysis / end screen IM41. (See FIG. 21) is displayed on the display 16 (S645). Thereby, a series of routine processing ends.

  Such a production navigation system 10 can detect the sampling of parts satisfactorily. Note that the production navigation system 10 can detect not only the movement of the hand but also the collection of the part by detecting the part. The production navigation system 10 can also detect that an operator has collected a plurality of parts at a time.

  Such a production navigation system 10 can prevent the operator from determining that the parts have been collected even if the parts are not actually collected. Further, when the production navigation system 10 detects a mistake in the parts, the production navigation system 10 reports an alarm by reporting an alarm sound from the speaker 18 or presenting an alarm display on the plate-like members 33 and 34. be able to.

  Further, the control device 11 of the production navigation system 10 displays, for example, work results analysis results IM21 to IM25 (see FIGS. 19A to 19E) corresponding to the items BT11 to BT15 of the main screen IM20 shown in FIG. can do. FIG. 18 is a diagram illustrating an example of the main screen IM20. 19A to 19E are diagrams illustrating examples of screens IM21 to IM25 of analysis results of work results, respectively.

  In the example shown in FIG. 18, the main screen IM20 has contents including items BT11 to BT15. “Item BT11” is a column for designating display of the production status. “Item BT12” is a column for designating display of production results for each order ID of a target worker designated as a target of work results. “Item BT13” is a column for designating display of a comparison result of work contents between the target worker and the skilled worker. “Item BT14” is a column for designating display of the work results by day of the target worker. “Item BT15” is a column for designating display of work results by process of the target worker. The target worker is a person who is working at the current work site (work station) unless a person is specified. However, the production navigation system 10 can also designate a person as a target worker by inputting the ID code of an arbitrary person using the code input unit 19 (see FIG. 2) or a keyboard.

  The main screen IM20 is displayed on a display 16a configured by a touch panel display, for example. For example, when any of the items BT11 to BT15 is designated on the display 16a, the production navigation system 10 displays a screen of analysis results of work results corresponding to the designated item on the display 16b.

  For example, when “item BT11” in FIG. 18 is designated, the production navigation system 10 displays the screen IM21 shown in FIG. 19A on the display 16b. The screen IM21 shows the work performance of the production status. In the example shown in FIG. 19A, the screen IM21 includes “today's target number”, “scheduled” number, “result” number, and “difference” number.

  For example, when “item BT12” in FIG. 18 is designated, the production navigation system 10 displays the screen IM22 shown in FIG. 19B on the display 16b. The screen IM22 represents production results for each order ID of the target worker as bar graph data. In the figure, the horizontal axis represents the product order ID, and the vertical axis represents the production time (seconds).

  For example, when “item BT13” in FIG. 18 is designated, the production navigation system 10 displays the screen IM23 shown in FIG. 19C on the display 16b. The screen IM23 represents a comparison result of work contents between the target worker (comparator) and the skilled worker. The screen IM23 has a content to be displayed while comparing a moving image of the target worker (comparator) working with the target worker (comparator) photographed by the USB cameras 14a and 14b with a previously prepared moving image of the skilled worker. ing. In FIG. 19C, the four rectangular portions on the side of the comparator are regions where moving images during the work of the target worker (comparator) are displayed. In addition, the four square portions on the expert side are areas in which moving images during the work of the expert are displayed. In addition, on the right side of the area where each moving image is displayed, a three-stage square image is arranged. The uppermost square image represents the position of the hand detected from the left captured image (image captured by the left USB camera 14a). The image of the lowermost square represents the position of the hand detected from the right captured image (image captured by the right USB camera 14b). The middle square image represents the position of the operator's hand determined based on the AND condition between the position of the hand detected from the left captured image and the position of the hand detected from the right captured image.

  For example, when “item BT14” in FIG. 18 is designated, the production navigation system 10 displays the screen IM24 shown in FIG. 19D on the display 16b. The screen IM24 represents the daily work performance of the target worker as bar graph data. In the figure, the horizontal axis indicates the date, and the vertical axis indicates the production time (minutes). The bar graph data is data including process information representing each process of the product assembly work and time information required for each process. Each bar graph data represents a width between a minimum work time and a maximum work time of a certain product produced by the target worker. A square in each bar graph data represents an average value of the minimum work time and the maximum work time. It is shown that the process having a longer process time has a larger process time variation as the bar graph has a longer length.

  For example, when “item BT15” in FIG. 18 is designated, the production navigation system 10 displays the screen IM25 shown in FIG. 19E on the display 16b. The screen IM25 represents the work results of each process of the target worker as bar graph data. In the figure, the horizontal axis indicates the process number, and the vertical axis indicates the production time (seconds). Each bar graph data represents a width between a minimum work time and a maximum work time of a certain product produced by the target worker. One of the two squares in each bar graph data is an average value of the minimum work time and the maximum work time, and the other represents the previous work time. It is shown that the process having a longer process time has a larger process time variation as the bar graph has a longer length.

  The control device 11 can create graph data of the screen IM24 (FIG. 19D) and the screen IM25 (FIG. 19E) for each semi-finished product and display the graph data on the display 16b. The control device 11 can also create graph data for each worker and display the graph data on the display 16b. An administrator or the like can grasp which process is taking time by referring to the graph data.

  The analysis result of the work results performed by the control device 11 of the production navigation system 10 is provided to the data collection device 61 (see FIG. 1) of the support system 60. As a result, the analysis result of the work results is browsed not only by the control device 11 of the production navigation system 10 but also by the data collection device 61, the data processing device 62, the support device 63, etc. constituting the support system 60 (see FIG. 1). be able to.

  Further, the control device 11 of the production navigation system 10 can analyze the operation of the worker based on the moving image of the worker taken by the motion capture camera 20 during the product assembly operation. For example, as shown in FIG. 20, the display information creation unit 72 of the control device 11 of the production navigation system 10 is based on the worker's moving image captured by the motion capture camera 20 during product assembly work. The line graph data shown in FIG. 20 can be created as display information by specifying the movement distance of the hand. FIG. 20 is a diagram illustrating an example of the motion analysis screen IM31 as a result of analyzing the worker's motion. In FIG. 20, the line graph data represents the correlation between the frame information of the moving image and the movement distance information of the worker's head and hand. The motion analysis screen IM31 is displayed on the display 16b, for example.

  In the example illustrated in FIG. 20, the motion analysis screen IM31 includes a moving image display unit Mo1 that displays a moving image, a command unit Co1 that displays a command instructing the operation of the moving image, and a graph display unit that displays line graph data. The content includes Gr1. The moving image in the moving image display section Mo1 displays small circle-shaped additional information representing analysis results such as the flow line of the worker and the orientation of the face. In the line graph data in the graph display portion Gr1, the horizontal axis indicates the number of frames of the moving image, and the vertical axis indicates the moving distance of the head or hand. Moreover, the vertical line in the graph display part Gr1 has shown the division part of each process, and the number represents the process number.

  Note that the analysis of the worker's motion such as the motion analysis screen IM31 shown in FIG. 20 is not limited to the control device 11 of the production navigation system 10, but the data collection device 61 and the data constituting the support system 60 (see FIG. 1). It can also be performed by the processing device 62, the support device 63, and the like.

  Further, the control device 11 of the production navigation system 10 can create a screen (e.g., an end screen IM41 (see FIG. 21)) indicating the elapsed time and the like based on the management information file D11 (see FIG. 17). FIG. 21 is a diagram illustrating an example of the end screen IM41. In the example illustrated in FIG. 21, the end screen IM41 includes a standard time column representing standard time, a productivity column representing production rate, an elapsed time column representing elapsed time, a start time column representing start time, and an end time representing end time. The contents include a column.

<Main functions of the production navigation system>
The production navigation system 10 has a control function, an image (moving image) shooting (capture) function, a voice recognition function, and a voice instruction function.
The production navigation system 10 includes a guidance function, an image acquisition function during component picking (picking) and its determination function, an image acquisition function during component installation and its determination function, a work flow line acquisition function and its analysis function, etc. have.
The production navigation system 10 outputs captured image (moving image) information, input voice information, and the like to the data collection device 61 of the support system 60, and these information is converted into big data to the data processing device 62 and the support device. (PC) 63 can perform analysis processing.
The production navigation system 10 can expand the image analysis function and the voice recognition function later.
The production navigation system 10 can guide assembly work by presenting various information to the worker.
The production navigation system 10 can analyze the type and number of parts collected by the worker, the details of work performed by the worker, and the like.
The production navigation system 10 can report an alarm sound or present alarm information.
As shown in FIG. 21, the production navigation system 10 can display the progress status of the assembly work on the display 16 at an arbitrary timing, for example, at the end of the assembly or in the middle of the assembly process.

<Main functions of the support system>
The data collection device 61 can acquire and store shooting information and audio information from the control device 11 of the production navigation system 10. The administrator or the like can check the state of the assembly work site such as the production progress from a remote place by browsing the shooting information stored in the data collection device 61 with the support device (PC) 63.

<Main features of production navigation system>
(1) About projection mapping (a) The production navigation system 10 can present (project) the position and quantity of parts to be collected by an operator. For example, in the production navigation system 10, the arrow mark Mk (see FIG. 8) including the arrow information indicating the position of the component placement unit 21 (collection port) on which the component 41 to be collected is placed and the number information of the components is included. Etc. can be projected. As a result, the production navigation system 10 can easily notify the operator of how many parts 41 should be collected from which sampling port by simply glance at the mark Mk (see FIG. 8).
(B) The production navigation system 10 can determine display information to be displayed on the display mechanism 15 in accordance with the position of the specific part of the worker photographed by the USB camera 14. For example, when the operator collects a part, the production navigation system 10 can automatically reduce the count quantity of the part placed on the shelf according to the number of the collected parts (collected parts). . In addition, the production navigation system 10 can present (project) display information for the next process. The production navigation system 10 also displays information according to the command corresponding to the input position when the operator's hand is held over the command input position of the special command input unit 23 (see FIG. 2). Can be determined.
(C) The production navigation system 10 can present (project) the position and quantity of the next part when the operator collects the part.
(D) The production navigation system 10 presents warning information such as error display information (see FIG. 7D) when the component placement unit 21 to be put in is wrong. As a result, the production navigation system 10 can present (project) the alert information even when the operator mistakes the parts.
(E) The production navigation system 10 can change the presentation position of the alert information for each part position.
(F) The production navigation system 10 receives at least one of the moving image representing the product assembly work, the manual image, the number information of the parts 41, and the product assembly process information from the projector 31 as display information. Can be projected. Thereby, the production navigation system 10 can efficiently support the product assembly work.
(G) The projector 31 corrects the display information and projects it on the surface of the plate members 33 and 34 and the work table 13 in a shape approximate to a rectangle. Thereby, the production navigation system 10 can present the display information with low distortion and high visibility to the operator.
(H) The production navigation system 10 includes a work acquisition mode for displaying navigation information from the projector 31 that includes work animation as display information and instruction information with detailed explanations regarding various items. It is possible to project selectably an expert mode in which there is no moving image and navigation information having instruction information with only explanations on relatively important matters is displayed. Thereby, the production navigation system 10 can change navigation according to the mastery of the assembly work of the product.
(I) The projector 31 can display (project) arrow information indicating the position of the component placement unit 21 on which the component 41 to be collected is placed as display information above the component placement unit 21. . Thereby, the projector 31 can display (project) the arrow information at a position that does not interfere with the hand detection.
(J) The production navigation system 10 can easily and inexpensively increase the number of parts for displaying the display information only by increasing the number of the plate-like members 34.
(K) The production navigation system 10 changes the angle of the plate member 34 according to the presentation (projection) position. The plate-like member 34 arranged at the lower position is in a state in which the plate-like member 34 is laid down. Thereby, the visibility of display information can be improved.
(L) The color of the plate-like members 33 and 34 on which the display information is projected is preferably white. Thereby, display information can be clearly presented (projected), and the visibility of display information can be improved.
(M) The production navigation system 10 can present on the plate-like members 33 and 34, the display 16 and the like what percentage of the entire process is completed when the collection and assembly of the parts are completed.
(N) The production navigation system 10 can project information such as worker ID, temporary ID, work progress information, position of parts to be collected, quantity, work procedure (including image diagrams), necessary assembly tools, and the like. it can.
(O) When the production navigation system 10 detects collection of necessary parts, the production navigation system 10 can manage the quantity of parts by reducing the quantity of parts.
(P) When the production navigation system 10 detects the collection of a necessary part, the production navigation system 10 can present the position and quantity of the next part as display information.
(Q) The production navigation system 10 can project the alert information if the collection is wrong. The production navigation system 10 can also detect when the operator returns the part.
(R) When presenting the alert information, the production navigation system 10 can adjust and present the alert information to an optimum position according to the position of the part.
(S) The production navigation system 10 can use the surface of the work table 13 as a projection unit.
(T) The production navigation system 10 can present the position and the like of the next part to the operator when the part collection is detected. The production navigation system 10 can also present the position of the next part to the worker when all the work procedures related to the desired task are completed.
(U) The production navigation system 10 may switch display information after completion of assembly using the electric screwdriver.

(2) Configuration of shelf (a) The shelf 12 is provided with a plurality of partition plates 22. The partition plate 22 has a shape including an inclined surface 22aa, 22ab, 22ac, 22ad (see FIG. 4B) or an R surface (not shown) in the upper portion. Thereby, the production navigation system 10 can partition adjacent component placement portions 21 with each partition plate 22, and is easy to insert when an operator's hand is inserted into the component placement portion 21. Can be secured.
(B) The plate-shaped member 34 is inclined and arranged such that the lower the plate-like member 34 is placed, the more the plate-like member 34 is placed (falling state) (FIG. 4B). reference). Thereby, the production navigation system 10 can favorably project the display information reflected by the mirror 32 onto the plate member 33.

(3) Image analysis of hand position (a) The production navigation system 10 can perform arbitrary image processing (for example, image acquisition processing, binarization processing, centroid detection processing, position detection processing, etc.) based on the captured image. By performing the above, image analysis of the hand position can be performed.
(B) The production navigation system 10 changes the color of the part as the background (for example, the color of the plate-like members 33 and 34, the surface of the work table 13, the partition plate 22 and the like) to a white color, The accuracy of identifying the background can be improved.

(4) Image detection (a) The production navigation system 10 can perform image determination satisfactorily even when there is a change in the environment (external brightness).
(B) The production navigation system 10 can suppress a decrease in detection accuracy (for example, a decrease in detection accuracy of whitish clothing) due to a difference in clothes.
(C) The production navigation system 10 can accurately detect the position of the hand by analyzing the captured image even when the hand is inserted into the sampling port from an oblique direction.
(D) The production navigation system 10 can identify the insertion amount (movement amount in the depth direction) of the operator's hand with respect to the sampling port by analyzing the captured image.
(E) The production navigation system 10 can use a commercially available general-purpose program as an image analysis program.

(5) Image Analysis (a) The production navigation system 10 can detect a mistake by image analysis even if an operator mistakes a part. In this regard, the prior art can identify only whether or not the operator's hand has passed the sensor provided at the sampling port (that is, whether or not the operator's hand has been inserted into the sampling port). Therefore, even if the operator did not actually collect the part, there was a possibility that the part was erroneously detected as being collected. Further, in the prior art, when an operator collects a plurality of parts, the apparatus cannot detect it, and as a result, there is a possibility that the count quantity of the stored parts is wrong. . On the other hand, the production navigation system 10 according to the present embodiment analyzes whether or not the worker has actually collected the component by analyzing the moving image at the time of collecting the component, and how many components the operator has collected. Can be identified. As a result, the production navigation system 10 can eliminate erroneous detection of an uncollected part as being collected and an erroneous count quantity of stored parts.
(B) The production navigation system 10 can identify that the worker has collected a plurality of parts by image analysis. In this regard, for example, when the operator collects two of the same parts, it is necessary to put the hand in and out of the sampling port twice. On the other hand, the production navigation system 10 according to the present embodiment can identify that two parts have been sampled by only putting the hand in and out of the sampling port once by the operator.
(C) The production navigation system 10 can confirm completion of parts assembly by image analysis. With respect to this point, the prior art only manages whether or not the assembly of parts has been completed only by the operator's declaration, forcing the operator to perform processing. On the other hand, the production navigation system 10 according to the present embodiment can automatically manage whether or not the parts assembly is completed without an operator reporting.
(D) The production navigation system 10 can detect a worker's skin color, a worker's work clothes, and the like that perform image analysis using a plurality of types (for example, two types) of filters. Then, the production navigation system 10 binarizes the photographed image, for example, and detects the position of the center of gravity (the root portion between the index finger and the middle finger in the illustrated example), thereby determining the position of the operator's hand. Can be identified.

(6) Centralized management of data (a) The support system 60 can store captured images in a batch by the data collection device 61.
(B) The support system 60 improves the work in a short period of time (for example, the work is performed) by analyzing the work content at each work site by an administrator or the like based on the captured image stored in the data collection device 61. Reduction of time variation and adjustment of line balance can be realized. That is, the support system 60 can confirm the assembly operation and the like from a remote location. With respect to this point, for example, in the prior art, it is necessary for an administrator or the like to go to the assembly work site and check the assembly operation or the like. On the other hand, the production navigation system 10 according to the present embodiment can confirm the completion of assembly from a remote location without the administrator or the like going to the assembly work site.
(C) The support system 60 can confirm the operation of a process with a variation in work time from a remote place.

(7) Voice confirmation (a) The production navigation system 10 can also confirm the completion of the assembling operation by acquiring the voice information of the operator with the microphone 17.

(8) Selection of assembly tool (a) The production navigation system 10 can confirm that tightening has been completed with an electric screwdriver.
(B) The production navigation system 10 can instruct the operator to select a desired electric driver when the torque of the electric driver is different. The production navigation system 10 also displays an error message as display information so that the process does not proceed to the next step when a plurality of electric drivers are provided and the operator makes a mistake in selecting an electric driver to use. Can be displayed to alert the operator.
(C) The production navigation system 10 can automatically adjust the torque of the electric driver by changing the current value according to the type of screw.

(9) Parts management, work management, etc. (a) The production navigation system 10 is low-priced by performing projection mapping with the projector 31 and presenting information, or performing image processing of a captured image taken with the USB camera 14. Thus, functions such as work instructions, production status presentation and alerting can be realized.
(B) The production navigation system 10 includes a management unit 71 (see FIG. 1), a display information creation unit 72 (see FIG. 1), and a storage unit 73 (see FIG. 1). Work data acquired at the time of product assembly work is stored in the storage unit 73, work results are managed by the management unit 71 based on the stored work data, and display information representing work results is arbitrarily displayed by the display information creation unit 72 It is possible to realize a data management method characterized by being created at the timing of Further, when the management unit 71, the display information creation unit 72, and the storage unit 73 are provided in the system, the support system 60 can realize the same data management method by using them. Thereby, for example, the production navigation system 10 and the support system 60 can confirm whether or not there is a missing part in the work site, whether the screw tightening or adjustment work, the test result, or the like is good.
(C) The production navigation system 10 can realize the strictness of the work history and the improvement of efficiency due to the use of IT.
(D) The production navigation system 10 can display the work time at the assembly site in real time. Thereby, improvement of work in a short period of time (for example, reduction of work time variation, adjustment of line balance, etc.) can be realized.
(E) The production navigation system 10 can give a semi-finished product code to the semi-finished product that is an intermediate product at an arbitrary timing, and can input the code from the code input unit 19 (see FIG. 2), a keyboard, or the like. The production navigation system 10 and the support system 60 can manage work data and work results based on the input code. For example, the production navigation system 10 and the support system 60 manage the storage unit 73 (see FIG. 1) by associating the work data with the finished product code assigned to the finished product that is the final product and the semi-finished product code. By registering in the information file D11 (see FIG. 17), it is possible to read work data related to the product associated at an arbitrary timing. As a result, the production navigation system 10 and the support system 60, for example, information about which worker has assembled which semi-finished product based on the finished product code of the finished product that is the final product, Work moving images and the like can be read from the database and verified. In addition, the production navigation system 10 and the support system 60, for example, read a work moving image or the like of the worker with the ID from the database based on the worker ID, and verify the quality of the work of the worker with the ID. Can do.
(F) The production navigation system 10 and the support system 60 can display the work moving image of the unskilled worker and the work moving image of the skilled worker on the display so as to be comparable (see FIG. 19C). . Thereby, the production navigation system 10 and the support system 60 can easily verify the task of the work performed by each worker (for example, a factor that requires time in each process, the suitability of the work content, etc.).
(G) The production navigation system 10 and the support system 60 can analyze the operation of the worker based on the moving image of the worker taken by the motion capture camera 20 during the product assembly operation (see FIG. 20). .
(H) Since the production navigation system 10 does not require the parts kit operation, it is possible to delete the parts arrangement work and the parts distributor. Therefore, the production navigation system 10 can reduce production costs.
(I) The production navigation system 10 can automatically leave the work history as electronic data by using information from a sensor, an electric driver, or the like or analyzing the work contents of the worker. As a result, the production navigation system 10 reduces the labor for creating the check sheet (that is, the trouble of writing the work history for each worker in the check sheet (work item list list), or performing the stamp or signature). Can be reduced). In addition, paperlessness can be promoted. Therefore, also by this, the production navigation system 10 can reduce production costs.
(J) The production navigation system 10 can respond immediately even if the operator is changed.
(K) The display information creation unit 72 (see FIG. 1) creates, as display information, graph data including process information representing each process of product assembly work and time information required for each process. Can be displayed (see, for example, FIGS. 19B, 19D, and 19E). Further, the display information creation unit 72 (see FIG. 1) creates and displays work image information that displays the work video image of the skilled worker and the work video image of the unskilled worker as display information so as to be comparable. (See, for example, FIG. 19C). In addition, the display information creation unit 72 (see FIG. 1) includes a work acquisition mode navigation information including a work movie as display information and instruction information with detailed explanations regarding various items, and a work movie. In addition, the navigation information of the skilled mode having the instruction information with only the explanation about the relatively important matter is created and can be displayed in a selectable manner (see FIG. 7A). The display information creation unit 72 (see FIG. 1) specifies the movement distance of the worker's hand based on the worker's moving image taken by the motion capture camera 20 during the product assembly operation, and displays the display information. As shown in FIG. 20, graph data representing the correlation between the frame information of the moving image and the movement distance information of the operator's hand can be created and displayed (see FIG. 20). The production navigation system 10 and the support system 60 can efficiently support the product assembly work by providing such a display information creation unit 72 (see FIG. 1) in the system.

(10) Function extensibility (a) The production navigation system 10 can add a USB camera 14 and the like. Thereby, the production navigation system 10 can also confirm the presence or absence of a nameplate missing item, for example.
(B) The production navigation system 10 can add a microphone 17 and a speaker 18. Thereby, the production navigation system 10 can improve the accuracy of voice recognition and the suitability of voice instructions.
(C) The production navigation system 10 can appropriately update various programs (for example, a control program, an image analysis program, a voice recognition program, etc. of the production navigation system 10).
(D) The production navigation system 10 may separately include a lighting device around the work table 13. Thereby, even if the brightness around the workbench 13 changes depending on the place, time zone, etc., the production navigation system 10 can acquire a photographed image with good brightness and analyze the photographed image. Accuracy can be improved.

  As described above, according to the production navigation system 10 according to the present embodiment, it is possible to present suitable information to the assembly operator when the apparatus is produced.

In addition, this invention is not limited to above-described embodiment, A various change and deformation | transformation can be performed in the range which does not deviate from the summary of this invention.
For example, the above-described embodiment has been described in detail in order to easily understand the gist of the present invention. Therefore, the present invention is not necessarily limited to the one provided with all the constituent elements described. Further, according to the present invention, other components can be added to a certain component, or some components can be changed to other components. In the present invention, some components can be deleted.

  Also, for example, the flowcharts shown in FIGS. 5, 6, 15, and 16 can be changed as appropriate according to the operation. Further, for example, the contents of the display screen, the graph data, and the like can be changed as appropriate according to the operation.

10 Production navigation system 11 Control device (PC)
11a database (storage unit)
12 shelves (parts warehouse)
13 Working table 14 (14a, 14b) Imaging unit (USB camera)
15 Display mechanism (display unit)
16 (16a, 16b) Display 17 Microphone 18 Speaker 19 Code input unit 20 Motion capture camera 21 (21a, 21b, 21c, 21d) Component placement unit 22 (22a, 22b, 22c, 22d) Partition plate 22aa, 22ab, 22ac , 22ad Inclined surface 23 Command input unit 31 Projector 32 Mirror 33, 34 Plate member (projection unit)
40 tray 41 parts 44 support frame 45 rail (hanging part)
46 Assembly tool (electric driver)
60 Support System 61 Data Collection Device 62 Data Processing Device 63 Support Device (PC)
71 management unit 72 display information creation unit 73 storage unit 81 acquisition unit 82 storage unit 83 analysis unit 84 provision unit DP imaging data D1 master image acquisition D2 captured image acquisition D3 binarization processing D11 management information file Gu guidance Mk mark P0 centroid detection R1, R2, R3 Position detection P1 position

Claims (28)

A shelf provided with a plurality of component placement units for placing components;
A display unit for presenting display information to the worker;
A shooting section for shooting workers;
A control device for controlling the operation of the display unit,
The said control apparatus determines the display information displayed on the said display part according to the position of the specific site | part of the operator image | photographed by the said imaging | photography part, The production navigation system characterized by the above-mentioned. The production navigation system according to claim 1,
A command input position is preset in the shelf,
The said control apparatus determines display information according to the command corresponding to an input position, when an operator's hand is put in the input position, The production navigation system characterized by the above-mentioned. The production navigation system according to claim 1,
The shelf is provided with a plurality of partition plates,
Each of the partition plates is arranged in a substantially vertical direction so as to divide each of the component mounting portions and protrude forward from the front surface of the shelf. The production navigation system according to claim 1,
The production navigation system according to claim 1, wherein a portion of the display unit for displaying the display information is disposed obliquely on the front surface of the shelf so as to extend in the left-right direction. The production navigation system according to claim 1,
The production navigation system according to claim 1, wherein the photographing unit is disposed at a position above the uppermost component placement unit and at a position ahead of the front surface of the shelf. The production navigation system according to claim 1,
The number of the photographing units is two or more,
A production navigation system, wherein two or more of the photographing units are arranged apart in the left-right direction. The production navigation system according to claim 1,
The production navigation system according to claim 1, wherein the display information includes information on the number of parts to be collected. The production navigation system according to claim 1,
The production navigation system according to claim 1, wherein the display information includes arrow information indicating a position of a component placement unit on which a component to be collected is placed. The production navigation system according to claim 1,
Based on the difference between the color of the operator's hand and the color of the part displaying the display information on the display unit, the control device selects the operator's hand from the captured images captured by the imaging unit. A production navigation system characterized by detecting. The production navigation system according to claim 9, wherein
The production navigation system characterized in that the color of the part displaying the display information of the display unit is white. The production navigation system according to claim 1,
The production control system, wherein the control device determines display information according to detection information of an operator's hand. The production navigation system according to claim 1,
The production control system, wherein the control device presents error display information when a component placement unit to be put in is wrong. The production navigation system according to claim 1,
The control device detects the position of the operator's hand based on an AND condition between a left captured image captured by the left capturing unit and a right captured image captured by the right capturing unit. Production navigation system. The production navigation system according to claim 13,
The control device sets horizontal and vertical squares corresponding to the individual component placement units provided on the shelf, and sets the hand in each square according to the position of the operator's hand with respect to each square. Production navigation system characterized by determining the position of The production navigation system according to claim 1,
A production navigation system, wherein a hanging portion of an assembly tool is provided at a position in front of the arrangement position of the photographing unit. A shelf provided with a plurality of component placement units for placing components;
A projector for projecting display information;
A production navigation system comprising: a mirror that reflects the display information projected from the projector to a projection unit provided on the shelf. The production navigation system according to claim 16, wherein
The said projector displays the information which points the position of the components mounting part in which the components to extract | collect are mounted as said display information above the said components mounting part. The production navigation system according to claim 16, wherein
The projector includes, as the display information, at least one of a moving image representing a product assembly operation, a manual image, information on the number of components, arrow information indicating the position of the component placement unit, and product assembly process information. A production navigation system characterized by being able to project more than one. The production navigation system according to claim 16, wherein
The projection unit is composed of a plate-like member disposed on the front surface of the shelf so as to extend in the left-right direction,
A production navigation system characterized in that the display information can be displayed simultaneously at a plurality of locations by increasing the number of the plate-like members. The production navigation system according to claim 16, wherein
The projection unit includes a plurality of types of plate-like members having different vertical widths arranged on the front surface of the shelf so as to extend in the left-right direction, and a table surface of a work table arranged in front of the shelf. A production navigation system characterized by A storage unit for storing work data of product assembly work, including a photographed image taken by the production navigation system during product assembly work;
And a providing unit that provides the production navigation system with navigation information displayed as display information in the production navigation system. Work data acquired during product assembly work is stored in the storage unit,
Based on the accumulated work data, manage the work results in the management unit,
A data management method, wherein display information representing the work performance is created at an arbitrary timing by a display information creation unit. The data management method according to claim 22, wherein
The management unit manages the work data and the work results based on a code input from a code input unit. The data management method according to claim 22, wherein
The management unit
Associating the work data with a finished product code given to a finished product as a final product and a semi-finished product code given at an arbitrary timing to a semi-finished product as an intermediate product, A data management method comprising registering in a management information file. The data management method according to claim 22, wherein
The display information creating unit creates, as the display information, graph data including process information representing each process of product assembly work and time information required for each process. Method. The data management method according to claim 22, wherein
The data management method, wherein the display information creation unit creates work image information that displays a work video image of an expert and a work video image of an unskilled person as the display information so as to be comparable. The data management method according to claim 22, wherein
The data management method, wherein the display information creation unit creates, as the display information, navigation information in a work acquisition mode with a work video and navigation information in an expert mode without a work video. The data management method according to claim 22, wherein
The display information creation unit specifies a movement distance of the worker's hand based on a worker's moving image captured by a motion capture camera during product assembly work, and uses the frame information of the moving image as the display information. A data management method characterized by creating graph data representing the correlation between the distance and the movement distance information of the worker's hand.

Images