JP2010197153A - Work support device and work support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a work support device which notifies abnormal components that are difficult to recognize with only the naked eyes, by displaying which component of a work object has abnormal temperature. <P>SOLUTION: The work support device 7 which uses a head-mounted display includes: a visible light ray imaging part 4 imaging visible light rays; an infrared ray imaging part 5 imaging infrared rays; a reporting part 1 reporting an abnormality; a temperature distribution image generating part 61 creating a temperature distribution image from an infrared-ray image obtained by the infrared-ray imaging part; a component detection part 62 detecting a component from a visible-light ray image obtained by the visible light ray imaging part; a proper temperature database 62 storing a proper temperature of each component; a temperature abnormality detecting part 64 detecting the component having abnormal temperature, on the basis of the component detected from the temperature distribution image and the visible-light ray image and the proper temperature of each component; and a report information creation part 65 creating information reported from the result of detecting the component having abnormal temperature. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a work support apparatus using a head-mounted display (hereinafter also referred to as HMD), and a work support system including a work support apparatus and a server apparatus capable of communicating with the work support apparatus.

  Conventionally, various head-mounted display units called HMDs have been proposed. In recent years, HMDs have begun to be used not only as general consumer products but also for use in supporting smooth operations by providing various information to users.

  For example, an operator who performs maintenance work on an apparatus or equipment needs to memorize and learn various trouble contents and maintenance methods and respond according to the situation. In addition, workers must accurately understand the situation at the work site, and if they do not know the appropriate countermeasures, they may need to respond quickly by receiving the necessary advice from the designer. is there. However, it may be difficult to receive accurate advice on the telephone.

  Therefore, a work support system using an HMD has been proposed. For example, Patent Document 1 discloses a work support system that displays work procedures and work instructions on the HMD via a network and gives instructions to the worker. .

  Further, Patent Document 2 discloses a device that uses an HMD capable of displaying an infrared image and a visible light image and captures heat generated by a human body or a fire source in the dark or in smoke during a fire. As a similar technique, there are techniques as described in Patent Documents 3 to 5.

JP 2004-102727 A JP-A-8-54282 JP 2007-286269 A Japanese Utility Model Publication No. 5-38541 Japanese Utility Model Publication No. 5-75892

  Here, consider a case where repair or maintenance work is performed inside an apparatus (for example, a copying machine). In the course of work, it is important that there are dangerous parts that are hot, the temperature of the parts is useful for detecting abnormalities, and it is important to know the temperature of the parts. However, an operator usually performs an operation in which both hands are blocked, and it is difficult to perform an operation while holding a device for temperature measurement.

  Then, although utilization of HMD mentioned above is considered, patent document 1 does not have description regarding temperature measurement. In addition, since an appropriate temperature zone is different for each component in the apparatus, there is a problem that it is not known which component is abnormal just by detecting the temperature as in Patent Documents 2 to 5.

  An object of the present invention is to provide a work support device and a work support system that notify which abnormal part is difficult to understand with the naked eye by notifying which part of the work object has an abnormal temperature.

  In order to achieve the above object, the present invention is a work support device using a head-mounted display, and includes a visible light imaging unit that captures visible light, an infrared imaging unit that captures infrared light, and a notification unit that notifies abnormality The notification unit includes a temperature distribution image generated from the infrared image acquired by the infrared imaging unit, a component detected from the visible light image acquired by the visible light imaging unit, and an appropriate temperature of each component. Based on the above, the information generated from the result of detecting the component having an abnormal temperature is notified.

  According to this configuration, it is possible to search for a part having an abnormal temperature from the field of view of the worker who is repairing the device, and notify the abnormality through the HMD.

  Further, the present invention is a work support system including the work support device described above and a server device capable of communicating with the work support device, wherein the temperature distribution generates a temperature distribution image from the infrared image acquired by the infrared imaging unit. An image generation unit, a component detection unit that detects a component from a visible light image acquired by the visible light imaging unit, an appropriate temperature database that stores an appropriate temperature of each component, and a detection from the temperature distribution image and the visible light image A temperature abnormality detection unit that detects a component having an abnormal temperature based on the detected component and an appropriate temperature of each component, and a notification information generation unit that generates information to be notified from a result of detecting the component having the abnormal temperature Is provided in either the work support device or the server device.

  According to this configuration, if many components are provided on the server device side, it is possible to reduce the size and cost of the work support device.

  The present invention is also a work support device using a head-mounted display, which includes a visible light imaging unit that captures visible light, an infrared imaging unit that captures infrared light, a notification unit that notifies abnormality, and the infrared imaging unit. A temperature distribution image generation unit that generates a temperature distribution image from the infrared image acquired in step 1, a component detection unit that detects a component from the visible light image acquired by the visible light imaging unit, and an appropriate temperature that stores an appropriate temperature of each component A temperature abnormality detecting unit for detecting a part having an abnormal temperature based on a database, a part detected from the temperature distribution image and the visible light image, and an appropriate temperature of each part; and a part having the abnormal temperature. And a notification information generation unit that generates information to be notified from the detected result.

  According to this configuration, it is possible to search for a part having an abnormal temperature from the field of view of the worker who is repairing the device using only the work support device without using the server device, and to notify the abnormality through the HMD.

  The notifying unit notifies the worker by using at least one of highlighting of the abnormal part, displaying a character indicating abnormality, displaying a symbol indicating abnormality, a voice message, and a beep sound. You can call attention.

  According to the present invention, by notifying which part of the work object has an abnormal temperature, it is possible to notify an abnormal part that is difficult to understand with the naked eye alone.

  In addition, by notifying the high-temperature location, the operator can be alerted and safety during work can be improved.

It is a perspective view which shows schematic structure of HMD which comprises the work assistance apparatus of this invention. It is a side view of HMD in the state where a worker equipped with HMD of the present invention. It is sectional drawing which shows schematic structure of the display part of this invention. It is a block diagram which shows schematic structure of the work assistance apparatus of this invention. 6 is an example of a display image of the present invention when a heating roller in a fixing unit of a copying machine is at an appropriate temperature or higher. 6 is an example of a display image of the present invention when a heating roller in a fixing unit of a copying machine is at an appropriate temperature or higher. 6 is an example of a display image of the present invention when a heating roller in a fixing unit of a copying machine is at an appropriate temperature or higher. It is a block diagram which shows schematic structure of the work assistance system of this invention.

  FIG. 1 is a perspective view showing a schematic configuration of the HMD constituting the work support apparatus, and FIG. 2 is a side view of the HMD in a state where the HMD is worn by an operator. The HMD includes a display unit 1, a support unit 2, a cable 3, a visible light imaging unit 4, and an infrared imaging unit 5. The work support device includes an HMD and a control unit connected to the cable 3. The display unit 1 and an earphone 25 described later are collectively referred to as a notification unit.

  The display unit 1 displays an image and provides it to the operator, and the support unit 2 supports the display unit 1 in front of the operator's eyes. 1 and 2, the support unit 2 supports the display unit 1 so as to be positioned in front of the operator's right eye. However, two display units 1 are provided, and these are positioned in front of both eyes. You may support it. In any case, by supporting the display unit 1 with the support unit 2, the operator can observe the image displayed on the display unit 1 in a hands-free manner. The cable 3 connects a control unit (not shown), the display unit 1, the visible light imaging unit 4, and the infrared imaging unit 5, and displays at least driving power and video signals from the control unit with the display unit 1 and the visible light imaging unit 4. And the infrared imaging unit 5.

  The support unit 2 includes a temple 21 and a frame 22. The temple 21 is composed of a pair of left and right temples 21R and 21L that come into contact with the side of the worker. The frame 22 supports the temples 21R and 21L so as to be rotatable, and supports the display unit 1 described above, and is formed in a substantially T-shape. The temple 21 and the frame 22 are made of resin, for example, and have flexibility. The frame 22 is provided with a pair of left and right nose pads 23 that come into contact with the nose of the operator.

Temple 21R includes a first temple portion 21R _{1, 2} and a second temple portion 21R is configured by integrally. Similarly, the temple 21L includes a first temple portion 21L _1, and a second temple portion 21L ₂ is configured by integrally. The first temple portions 21R ₁ and 21L ₁ and the second temple portions 21R ₂ and 21L ₂ are formed in a long shape in the front-rear direction, and the second temple portions 21R ₂ and 21L ₂ are the first temple portions. It is formed slightly longer than 21R ₁ · 21L ₁ .

The first temple portions 21R ₁ and 21L ₁ are pivotally connected to the frame 22 at the left and right ends of the frame 22 and are displaced forward with respect to the second temple portions 21R ₂ and 21L ₂ . It is connected so that it may be located in the state above.

The second temple portions 22R ₂ and 21L ₂ are supported by the frame 22 via the first temple portions 21R ₁ and 21L ₁ . The second temple portion 21R ₂ has an earphone holding portion 24 that holds the earphone 25 so as to be movable in the front-rear direction, and the operator holds the earphone 25 in the earphone holding portion 24 (the earphone 25 is heard in the ear). It is possible to listen to the audio even if it is not plugged in.

The second temple portions 22R ₂ and 21L ₂ have exposed surfaces 21R _S and 21L _S that are exposed forward at a height position different from the support position of the first temple portions 21R ₁ and 21L ₁ with respect to the frame 22. Yes. The cable 3 described above passes, for example, through the second temple portion 21R ₂ , is pulled out from the exposed surface 21R _S and connected to the display unit 1, and further passes through the display unit 1 and the visible light imaging unit 4 and infrared imaging. Connected to the unit 5. With such a connection method, the following first to fourth effects can be obtained.

First, since the cable 3 is connected to the display unit 1 once supported by the temple 21R (second temple unit 21R ₂ ), as in the case where the cable 3 is directly connected to the display unit 1, The weight of the cable 3 is not directly applied to the display unit 1. As a result, even if the worker uses the HMD for a long time, it is possible to avoid an increase in feeling of fatigue and discomfort, and the use burden on the worker for a long time use can be reduced.

In particular, in this embodiment, the temple 21R has a two-stage configuration of the first temple portion 21R ₁ and the second temple portion 21R ₂ , so that the position of the temple 21R supported by the frame 22 (the first temple portion 21R ₁ and the frame a connecting position) with 22, it can be varied to ensure the position of the exposed surface 21R _S in height direction. Accordingly, the cable 3 can be reliably pulled out from the inside of the second temple portion 21R ₂ to the outside through the exposed surface 21R _S in a state where the frame 22 and the first temple portion 21R ₁ are rotatably connected. Thus, it is possible to reliably employ the above connection method.

  Second, since the temple 21R can support the cable 3 in a stable state by abutting with the temporal side of the operator, when the cable 3 is supported by one temple 21R as in the present embodiment. However, the operator can stably wear the HMD (the HMD does not wobble from front to back and from side to side).

Third, the cable 3, it means that through different heights from the frame 22 via the exposed surface 21R _S of the temple 21R, during turning of the temple 21R relative to the frame 22, the cable 3 to the rotation part The winding does not hinder the rotation of the temple 21R. Therefore, the temple 21R can be easily folded.

  Fourth, since the cable 3 does not pass through the inside of the frame 22, it is not necessary to form the frame 22 thicker than necessary. Thereby, the frame 22 is easily deformed in accordance with the size of the operator's head, and the operator can stably wear the HMD without feeling uncomfortable.

As shown in FIG. 2, the cable 3 enters the second temple portion 21R ₂ from a position behind the position where the second temple portion 21R ₂ comes into contact with the operator's ear. As a result, the cable 3 hangs behind the operator's ear, so that the weight balance in the front-rear direction with respect to the operator's ear can be improved, and the usage burden on the operator can be reliably reduced. . In addition, the cable 3 does not obstruct the side view of the operator and obstruct it.

Further, the first temple portion 21R ₁ and the second temple portion 21R ₂ are formed in an elongated shape in the front-rear direction, and the first temple portion 21R ₁ is displaced forward with respect to the second temple portion 21R ₂ . Since the first temple portion 21R ₁ and the second temple portion 21R ₂ are formed to be thin with a minimum necessary thickness (height), the entire temple 21R is made small and lightweight. can do. In addition, when the first temple portion 21R ₁ is rotated with respect to the frame 22, the second temple portion 21R ₂ is folded inward from the first temple portion 21R ₁ , and the second temple portion 21R ₂ is The HMD can be stored compactly without projecting outside.

Further, since the first temple portion 21R ₁ is positioned above the second temple portion 21R _2, when the HMD is mounted on the operator's head (the second temple portion 21R ₂ is the ear of the operator). The first temple portion 21R ₁ is positioned above the operator's eyes when it is positioned at the top. Therefore, the frame 22 connected to the first temple portion 21R ₁ can be positioned above the operator's eyes at least from the front to the side of the operator, and the worker's field of view can be secured widely. It becomes. Further, even when the operator wears general glasses, the HMD frame 22 is less likely to interfere with the temples and frames of the glasses, and the combined use of the HMD and glasses is facilitated.

Meanwhile, in the second temple portion 21R ₂ · 21L _2, the side head and abuts the surface of the operator, it is desirable that the planar or substantially planar. As shown in FIG. 1, the edged surface along the outer shape of the second temple portions 21R ₂ and 21L ₂ is a flat surface. As a result, the second temple portions 21R ₂ and 21L ₂ have a larger contact area with the side of the operator's side than in the case of line contact, so that the operator can stably wear the HMD. It becomes. Also, a force sandwiching the head is dispersed by the second temple portion 21R ₂ · 21L ₂ of the right and left, the operator is allowed be used for a long time without discomfort HMD.

  Next, the configuration of the display unit 1 will be described. FIG. 3 is a cross-sectional view illustrating a schematic configuration of the display unit 1. The display unit 1 includes a light source 11, a unidirectional diffuser plate 12, a condenser lens 13, a display element 14, and an eyepiece optical system 16. The light source 11, the unidirectional diffuser plate 12, the condenser lens 13, and the display element 14 are accommodated as a unit 10 in the housing 15, and a part of the eyepiece optical system 16 (a part of an eyepiece prism 17 described later) is also included. It is located in the housing 15. The cable 3 described above is provided so as to penetrate the housing 15, and driving power and a video signal are supplied to the light source 11 and the display element 14.

  For convenience of explanation below, directions are defined as follows. First, an axis that optically connects the center of the display area of the display element 14 and the center of the optical pupil E formed by the eyepiece optical system 16 is an optical axis. The optical axis direction when the optical path from the light source 11 to the optical pupil E is developed is taken as the Z direction. In addition, a direction perpendicular to an incident surface of an optical axis to a hologram optical element 19 to be described later of the eyepiece optical system 16 is defined as an X direction, and a direction perpendicular to the ZX plane is defined as a Y direction. The incident surface of the optical axis to the hologram optical element 19 refers to a plane including the optical axis of incident light and the optical axis of reflected light in the hologram optical element 19, that is, the YZ plane. Hereinafter, the incident surface is simply referred to as an incident surface or an optical axis incident surface. It should be noted that positive and negative are irrelevant in each direction of XYZ.

  The light source 11 illuminates the display element 14. For example, the light source 11 is 462 ± 12 nm (B light), 525 ± 17 nm (G light), 635 ± 11 nm (R It is composed of RGB-integrated LEDs that emit light in three wavelength bands. As described above, the light source 11 emits light having a predetermined wavelength width, whereby the image light obtained by illuminating the display element 14 can have a predetermined wavelength width. When the image light is diffracted, the operator can observe the image over the entire observation angle of view at the position of the optical pupil E. Further, the peak wavelength for each color of the light source 11 is set in the vicinity of the peak wavelength of the diffraction efficiency described later of the hologram optical element 19, so that the light use efficiency is improved.

  In addition, since the light source 11 is composed of LEDs that emit RGB light, the light source 11 can be realized at low cost and a color image is displayed on the display element 14 when the display element 14 is illuminated. The color image can be provided to the worker. Further, since each LED has a narrow emission wavelength width, the use of a plurality of such LEDs enables high color reproducibility and bright image display.

  The unidirectional diffuser plate 12 diffuses the light emitted from the light source 11, but the degree of diffusion differs depending on the direction. More specifically, the unidirectional diffuser 12 diffuses incident light in the X direction by about 40 ° and diffuses incident light in the Y direction by about 0.5 °. The arrangement of the unidirectional diffuser 12 can be omitted.

  The condensing lens 13 is composed of a cylinder lens that condenses the light diffused by the unidirectional diffusing plate 12 in the Y direction, and is arranged so that the diffused light efficiently forms the optical pupil E. .

  The display element 14 displays an image by modulating the light emitted from the light source 11 in accordance with image data, and is configured by a transmissive liquid crystal display element having pixels in a matrix in which light is transmitted. Has been. The display element 14 is arranged such that the long side direction of the rectangular display region is the X direction and the short side direction is the Y direction. The display element 14 may be a reflective type. As the reflective display element 14, for example, a reflective liquid crystal display element or DMD (Digital Micromirror Device; manufactured by Texas Instruments Inc., USA) can be used.

  The eyepiece optical system 16 is an enlargement optical system that guides the light of the image displayed on the display element 14 to the optical pupil E, and provides the operator with an enlarged virtual image of the display image on the display element 14. A transparent substrate), a deflection prism 18 (second transparent substrate), and a hologram optical element 19.

  The eyepiece prism 17 totally reflects the image light from the display element 14 incident through the surface 17a by the two opposing surfaces 17b and 17c and guides it to the operator's pupil through the hologram optical element 19, while The light is transmitted to the operator's pupil and is made of, for example, an acrylic resin together with the deflecting prism 18. The eyepiece prism 17 is formed in a shape in which the lower end portion of the parallel plate is thinned toward the lower end so as to be wedge-shaped, and the upper end portion thereof is thickened toward the upper end. Further, the eyepiece prism 17 is bonded to the deflection prism 18 with an adhesive so as to sandwich the hologram optical element 19 disposed at the lower end thereof.

  The deflection prism 18 is configured by a substantially U-shaped parallel plate in plan view (see FIG. 1), and when the deflection prism 18 is bonded to the lower end portion and both side surface portions (left and right end surfaces) of the eyepiece prism 17, the eyepiece prism. 17 and a substantially parallel flat plate. By joining the deflecting prism 18 to the eyepiece prism 17, it is possible to prevent distortion of the external image observed by the operator through the eyepiece optical system 16.

  That is, for example, when the deflection prism 18 is not joined to the eyepiece prism 17, the external light is refracted when passing through the wedge-shaped lower end of the eyepiece prism 17, so that the external field image observed through the eyepiece prism 17 is distorted. Arise. However, the deflection prism 18 is joined to the eyepiece prism 17 to form an integral substantially parallel plate, so that the deflection when the external light passes through the wedge-shaped lower end of the eyepiece prism 17 is canceled by the deflection prism 18. Can do. As a result, it is possible to prevent distortion in the external image observed through the see-through.

  The two surfaces of the eyepiece prism 17 and the deflection prism 18 facing each other may be flat or curved. In the latter case, the eyepiece optical system 16 can have a function as a correcting eyeglass lens.

  The hologram optical element 19 diffracts and reflects the image light (wavelength light corresponding to the three primary colors) emitted from the display element 14 and guides it to the optical pupil E, and enlarges the image displayed on the display element 14. It is a volume phase type reflection hologram guided as a virtual image to the pupil. The hologram optical element 19 has, for example, three wavelength ranges of 465 ± 5 nm (B light), 521 ± 5 nm (G light), and 634 ± 5 nm (R light) with a peak wavelength of diffraction efficiency and a wavelength width of half the diffraction efficiency. The light is diffracted (reflected). Here, the peak wavelength of diffraction efficiency is the wavelength at which the diffraction efficiency reaches a peak, and the wavelength width at half maximum of the diffraction efficiency is the wavelength width at which the diffraction efficiency is at half the peak of the diffraction efficiency. It is.

  The reflection-type hologram optical element 19 has high wavelength selectivity, and only diffracts and reflects light having a wavelength in the wavelength range (near the exposure wavelength). The hologram optical element 19 is transmitted, and a high external light transmittance can be realized.

  Further, the hologram optical element 19 has an axially asymmetric positive optical power. That is, the hologram optical element 19 has the same function as an aspherical concave mirror having positive power. Thereby, the degree of freedom of arrangement of each optical member constituting the apparatus can be increased, the apparatus can be easily reduced in size, and an image with good aberration correction can be provided to the operator.

  Next, the operation of the display unit 1 having the above configuration will be described. The light emitted from the light source 11 is diffused by the unidirectional diffusion plate 12, condensed by the condenser lens 13, and enters the display element 14. The light incident on the display element 14 is modulated for each pixel based on the image data and is emitted as video light. That is, a color image is displayed on the display element 14.

  The image light from the display element 14 enters the eyepiece prism 17 of the eyepiece optical system 16 from its upper end surface (surface 17a), and is totally reflected a plurality of times by the two opposing surfaces 17b and 17c, thereby being a hologram optical element. 19 enters. The light incident on the hologram optical element 19 is reflected there, passes through the surface 17b, and reaches the optical pupil E. At the position of the optical pupil E, the operator can observe an enlarged virtual image of the image displayed on the display element 14.

  On the other hand, the eyepiece prism 17, the deflecting prism 18, and the hologram optical element 19 transmit almost all of the outside light, so that the operator can observe the outside world image through them. Therefore, the virtual image of the image displayed on the display element 14 is observed while being overlapped with a part of the external image.

  As described above, in the display unit 1, the image light emitted from the display element 14 is guided by total reflection in the eyepiece prism 17 and guided to the operator's pupil through the hologram optical element 19. Similar to the lens, the thickness of the eyepiece prism 17 and the deflecting prism 18 can be about 3 mm, and the display unit 1 can be reduced in size and weight. Further, by using the eyepiece prism 17 that totally reflects the image light from the display element 14, a high external light transmittance can be secured and a bright external image can be provided to the operator.

  Further, the volume phase type reflection type hologram optical element 19 has a narrow diffraction efficiency half-value wavelength width and a high diffraction efficiency. By using such a hologram optical element 19, a color image with high color purity and high brightness can be obtained. In addition to providing high external light transmittance, the operator can observe a bright external image.

  Further, as can be seen from the above description, the hologram optical element 19 functions as a combiner that simultaneously guides the image light and the external light from the display element 14 to the operator's pupil. As a result, the operator can simultaneously observe the image provided from the display element 14 and the external image via the hologram optical element 19.

  Next, the visible light imaging unit 4 and the infrared imaging unit 5 will be described. The visible light imaging unit 4 and the infrared imaging unit 5 are provided on the top of the housing 15 and are connected to the cable 3 through the inside of the housing 15. The visible light imaging unit 4 and the infrared imaging unit 5 have substantially the same imaging range as the worker's field of view. The visible light imaging unit 4 selectively captures visible light to obtain a visible light image, and the infrared imaging unit 5 selectively captures infrared light to obtain an infrared image.

  These visible light image and infrared image are sent to the control unit through the cable 3 and processed. The visible light image can be displayed on the display unit 1 as it is. For example, a CCD (Charge Coupled Device) can be used as an imaging device provided in the visible light imaging unit 4. Moreover, as an image pick-up element with which the infrared imaging part 5 is equipped, thermal type | mold infrared detection elements, such as a bolometer arranged in two dimensions, can be used, for example.

(First embodiment)
As a first embodiment, a work support apparatus using the above HMD will be described. FIG. 4 is a block diagram illustrating a schematic configuration of the work support device 7. The control unit 6 includes a temperature distribution image generation unit 61, a component detection unit 62, an appropriate temperature database 63, a temperature abnormality detection unit 64, and a notification information generation unit 65.

  The temperature distribution image generation unit 61 converts the infrared image acquired by the infrared imaging unit 5 into a visible light image and generates a temperature distribution image. In this image, for example, the low temperature can be displayed in blue, the medium temperature in yellow, and the high temperature in red. This temperature distribution image is used for temperature abnormality detection, which will be described later, or displayed on the display unit 1 as technical information for the operator.

  The component detection unit 62 detects each component in the image by performing image processing on the visible light image acquired by the visible light imaging unit 4. Information such as the shape, name, and number of each component necessary for component detection is stored in the component detection unit 62 as necessary.

  As a premise for detecting parts, it is desirable to input in advance which model is currently being repaired. As a result, it is possible to detect parts by focusing on the model. As a detection method in the component detection unit 62, for example, when characteristic coloring is performed for each component, a characteristic coloring component is selected from the visible light image and stored. And the part can be identified from the relationship between colors. If a certain part in the visible light image can be specified, other parts can be specified by normalization.

  As another detection method, when a character string such as a product number is described in the part, an identifiable character string is extracted from the visible light image, and the relationship between the stored part and the character string is used. The part can be specified. If a certain part in the visible light image can be specified, other parts can be specified by normalization.

  As another detection method, there is a method using a marker. It is assumed that the operator looks at the machine to be repaired from a certain position. Therefore, markers are attached to the four corners and the center (depth information) of the visible light image at the beginning of the work, and thereafter, three-dimensional information viewed by the operator is acquired by the movement of the five markers. And a component can be specified by collating a visible light image with the memorized three-dimensional information for every component.

  As another detection method, graphic detection processing such as pattern matching and Hough conversion can also be used. For example, in order to perform pattern matching, first, a correction process for geometrically converting a visible light image and a feature extraction process for extracting a geometric feature are performed to generate feature data. Then, a part can be identified by selecting a pattern data obtained by shaping each stored part from each direction, and selecting a pattern data that most closely matches the feature data just extracted.

  The appropriate temperature database 63 stores the appropriate temperature of each component. The temperature abnormality detection unit 64 detects a component having an abnormal temperature based on the temperature distribution image, the component detected from the visible light image, and the appropriate temperature of each component. Specifically, the appropriate temperature is read from the appropriate temperature database 63 for the component in the visible light image, the temperature corresponding to the position of each component in the temperature distribution image is specified, and the appropriate temperature is compared with the measured actual temperature. Then, a part whose actual temperature is not within the appropriate temperature range is extracted as a temperature abnormal part.

  The notification information generation unit 65 generates display information to be displayed on the display unit 1 from the detection result of the temperature abnormal component, and outputs the display information to the display unit 1. This display information includes at least one of highlighting of temperature abnormal parts, display of characters indicating abnormality, and display of symbols indicating abnormality. For example, the images shown in FIGS. 5 to 7 can be used as display information. 5 to 7 are examples of display images in the case where the heating roller in the fixing unit of the copying machine has a temperature equal to or higher than the appropriate temperature. The heating roller displayed on the display unit 1 appears to overlap the heating roller.

  In the image 81 of FIG. 5, the heating roller 82 that is a temperature abnormal part is displayed in red or the like or blinked in red or the like to emphasize that the temperature of the heating roller 82 is abnormal, To inform you of the important parts. The color may be changed to yellow, orange, red, etc. according to the difference from the appropriate temperature.

  In the image 83 of FIG. 6, the symbol “!” 84 is displayed on the image of the heating roller 82 to indicate that the temperature of the heating roller 82 is abnormal and to notify the operator of abnormal parts. In the image 85 of FIG. 7, by using an arrow toward the heating roller 82 to display a character 86 “abnormal!”, It is indicated that the temperature of the heating roller 82 is abnormal and abnormal parts are shown to the operator. To let you know. Furthermore, in addition to or instead of the display in FIGS. 5 to 7, a voice message or a beep sound that calls attention may be output.

  In this way, by notifying an abnormal temperature component using the HMD, the operator can easily know an abnormal component that requires repair that is difficult to understand with the naked eye while working freely with both hands.

  In addition, not only the parts whose temperature is not in the proper range as described above, but from the viewpoint of work safety, high-temperature parts (for example, there is a risk of burns to the worker even at the proper temperature) The above-described method may also be used for notification when a component of a developing unit or a fixing unit of a copying machine is detected. For example, if a part having a temperature distribution image of 60 ° C. or higher is notified, the risk of burns is reduced.

(Second Embodiment)
As a second embodiment, a work support system including a work support device using the above HMD and a server device that can communicate with the work support device will be described. FIG. 8 is a block diagram showing a schematic configuration of the work support system 9. The work support system 9 includes a work support device 7 ′, a network 100, and a server device 110. Hereinafter, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  The work support device 7 ′ includes a display unit 1, a visible light imaging unit 4, an infrared imaging unit 5, and a control unit 6 ′. The control unit 6 ′ includes a temperature distribution image generation unit 61, a transmission unit 66, and a reception unit 67.

  The transmission unit 66 transmits the visible light image acquired by the visible light imaging unit 4 and the temperature distribution image generated by the temperature distribution image generation unit 61 to the server device 110 via the network 100. On the other hand, the receiving unit 67 receives display information to be displayed on the display unit 1 from the server device 110 via the network 100 and outputs the display information to the display unit 1.

  The server device 110 includes a reception unit 111, a transmission unit 112, a component detection unit 62, an appropriate temperature database 63, and a notification information generation unit 65.

  The receiving unit 111 receives the visible light image and the temperature distribution image from the work support apparatus 7 ′ via the network 100, and transmits the visible light image to the component detection unit 62 and the notification information generation unit 65, and converts the temperature distribution image into a temperature abnormality. The information is output to the detection unit 64 and the notification information generation unit 65, respectively. On the other hand, the transmission unit 112 transmits the display information generated by the notification information generation unit 65 to the work support device 7 ′ via the network 100. The display information generation method is the same as in the first embodiment.

  As described above, in the second embodiment, the component detection unit 62, the appropriate temperature database 63, the temperature abnormality detection unit 64, and the notification information generation unit 65 provided in the work support device 7 in the first embodiment are replaced with the server device. By providing in 110, by simplifying the configuration of the work support device 7, the work support device 7 can be reduced in size and cost.

  In the second embodiment, the temperature distribution image generation unit 61, the component detection unit 62, the appropriate temperature database 63, the temperature abnormality detection unit 64, and the notification information generation unit 65 are the work support device 7 ′. Alternatively, any server device 110 may be provided. For example, when the work support device 7 ′ has a minimum configuration, the temperature distribution image generation unit 61 may be moved to the server device 110 side in FIG. 8.

  It should be noted that the work support apparatus of the present invention has no problem even if it has a function of receiving an image or sound as a work instruction from the outside by communication as conventionally proposed.

  INDUSTRIAL APPLICABILITY The work support device and work support system of the present invention can be used to support workers who perform maintenance work on devices and equipment.

DESCRIPTION OF SYMBOLS 1 Display part 4 Visible light imaging part 5 Infrared imaging part 7, 7 'Work support apparatus 9 Work support system 11 Server apparatus 61 Temperature distribution image generation part 62 Component detection part 63 Appropriate temperature database 64 Temperature abnormality detection part 65 Notification information generation part 84 symbols 86 characters

Claims (4)

A work support device using a head mounted display,
A visible light imaging unit that captures visible light; and
An infrared imaging unit for imaging infrared;
An informing unit for informing the abnormality,
The notification unit is based on the temperature distribution image generated from the infrared image acquired by the infrared imaging unit, the components detected from the visible light image acquired by the visible light imaging unit, and the appropriate temperature of each component, A work support apparatus that reports information generated from a result of detecting a component having an abnormal temperature. A work support system comprising the work support device according to claim 1 and a server device capable of communicating with the work support device.
A temperature distribution image generation unit that generates a temperature distribution image from an infrared image acquired by the infrared imaging unit, a component detection unit that detects a component from the visible light image acquired by the visible light imaging unit, and an appropriate temperature of each component An appropriate temperature database to be stored, a temperature abnormality detection unit that detects a component having an abnormal temperature based on the temperature distribution image, the component detected from the visible light image, and an appropriate temperature of each component, and the temperature is abnormal A work support system comprising a notification information generation unit that generates information to be notified from a result of detecting a component that is a part of the work support apparatus or the server apparatus. A work support device using a head mounted display,
A visible light imaging unit that captures visible light; and
An infrared imaging unit for imaging infrared;
An informing unit for informing the abnormality;
A temperature distribution image generation unit that generates a temperature distribution image from an infrared image acquired by the infrared imaging unit;
A component detection unit for detecting a component from a visible light image acquired by the visible light imaging unit;
An appropriate temperature database for storing the appropriate temperature of each component;
A temperature abnormality detection unit for detecting a component having an abnormal temperature based on the temperature distribution image and the component detected from the visible light image and an appropriate temperature of each component;
A work support apparatus comprising: a notification information generation unit that generates information to be notified from a result of detecting a component having an abnormal temperature.   The notifying unit notifies using at least one of highlighting of the abnormal part, displaying a character indicating abnormality, displaying a symbol indicating abnormality, a voice message, and a beep sound. The work support apparatus according to 1 or 3.

Images