JP3901702B2 - Imaging / display device - Google Patents

Description

  The present invention relates to an imaging / display apparatus, and more particularly to an optical apparatus that can capture a subject with a camera and project the captured image after image processing.

  It is a well-known technique to photograph a subject with a camera, image-process the image, and use the image for product defect or determination at a production site. There are already devices that project and project an image using a projector such as a liquid crystal display (LCD) or DMD (Digital Micromirror Device).

  However, even if the subject is imaged, the captured image is subjected to image processing for a certain purpose, the image is displayed on a liquid crystal display device or DMD, and further projected using a projection light source, It is impossible to recognize the actual state of

  For example, when shooting an internal defect such as a wall with an infrared camera, you can image the captured image by capturing an image of peeling or water leakage in the wall with an infrared camera, and display it on the monitor. The position of the defect inside the wall displayed on the monitor could only be recognized by measuring the wall with a measure and adding a mark to the defect.

  Also, in the case of mounting electronic components, it is possible to take an image with a camera and display on the monitor that the electronic components are not correctly mounted on the printed circuit board. Where is the defect on the actual printed circuit board? Could not display. When similar electronic parts are lined up, identifying a place where the electronic parts are located is a troublesome operation. Therefore, the current situation is that the defect must be searched for by alternately comparing the monitor screen and the printed circuit board.

  An object of the present invention is to provide an optical device capable of accurately and easily confirming a defect, a measurement result, and the like in real time based on a result of image processing of a captured image.

That is, the present invention relates to an imaging unit that captures an image of a subject through a half mirror or a dichroic mirror, an image processing unit that performs image processing on the captured image, and a liquid crystal display (LCD) or DMD that displays the processing result. a display means composed of, the display contents, on the imaging optical systems coaxial, possess a projection means for displaying superimposed on the object by the projection light source, the liquid crystal display illuminated by the projection light source The present invention relates to an imaging / display device in which light from the device or DMD is projected so as to overlap the subject via the half mirror or dichroic mirror . In particular, in order to be able to know the position of the part where the defect etc. of the subject exists on the actual subject, the captured image is fed back, and the result is displayed on the subject, The result of feedback is confirmed on the subject.

According to the present invention, a subject is imaged through a half mirror or a dichroic mirror, the captured image of the subject is subjected to image processing, and the result is displayed on a display means including an LCD or DMD , and the display content is displayed as a projection light source. Is displayed so as to be superimposed on the subject via the half mirror or dichroic mirror, so that the position of the portion where the subject has a defect, etc. can be captured on the actual subject. It is possible to apply feedback to the image, display the result superimposed on the subject, and check the feedback result on the subject. As a result, the defect of the subject and the measurement result can be confirmed accurately and easily in real time on the actual subject.

In the present invention, a camera that captures an image of the object, said image processing means for performing image processing on the captured image, said display means comprising an LCD or DMD display the processing result, the display contents It is desirable that the projection means for displaying the image superimposed on the subject by a projection light source is provided coaxially with the imaged optical system.

  In this case, it is preferable to have a lens and a camera for imaging the subject through a half mirror or a dichroic mirror.

  Further, it is preferable to include a liquid crystal display device or DMD on which an image obtained as a result of image processing is displayed, and a scanning light source that scans the liquid crystal display device or DMD.

  Then, after the light from the liquid crystal display device or DMD irradiated by the scanning light source passes through the lens, it is preferably projected so as to be superimposed on the subject via the half mirror or dichroic mirror.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  In the example shown in FIG. 1, an image of a subject 1 is formed on an imaging surface 6 of a CCD (Charge Coupled Device) camera 5 by a lens 3 through a half mirror (or dichroic mirror) 2.

  The captured image is sent to the image processor 7, subjected to image processing (for example, image processing such as center of gravity detection and pattern matching) according to the purpose, and displayed on the monitor screen 8.

  On the other hand, the processed image is sent to the liquid crystal controller 10 and displayed on the liquid crystal display device 9. Then, the image of the liquid crystal display device 9 is guided to the half mirror (or dichroic mirror) 2 through the lens 4 by the projection light source 11, reflected there, and projected coaxially with the subject 1, the lens 3 and the camera 5. As a result, the image on the liquid crystal display device 9 is superimposed on the subject 1 and projected.

  In this case, for the half mirror (or dichroic mirror) 2, when the half mirror is used, a part of the light from the projection light source 11 is reflected toward the subject 1 (and further reflected from the subject 1 toward the camera 5. At this time, a filter 41 for cutting the wavelength of the light of the projection light source 11 is inserted on the optical path like an imaginary line at the front of the imaging surface 6 (or with respect to the optical path). If it can be taken out and put in, the image of the subject 1 itself and the image that is the result of image processing of the image of the subject 1 can be simultaneously viewed. When a dichroic mirror is used, it is a mirror that transmits light of a certain wavelength and reflects light of a certain wavelength, but reflects only the light of the wavelength of the projection light source 11 and transmits light of other wavelengths. For example, the camera 5 can capture an image of only the subject 1 and can project only an image that has undergone image processing.

  In the example illustrated in FIG. 2, the positions of the camera 5, the liquid crystal display device 9, and the projection light source 11 are switched with respect to the half mirror (or dichroic mirror) 2 as compared with the example of FIG. 1.

  Light from the subject 1 is reflected by the half mirror (or dichroic mirror) 2 and guided to the camera 5, while light emitted from the projection light source 11 passes through the half mirror (or dichroic mirror) 2 and is on the subject 1. To be projected. Here, when a dichroic mirror is used, the dichroic mirror may be designed so as to transmit the wavelength light of the projection light source 11 and reflect other wavelength light.

  In the example shown in FIG. 3, a DMD (Digital Micromirror Device) 12 is used as a display instead of the liquid crystal display device 9.

  In the DMD, small mirrors are arranged in a matrix so that the reflection angle of each mirror can be controlled by the DMD controller 13 and is commercially available as a projection display device in the same manner as a liquid crystal display device. In this example, with respect to the half mirror (or dichroic mirror) 2, the DMD 12 is disposed on the reflection side, and the camera 5 is disposed on the transmission side.

  In the example illustrated in FIG. 4, the camera 5 is disposed on the reflection side and the DMD 12 is disposed on the transmission side with respect to the half mirror (or dichroic mirror) 2 as compared with the example in FIG. 3.

  FIG. 5 shows an example of the projection light source 11, and the laser light emitted from the laser light source 14 is processed into a sheet-like laser light 24 as shown in FIG. 6 by the sheet-like light production lens 15. Then, the light is reflected by the rotating polygon mirror 16, and the surface of the liquid crystal display device 9 is scanned by the scanning lens 17.

  The sheet-like laser light 24 is condensed in a linear form by a scanning lens 17 such as a cylindrical lens 23 shown in FIG. 6 and scans the surface of the liquid crystal display device 9. That is, if each column is scanned from, for example, the left column to the right column on the surface of the liquid crystal display device, one screen can be scanned uniformly, and the light transmitted through the liquid crystal display device 9 is transmitted to the lens shown in FIG. 4 will be projected. Note that scanning may be performed using a galvanometer mirror instead of the polygon mirror 16.

  FIG. 7 shows an example in which a two-dimensional scanning galvanometer mirror is used as the projection light source 11. Laser light emitted from the laser light source 14 is transmitted by a horizontal scanning galvanometer mirror 18 and a vertical scanning galvanometer mirror 19. A two-dimensional surface can be scanned. The scanning light is condensed on the surface of the liquid crystal display device 9 by a scanning lens 20 (mainly an fθ lens is used), and each bit of each matrix of the liquid crystal display device is scanned. Of course, the projection light source 11 may scan not only the liquid crystal display device 9 but also the surface of the DMD 12 described above.

  FIG. 8 shows an example in which the optical system is simplified by combining the imaging lens and the projection lens into one and sharing them.

  In this case, the half mirror (or dichroic mirror) 2 is disposed between the lens 28 and the camera 5. Here, the liquid crystal display device 9 is used, but the DMD 12 may be used, and the liquid crystal display device 9 is disposed on the reflection side and the camera 5 is disposed on the transmission side, but the reverse is also possible.

In FIG. 8, as indicated by a virtual line, it is conceivable that the lens 28 is used for imaging, and a projection lens 28A is arranged separately from this (this is the same in the other examples described above). Good). In this case, the optical axis at the time of imaging is different from the optical axis at the time of projection, and the processed image is projected in the vicinity of the subject 1.

  Next, a specific example in which a subject is detected by the optical system described above will be described.

  FIG. 9 is an example of displaying the internal defect of the wall surface 21.

  That is, for example, as shown in FIG. 1, when the wall surface 21 of the house is imaged by the infrared camera 5 (FIG. 9A), in the location where there is an internal cavity or water leak, a defect is detected in the infrared image due to the temperature difference. Part can be detected. An apparatus for displaying an image obtained as a result of image processing of the defective portion on the monitor screen 8 is commercially available. The image of the defect is displayed on the display of the liquid crystal display device 9 and this display information is projected on the wall surface 21. The obtained image (FIG. 9B) corresponds to the defect image 22 in the wall surface 21.

  In this way, the result of direct measurement can be immediately displayed in real size and in real time, so that the target location can be easily and reliably confirmed, which is effective and easy to understand.

  FIG. 10 shows an example in which a mounting inspection of an electronic component mounted on the printed circuit board 25 is performed.

  For example, as shown in FIGS. 10A and 10B, the mounting positions of the electronic component 26 and the electronic component 27 may be different. In recent years, it has become increasingly difficult to determine the orientation, type, presence / absence, position, and the like of high-density and miniaturized electronic components. For example, the mounting state shown in FIG. 10B is a normal state. However, when the state of FIG. 10A is actually picked up, this picked-up image is compared with the regular information storing the mounting state of FIG. 10A, and the defective portion is shown in FIG. As shown by the broken line, it can be displayed on the actual printed circuit board 25 immediately.

  In this way, the mounting state and position can be easily and reliably determined, and based on this, it can be reliably restored or changed to the regular mounting state. Also. Such a discriminating system can also be applied so as to instruct an assembly procedure for high-mix, low-volume production. The stored information can be provided in the image processor 7 shown in FIG. 1, for example.

  FIG. 11 shows an example of an application to surgery for affected areas 29 and 30 such as cancer on a living organ 31.

  If the position and size of the affected areas 29 and 30 of the organ 31 are imaged in advance by CT (Computed Tomography) or the like, and the operation is performed while projecting the portion to be excised during the operation using the system according to the present invention, It is possible to prevent mistakes in which other parts are accidentally excised, and to indicate a surgical procedure. Alternatively, it is possible to prevent misplacement of scissors or the like in the surgical site by detecting the surgical site after surgery.

  FIG. 12 shows an example applied to level-up of athletes and the like.

  For example, when a running player 34 is imaged using the system according to the present invention, the gravity center position 32 of the player is calculated by image processing, and projected onto the person's body in real time, the vertical movement of the gravity center is detected. It is possible to correct this. As a result, correction can be made so that the vehicle can run in a better state. For example, it can be used for checking and correcting the exercise state, and also for diagnosis during exercise.

  FIG. 13 shows an example of displaying the center of gravity of the car.

  That is, in order to improve the running performance of the car 35, the center of gravity position 33 and the like are projected on the running car in real time using the system according to the present invention, and the running performance is similar to the above-described example. It can be used for improvement.

  As described above, according to the present invention, the result of image processing of a captured image is immediately displayed on a display such as a liquid crystal display device, projected on the same axis as the captured optical system, and displayed superimposed on the subject. Since it is possible to check the defects and measurement results on the subject easily and reliably on the actual subject, detect defects such as house walls and printed circuit board mounting status, repair them, and instruct the processing procedure Etc. can be performed.

It is the schematic which shows the example of the imaging and display apparatus by embodiment of this invention. It is the schematic which shows the other example of an imaging and a display apparatus similarly. It is the schematic which shows the other example of an imaging and a display apparatus similarly. It is the schematic which shows other example of an imaging and a display apparatus similarly. It is the schematic which shows the example of the light source for projection in an imaging / display apparatus similarly. It is the schematic which shows the example of the condensing system of an imaging and a display apparatus similarly. It is the schematic which shows the other example of the projection light source of an imaging and display apparatus similarly. It is the schematic which shows the other example of an imaging and a display apparatus similarly. It is a schematic front view which shows the application example which detects and displays internal defects, such as a wall surface, using an imaging / display apparatus. It is a schematic plan view which shows the application example which performs the electronic component mounting test | inspection on a printed circuit board using an imaging / display apparatus similarly. It is the schematic which shows the application example which performs organ surgery etc. using an imaging / display apparatus similarly. It is a schematic side view which shows the example applied to checking the exercise | movement of an athlete using the imaging / display apparatus. It is a schematic side view which shows the example applied to checking the driving | running | working state of a vehicle using an imaging and display apparatus.

Explanation of symbols

1 ... subject, 2 ... half mirror or dichroic mirror,
3, 4, 28, 28A ... lens, 5 ... camera, 6 ... imaging surface, 7 ... image processor,
8 ... monitor screen, 9 ... liquid crystal display device, 10 ... liquid crystal controller, 11 ... light source for projection,
12 ... DMD (Digital Micromirror Device), 13 ... DMD controller,
14 ... laser light source, 15 ... sheet-shaped light producing lens, 16 ... polygon mirror,
17, 20 ... lens for scanning, 18 ... galvanometer mirror for horizontal scanning,
19 ... Galvano mirror for vertical scanning, 21 ... Wall surface, 22 ... Defect image in the wall surface,
23 ... Cylindrical lens, 24 ... Sheet-shaped laser beam, 25 ... Printed circuit board,
26, 27 ... electronic components, 29, 30 ... affected area, 31 ... organ, 32, 33 ... center of gravity,
34 ... Athlete, 35 ... Car, 41 ... Filter

Claims (2)

Image display means for picking up an image of a subject through a half mirror or dichroic mirror, image processing means for image processing of the picked- up image, and display means comprising a liquid crystal display device or DMD (Digital Micromirror Device) for displaying the processing result When, the display contents, on the imaging optical systems coaxial, possess a projection means for displaying superimposed on the object by the projection light source, a liquid crystal display device or DMD illuminated by the projection light source The image pickup / display device projects the light so that it overlaps the subject via the half mirror or dichroic mirror . The imaging / display device according to claim 1 , comprising: a liquid crystal display device or DMD that displays an image obtained as a result of image processing; and a scanning light source that scans the liquid crystal display device or DMD.

Images