KR20160124334A - Image diagnostic apparatus using multi-camera - Google Patents

Abstract

The present invention relates to an image diagnosing device using multiple cameras. By using the image diagnosing device using multiple cameras according to the present invention, it is possible that reasons, positions, and degrees of excessive heat and noise, which occur during operation of power stations or operation systems and other industrial facilities, etc., can be measured at the same time by implementing sensors for detecting an image, heat, and a sound in one entity. Also, an operation fault or a malfunction of a target can be diagnosed and analyzed quickly and accurately. The image diagnosing device using multiple cameras according to the present invention comprises: a sensing unit which includes an image camera module, a thermal imaging camera module, and a sound camera module, and photographs an operation state of a facility to generate respective input images; a control unit which generates a combined image by using the input images photographed in the sensing unit; a storage unit which stores the input images photographed in the sensing unit and the combined image generated in the control unit; and an output unit which displays at least one among the input images and the combined image on a screen, wherein the control unit is characterized in that it receives the input images from at least two of the image camera module, the thermal imaging camera module, and the sound camera module and combines the input images to generate the combined image.

Description

[0001] The present invention relates to an image diagnostic apparatus using multi-camera,

The present invention relates to an image diagnostic apparatus using multiple cameras, and more particularly, to an image diagnostic apparatus using multiple cameras, in which each sensor for sensing image, heat and sound is integrally integrated to generate overheating and noise generated in operation of a power plant, And more particularly, to an image diagnostic apparatus using a multi-camera capable of performing simultaneous measurement of a cause, an occurrence position, and the intensity thereof, thereby quickly and accurately diagnosing and analyzing malfunctions or failures of an object.

Generally, during maintenance of power generation facilities, contact type equipment is used to check major facilities, but there is a difficulty in securing reliability and safety in diagnosing and checking the state of power generation facilities in operation. On the other hand, existing fault diagnosis methods for ventilation systems (fans, etc.) are based on the design of bearing temperature, determine whether there is a facility fault or not, check the vibration, operation and hydraulic conditions of the lubricating oil pump, The operator of the power plant hears the direct noise to determine whether it is faulty. However, due to the noise and the ability to measure by individual, accurate predictive diagnosis is difficult, and the noise due to the operating conditions of the equipment, the flow conditions of the internal media, It is not easy to determine whether or not the fault is correct.

Korean Patent Registration No. 10-0018858 entitled " Method and Apparatus for Diagnosing Reactor Facilities Faults Using a CCD Camera and an Infrared Thermal Imaging Camera ", which is a prior art, discloses a method of processing a DB image and storing the exploded image in a state of good visual observation, After comparing the stored DB image with the actual image to be observed, a CCD camera capable of matching the DB image and the actual image selectively with the acquired image of the infrared thermal imaging camera to identify the abnormal occurrence site, and a nuclear reactor using the infrared thermal imaging camera An abnormal diagnosis method and apparatus thereof are described.

Currently, thermal cameras are used to diagnose malfunctions and defects in vehicles and electronic equipment. It is possible to know only the part where heat is generated in the object, and it is difficult to grasp a defect such as a crack which does not accompany heat generation or noise caused by the heat. There is a problem.

On the other hand, portable real-time audio camera technology based on FPGA (Field Programmable Gate Array, IC) technology has been commercialized and used to find and analyze specific noise sources. However, since it only displays the area where noise is generated, It is impossible to grasp that heat is generated due to defects, so that it is limitedly applied to a portion where noise is generated in a driving system or the like. That is, when a driving system such as a compressor, a turbine, or a gear box is accompanied by heat and noise, it is difficult to easily grasp whether heat and noise occur at the same position or independently at different positions. There is no equipment for visualizing and matching the noise source and the heat source with one image data.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-mentioned problems of the prior art, and it is an object of the present invention to simultaneously measure a cause of occurrence of overheat and noise, an occurrence position and intensity thereof generated in operation of a power plant, , And it is an object of the present invention to quickly and accurately perform diagnosis and analysis of malfunction or failure of an object.

On the other hand, there is another purpose of enabling low cost, small size, and light weight as compared with the case where each sensor for sensing video, heat and sound is integrally integrated.

The above object of the present invention is also achieved by an image diagnostic apparatus using multiple cameras, comprising: a sensing unit including a video camera module, a thermal camera module, and a sound camera module, A storage unit configured to store the input image captured by the sensing unit and the composite image generated by the control unit, and a storage unit configured to store the composite image using the input image or the composite image Wherein the control unit receives the input image from at least two of the image camera module, the thermal image camera module, and the sound camera module, and synthesizes the composite image, To an imaging device using a camera It is achieved.

Therefore, the image diagnostic apparatus using the multiple cameras of the present invention has the effect of enabling low cost, small size, and light weight as compared with the case where the respective sensors for sensing the image, the heat and the sound are integrally integrated .

In addition, by mapping sound and thermal image information based on data acquired from a video camera, a thermal imaging camera, and a microphone, it is possible to output in a single image form, It is possible to visualize the image, thermal image, and sound information generated in the operation of the facility to grasp the current state and to easily and accurately grasp the malfunction or the occurrence and the cause of the malfunction.

1 is a configuration diagram of an image diagnostic apparatus using multiple cameras according to the present invention,
2 is an exemplary view of a sensing unit of an image diagnostic apparatus using multiple cameras according to the present invention,
3 is an outline view of an image diagnostic apparatus using multiple cameras according to the present invention,
4 is a block diagram of an image diagnostic apparatus using multiple cameras according to the present invention.
5A to 5D are views illustrating an example of image synthesis of an image diagnostic apparatus using multiple cameras according to the present invention.

The terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary terms and the inventor may appropriately define the concept of the term in order to best describe its invention It should be construed as meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in this specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and do not represent all the technical ideas of the present invention. Therefore, It is to be understood that equivalents and modifications are possible.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a configuration diagram of an image diagnostic apparatus using multiple cameras according to the present invention, and FIG. 2 is an exemplary view of a sensing unit of an image diagnostic apparatus using multiple cameras according to the present invention. 1 and 2, the present invention includes a sensing unit 100, a control unit 200, an output unit 300, a storage unit 310, a communication unit 320, a power unit 330, and a user input unit 340).

The sensing unit 100 includes a video camera module 110, a thermal camera module 120, and a sound camera module 130. The sensing unit 100 can measure the operation states of objects such as a power generation facility or a driving system, To simultaneously generate an input image, which includes a video image, an infrared image and a sound image.

2, the image camera module 110 is mounted on the upper side and the thermal camera module (not shown) is mounted on the lower side of the camera module 110 120 may be arranged. As the arrangement interval between the two sensors becomes narrower, an error that may occur in the synthesis of the image and the thermal image may be reduced.

The control unit 200 receives the input image captured by the sensing unit 100 and synthesizes a final image using an image visualization algorithm. The first control module 210, the second control module 220 And a third control module 230, as shown in FIG.

In detail, the first control module 210 controls the image and the thermal image to fuse with each other to generate a first image, and the second control module 220 processes the sound data, A second control module 220, and a third control module 220. The third control module 220 generates a composite image by combining the first image and the second image received from the first control module 210 and the second control module 220, (230).

The first image generated by the first control module 210 may be generated by fusing an outline of an object on the image image to the thermal image (Edge mode) (Threshold mode).

That is, the composite image generated by the third control module 230 may be obtained by synthesizing the image image and the thermal image, compositing the image image with the sound image, synthesizing the thermal image and the sound image Or a combination of the image image, the thermal image, and the sound image. In particular, when the composite image is synthesized with the image image, the synthesized image is generated by combining at least one of the thermal image or the sound image with the image It is possible to display only the outline portion of the object on the image.

 The output unit 300 displays at least one of the input image or the composite image as a display screen or a voice through a speaker.

The output unit 300 transmits the result of comparing the composite image with the reference image image, the reference thermal image, and the reference sound image stored in the storage unit 310 or the external database server in the control unit 200 A comparison of visualized images or a comparison with a preset threshold of heat and noise is performed.

The output unit 300 includes an alarm module (not shown) for alarming malfunctions. The malfunction that the alarm module warns may include any one of cracks, erroneous installation, non-installation, abnormal overheating, Or more.

The storage unit 310 stores not only the reference image, the reference thermal image, and the reference sound image indicating the steady state of the object, but also the input image captured by the sensing unit 100 and the input image captured by the controller 200 Include a composite image and save it.

In addition, the present invention includes a communication unit 320 for communicating with the image diagnostic apparatus and the external database server, a power supply unit 330 for stable power supply, and a user input unit 340 for controlling operations and functions of the image diagnostic apparatus .

FIG. 3 is a schematic diagram of an image diagnostic apparatus using multiple cameras according to the present invention, and FIG. 4 is a block diagram of an image diagnostic apparatus using multiple cameras according to the present invention.

3 to 4, the image data sensed by the image camera module 110 is converted into Y (luminance signal) and C (chrominance signal) from YCbCr 8 bits through the first control module 210 Buffering for outputting in 8 bits and tilting for adjusting the position of the thermal image data and the image data and then fusion with the thermal image. Meanwhile, the fused image is finally sent to the third control module 230 and synthesized with the following sound image.

The infrared camera module 120 is a camera that measures energy in the form of infrared rays emitted from the surface of an object and displays the infrared energy in different colors according to the amount of energy. The infrared camera is a micro- a bolometer, and a photodetector), the electrical resistance of the sensor changes due to the temperature rise due to the absorption of light.

The thermal image data sensed by the thermal imaging camera module 120 is converted into an image corresponding to a temperature value through sampling and data conversion to generate a thermal image. More specifically, the thermal image data is output from the infrared camera, A non-uniformity correction (NUC) for correcting a non-uniformity value of each pixel of the deterioration image, a DPC (non-uniformity correction) for correcting using a normal pixel around the dead pixel, dead pixel correction, NR (noise reduction), BIT conversion to convert 14-bit data into 8-bit data, Upscale to enlarge by 2 times with digital zoom, and Color Palette to color the temperature image in black- Similarly, Y (luminance signal) and C (chrominance signal) are respectively converted to 8 bits and fused with the converted image. Meanwhile, the fused image is finally sent to the third control module 230 and synthesized with the following sound image.

The sound image is generated by processing through a FPGA (Field Programmable Gate Array (IC)) together with a Micro-phone and a DAQ (Data Acquisition) device based on a MEMS (Micro Electro Mechanical System) This method provides a visual representation of noise while increasing accuracy and processing speed.

The sound data sensed by the sound camera module 130 is transmitted to the second control module 220 through a DAQ for converting an analog signal into a digital signal and conditioning the signal, And then transmitted to the third control module 230.

Meanwhile, the third control module 230 controls the first control module 210 and the second control module 220 according to an algorithm for combining the image image received from the first control module 210 and the thermal image and the sound image, And outputs the composite image to the output unit 300. Meanwhile, the third control module 230 may be an AP (Application Processor) serving as a central processing unit of a general computer with a non-memory semiconductor.

Meanwhile, the database server receives data in various directions through various camera modules including the video camera module 110, the thermal camera module 120, and the sound camera module 130 under various conditions such as temperature, And also supports the management and utilization of multiple visualization big data through the collection of the composite images. At this time, a multichannel analysis algorithm having a fast convergence characteristic and a high safety is developed to perform Fast Fourier Transform (FFT) analysis and trend analysis of the multiple visualization data for each power generation facility, Diagnosis and early warning.

5A to 5D are views illustrating an example of image synthesis of an image diagnostic apparatus using multiple cameras according to the present invention. 5A and 5B illustrate a composite image obtained by synthesizing the image image and the thermal image and outputting them as one screen. In particular, FIG. 5A illustrates a composite image generated by combining a temperature FIG. 5B shows an image obtained by synthesizing only the outline portion of the object on the image image in the thermal image with respect to the circuit breaker (FIG. 5B) mode).

On the other hand, FIG. 5C shows a synthesized image of the sound image in the engine room of the vehicle, which is represented by one image. In this case, the sound is expressed in color, and the more the red, the stronger the noise intensity is.

FIG. 5D shows that the image image, the thermal image, and the sound image are synthesized with respect to the differentiator cylinder portion of the thermal power plant and output as a single image. In the case of degradation, the image is expressed in black and white, In the case of sound, it is expressed by color, and as it goes to red, it is shown that the generation of noise in the corresponding region is large, so that it is possible to grasp the intensity of heat or noise and the position where the noise is generated. It can be easily understood whether the heat and noise are generated at the same position or independently at different positions or individually at different times.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation, Various changes and modifications will be possible.

100: sensing unit 110: video camera module
120: thermal imaging camera module 130: sound camera module
200: control unit 210: first control module
220: second control module 230: third control module
300:

Claims (8)

1. An image diagnostic apparatus using multiple cameras,
A sensing unit including a video camera module, an infrared camera module, and a sound camera module, for sensing an operation state of the equipment and generating an input image;
A control unit for generating a composite image using the input image picked up by the sensing unit;
A storage unit for storing the input image captured by the sensing unit and the composite image generated by the control unit; And
And an output unit that displays at least one of the input image or the composite image on a screen,
Wherein the control unit receives the input image from at least two of the image camera module, the thermal image camera module, and the sound camera module to generate the composite image.
The method according to claim 1,
Wherein the input image includes a video image, an infrared image, and a sound image, and the sensing unit simultaneously captures and generates the input image.
The method according to claim 1,
The composite image may be obtained by combining the image image and the thermal image, compositing the image image and the sound image, compositing the thermal image and the sound image, or compositing the thermal image and the thermal image, And the image is synthesized and generated.
The method of claim 3,
Wherein when the combined image is generated by combining the image with the image, only the outline part of the object on the image is superimposed on at least one of the thermal image and the sound image.
The method according to claim 1,
Wherein the control unit comprises: a first control module for controlling the image and the thermal image to generate a first image by merging with each other; a second control module for controlling the sound image to generate a second image; And a third control module for synthesizing the first image and the second image received from the second control module to generate the composite image.
6. The method of claim 5,
Wherein the first image generated by the first control module is generated by fusing an outline of an object on the image image to the thermal image or by merging the thermal image over a temperature set in the image, An imaging device using multiple cameras.
The method according to claim 1,
Wherein the output unit includes an alarm module for comparing the composite image with at least one of a reference image image, a reference thermal image, and a reference sound image stored in the storage unit, thereby alerting an operation failure. Imaging device.
8. The method of claim 7,
Wherein the malfunctioning warning by the alarm module includes at least one of a crack, an erroneous setting, an uninstalled state, an abnormal overheating, and a noise of a facility or an apparatus.

Images