JP2003187223A - Device, system and method for diagnosing image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To extract color distribution of a surface from an image by photographing an object to be diagnosed with high precision and to diagnose a state of the object from the extracted color distribution. <P>SOLUTION: An image diagnostic device is constituted of an image data storage part 1 for storing image data by photographing the object to be diagnosed with a camera, etc., a CAD (computer aided design) data storage part 12 for storing three-dimensional CAD data of the object to be diagnosed, a brightness noise removal part for removing brightness noise on the image data from a position/intensity of a light source by using the three-dimensional CAD data to the image data and a three-dimensional image display part 4 for displaying an image from which the noise is removed by sticking the image on the threedimensional CAD data. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image diagnostic apparatus, an image diagnostic system and an image diagnostic method, a technical support system, a technical support method, a maintenance support system, and a maintenance support method, and particularly, using three-dimensional CAD data. An image diagnostic apparatus, an image diagnostic system, an image diagnostic method, and a technical support system that remove the influence of the surface condition of a diagnostic object,
Technical support method, maintenance support system, maintenance support method.

[0002]

2. Description of the Related Art There are many fields in which experts make judgments based on their experience by observing actual objects such as the deterioration of metal materials and oils, and the health condition diagnosis of patients. When these are analyzed, it is considered that the expert mainly processes the color information of the diagnosis target and comprehensively determines it.

As an example, when attention is paid to a method of diagnosing deterioration of high temperature materials such as blades of a thermal power turbine, since the surfaces of these metal materials are operated at a high temperature, phenomena such as thermal oxidation and corrosion occur, which causes the metal. The surface color of itself changes or irregularities occur. Conventionally, the inspection committee judges the color and unevenness of the material surface at the time of periodic inspection, and judges the operating temperature and the estimated remaining life.

Further, as a technique for monitoring the life of a metallic material, for example, as in the "lifetime monitoring device for high temperature equipment" described in Japanese Patent Laid-Open No. 9-304131, material deterioration and damage measurement information and roughness change. There has been a known example in which changes in temperature and stress conditions during use are estimated based on tendency information, and the damage rate and remaining life of materials are evaluated. The known example also includes a realization means for correcting the temperature estimated from the measurement result of the surface roughness with the "simple temperature distribution" estimated from the color tone of the surface.

[0005]

In the conventional judgment method used in the periodic inspection of the thermal power turbine, the inspector visually judges the material surface state of each moving blade. There was a problem that took In addition, since the color information obtained by visual inspection cannot be quantified, it is difficult to share it with others, and there is a problem that it is not reproducible. Further, there is a problem that the inspector has to go to the site every time the inspection is performed because it depends on the visual judgment.

Further, with regard to automation of the color recognition technique by human eyes, there is a problem that the color on the image easily changes depending on the brightness of the place where the image is taken and the kind of light. In order to evaluate the color, it is necessary to accurately evaluate the color of the image regardless of the environment.

The above-mentioned Japanese Patent Laid-Open No. 9-304131 discloses
Although it is configured to have a means for automatically measuring the surface color tone, there is no description of means for solving the above problems.

An object of the present invention is to provide a processing device for processing information on the surface condition of an object using the information on the three-dimensional shape of the object.

Another object of the present invention is an image diagnostic apparatus, an image diagnostic system, or an image diagnostic capable of accurately extracting a surface color distribution from an image of a diagnostic object and diagnosing the state of the object from the extracted color distribution. Is to provide a method.

The other objects of the present invention will be explained by the description of the specification.

[0011]

One of the features of the present invention is:
A processing device having a removal processing unit that removes an influence on information of a target surface state from information of a target three-dimensional shape, and a processing unit that superimposes information of the target surface state on the information of the three-dimensional shape. is there.

The present invention is also directed to an image processing apparatus for diagnosing the state of a diagnosis target from image information of the surface state of the diagnosis target, based on the image information of the surface state of the diagnosis target and the three-dimensional shape information of the diagnosis target. It is characterized by having a removal processing unit that removes the influence on the surface state of the target and a processing unit that superimposes an image of the diagnostic target on the three-dimensional shape.

Further, according to the present invention, in an image diagnostic apparatus for diagnosing a condition of a diagnosis target from a surface condition, the influence on the surface condition is removed from the three-dimensional shape of the diagnosis target, and the diagnosis target is placed on the three-dimensional shape. The feature is that the photographed image is pasted and displayed.

Further, the present invention is characterized in that the influence on the surface state is eliminated and then the state of the diagnosis object is estimated.

Further, the present invention is characterized in that the influence on the surface state is removed from the three-dimensional shape of the diagnosis target for each of the images obtained by photographing the diagnosis target from a plurality of different angles.

In the present invention, as an example of diagnosis, the physical state of an object may be diagnosed from colors.

Further, according to the present invention, the influence of light on the surface of the object is calculated from the three-dimensional shape of the object to be diagnosed, the position of the illumination of the environment where the image is taken, and the intensity of the illumination, and from the color on the image, The feature is that the color of the object and the influence of the brightness of light are separated.

Further, according to the present invention, when the position and intensity of the illumination of the environment in which the image is captured are unknown, a clear boundary of the color is extracted from the color distribution on the image of the captured target, and the extracted color is extracted. It is characterized in that a shadow position candidate area is set from the boundary and the position and intensity of the illumination are estimated from the candidate area.

Further, the present invention is characterized in that the object is photographed outdoors and the position of the sun at the time of photographing is obtained from the photographing time and the latitude / longitude of the photographing point.

Further, according to the present invention, the surface of the photographed object is
The feature is that the region is divided based on the three-dimensional shape, and the feature amount obtained from the image of each region is adopted as the target attribute.

Further, the present invention is characterized in that, as the feature amount obtained from the image area, any one of the average value, the variance, the mode value of the color of the image area, or a combination thereof is adopted.

Further, according to the present invention, as the feature amount obtained from the image area, the surface of the image area is grouped into areas having similar colors, and the surface area of the area is calculated together with the color information of each area to calculate the characteristics of the object. It is characterized by the quantity.

Further, according to the present invention, the individual number displayed on the surface of the part to be diagnosed and the three-dimensional CAD data are linked, and the individual number can be read from the image when the object is photographed. As described above, the three-dimensional CAD data is referred to by taking an image and reading the individual number with the image diagnostic apparatus.

Further, according to the present invention, as a result of projecting an image on a three-dimensional shape and associating each pixel on the image with each point on the three-dimensional shape, the contour of the evaluation target on the image and the three-dimensional shape are evaluated. It is characterized in that the positions of the contours in the shapes are compared with each other and the resulting difference is used as a change in the shape of the material.

Further, according to the present invention, at the same time as photographing the object, the three-dimensional shape of the object is measured by means such as laser measurement,
The three-dimensional shape is used for estimating the surface state.

Further, according to the present invention, information such as a past image of a subject to be evaluated or a feature amount obtained from the image is accumulated, and the past information is compared with information of a newly input image to obtain a surface image. It is characterized in that the state is estimated.

Further, the present invention is characterized in that the object to be evaluated is photographed at the actual driving site, only the image information thereof is transmitted using communication means such as the Internet, and the image is diagnosed by the image diagnostic system. It is characterized by providing an image diagnostic system.

The above features and other features of the present invention are as follows.
Further description is provided by the following description.

[0029]

BEST MODE FOR CARRYING OUT THE INVENTION Next, referring to FIGS.
Embodiments of the image diagnostic apparatus, the image diagnostic system, and the image diagnostic method will be described.

<< Embodiment 1 >> FIG. 1 is a block diagram showing a configuration of an embodiment 1 of an image diagnostic apparatus according to the present invention.

The image diagnostic apparatus shown in FIG. 1 stores an image data storage unit 1 for storing image data of a diagnosis target photographed by a photographing unit such as a camera, and three-dimensional CAD data for the diagnosis target.
A luminance noise removal processing unit 3 that removes luminance noise on the image data from the light source position and intensity by using three-dimensional CAD data for the image data of the CAD data storage unit 2 and the image data storage unit 1; The three-dimensional CAD image
It is composed of a three-dimensional image display unit 4 which is pasted and displayed on the data. The image data is stored in the image data storage unit 1,
It may be read at the time of processing, or may be input from the outside and input to the luminance noise removal processing unit 3 without or via the image data storage unit 1. Further, the three-dimensional CAD data may be stored in the CAD data storage unit 2 and read out at the time of processing, or may be input from the outside and pass through the CAD data storage unit 2 or not to pass through the luminance noise removal processing unit 3.
Can be input to. The same applies to the following, but various storage devices can be used as the storage unit according to the capacity to be stored. For example, solid-state memory using semiconductors, light and / or
Alternatively, a DVD or CD storage device using magnetism can be used. The processing units such as the luminance noise removal processing unit 3 can use an arithmetic unit such as a microprocessor or a personal computer (PC) as hardware. The display unit such as the three-dimensional image display unit 4 can use an arithmetic unit such as a personal computer (PC) and an output unit such as a CRT monitor, an LCD monitor, and a printer. Further, pasting and displaying the noise-removed image on the three-dimensional CAD data is considered to be an aspect of a superimposing process for superimposing the noise-removed image information and the three-dimensional CAD data, and the superimposed information is displayed. Instead of displaying, it may be output as output information, or this output information may be used for other processing.

The image data of the image data storage unit 1 is image data of a subject to be diagnosed. Although the image format may be any format, since the image needs to be input as digital information in the configuration of the present embodiment, for example, when directly input from a CCD camera, this input section is provided with an A / D conversion process, A process of converting an analog signal into a digital signal is required. It is also possible to directly import images taken by a digital camera or images obtained by converting still images taken by a still camera with a digitizing device such as a scanner.

However, in consideration of the object of the present invention, it is better to input the evaluation target in a state where it is directly photographed as much as possible, and it is more preferable that the scanner or the like is not used.

As for the data format of a digital image, for example, when the image size is 220 pixels × 256 pixels, the screen is composed of an aggregate of 256 pixels in the x direction and 220 pixels in the y direction, and For each pixel, R (red), G (green),
It has information on a color determined by three values of B (blue) (details about the color will be described later).

Next, the three-dimensional CA of the CAD data storage unit 2
The D data is data describing the three-dimensional shape of the diagnosis target, and the coordinates when it is placed at an arbitrary position and orientation in the three-dimensional space can be obtained.

The three-dimensional shape of interest here means a shape in which a shadow is formed on the surface by one light source. Further, when the surface is irregularly formed with a curved surface or a flat surface and it is difficult to identify an arbitrary point on the surface, further effects can be expected.

Next, the processing of the luminance noise removal processing section 3 will be described. The luminance noise removal processing unit 3 is a diagnostic target 3
This is a process of calculating the influence of the shadow on the surface of the diagnosis target from the dimensional CAD data and the position of the light source, and correcting the brightness on the actual input image.

FIG. 2 shows the flow of the luminance noise removal processing section 3.
Will be explained. In process 21, the color information of the input image is associated with the three-dimensional shape. In process 22, with respect to the coordinates P (x, y) of the image projected on the surface of the 3D shape,
The processes starting from the process 23 are performed with reference to the sequence. In process 23,
From the 3D shape of the part, the position and intensity of the light source, the pixel P (x,
The color I '(Rxy, Gxy, Bxy) of the object itself in y) is calculated. In the process 24, it is determined whether the calculation has been completed for all the pixels. With the above processing, the influence of luminance due to the influence of light can be corrected from the color information on the image projected in the 3D shape.

The process 21 of projecting the color information of the image onto the three-dimensional shape corresponds to each coordinate on the surface of the 3D shape and the coordinate on the image, and is on the coordinate system on the image. The color information is made to correspond to the coordinate system of the surface of the 3D shape.
If the relationship between the position and angle of the camera and the object and the camera parameters are known, it is possible to easily take correspondence between both coordinates. Even if the relationship between the position and angle of the camera and the object is not known, the three-dimensional shape of the target object can be found, for example, as described on pages 91 to 99 of "3D Image Measurement" published by Shokoido. At least 6 corresponding points are taken between the captured image and the captured image (the point (X, Y, Z) on the three-dimensional shape is
Corresponding to the point (x, y) on the image), thereby 3
A method of obtaining a conversion formula between the coordinate on the dimensional shape and the coordinate on the image can be adopted.

Since the image is two-dimensional information obtained by viewing a three-dimensional object from a certain viewpoint, color information corresponding to all surface coordinates is set by complementing the missing information. Further, in order to minimize the lacking color information, it is possible to use a plurality of images taken from different angles as the input image.
Further, for example, in the case of a moving blade of a gas turbine, it is conceivable that the surface of the material is shaved and the shape is changed due to deterioration. In such a case, when projecting the color information on the image into a three-dimensional shape, it is necessary to consider the properties of the parts and select a point whose shape does not change as much as possible. Further, if the coordinates of the contour line of the evaluation target on the image are obtained in advance, the difference information is adopted as the deterioration diagnosis material assuming that the shape of the part has changed in the portion where the contour of the three-dimensional shape and the contour of the image do not match. You can also

Next, the method of calculating the color of the object itself corresponding to the process 23 will be described in more detail. A method for calculating color information on the surface of a three-dimensional object in the presence of illumination is a method called ray tracing described in various documents, for example, "CG Laboratory" published by Nikkei McGraw-Hill. As described on pages 47 to 108, this is a method of calculating the color of the surface of the object from the position of the three-dimensional object and the position of the illumination. Specifically, it will be described with reference to FIG. In FIG. 3, an evaluation target object is 31, an illumination position is 33, a viewpoint (camera position) is 37, and a normal vector at a point 32 on the target 31 is 35.

The calculation method of the brightness of the object in consideration of the influence of the illumination is as follows.
It is expressed as in Equation 1.

C = (s * kd * cosα + ke) * Cs + s * ks * cosβ * Cw (Equation 1) where cosα = −L * N cosβ = 2 (L * N * N * V) −L * V s: Intensity of incident light kd: Diffuse reflection coefficient Cs: Surface color of object when light is not considered ke: Environmental reflection coefficient ks: Specular reflection coefficient Cw: White (value 255) Equation 1 is the color Cs of the object itself. This is a method in which the color C that is known and that is visible under a predetermined illumination condition is calculated based on the color of the object itself.

Since the purpose of the present invention is to calculate the color of the object itself when the color of the object under the predetermined illumination is known as the color on the image, Cs = (Cs * ks * cosβ * Cw) / (s * kd * cosα + ke) (Equation 2), and the color of the object itself is calculated by using Equation 2 in the process 3.

By performing the above processing for all the pixels of the target object on the image and converting the image with the color of the obtained object itself, the evaluation target can be converted into a color that is not affected by illumination.

When the target image is color,
The point P on the target surface for each of Cw in the equation 2
The values of Rxy, Gxy, and Bxy of (x, y) are obtained.

The three-dimensional image display unit 4 takes the correspondence in the luminance noise removal processing unit 3 and displays the image converted into the color of the object itself on the three-dimensional shape. This is a three-dimensional C
This can be realized by adopting the color information of the points on the image that the luminance noise removal processing unit has taken correspondence with as the color information of each point on the surface of the three-dimensional CAD data when displaying AD.

The first embodiment of the image diagnostic apparatus according to the present invention has been described above. By taking this configuration,
By only capturing a two-dimensional image of the diagnosis target, a three-dimensional image in which the influence of light is removed can be created, and the inspector can remotely perform the condition diagnosis of the target without going to the site.

<Embodiment 2> FIG. 4 is a block diagram showing the configuration of Embodiment 2 of the image diagnostic apparatus according to the present invention.

The image diagnostic apparatus of FIG. 4 comprises the image data storage unit 1, the CAD data storage unit 2, the luminance noise removal processing unit 3, the material deterioration diagnosis unit 5, and the deterioration diagnosis result data 6 of the first embodiment. Become.

The configuration of the present embodiment is a configuration in which an image converted into the color of the object itself by the luminance noise removal processing unit 3 is input, and deterioration diagnosis is performed using the image processing function.

The processing of the material deterioration diagnosing unit 5 in FIG. 4 will be described. The present processing unit determines a deterioration state from an image of a subject to be evaluated, and various embodiments can be considered. A first embodiment of the material deterioration diagnosis unit 5 will be described next.

For convenience of explanation, here, a moving blade of a gas turbine is taken as an example of an object to be evaluated. It is considered that the distribution of thermal stress on the surface is determined to some extent depending on the shape of the object and the operating conditions. For example, the tip of the rotor blade is likely to reach the highest temperature and is likely to deteriorate. Based on these findings, the surface to be evaluated is divided in advance into regions where the deterioration situation is likely to be uniform. Then, the surface color information is extracted for each area, and information is provided for each individual as area-specific color information data. 1 and environmental conditions such as operating time and temperature
If a target image is newly input as a record, it is divided into areas, the feature amount is calculated, and images with similar feature amounts are searched from the database to determine the state. Is the way to do it.

The process flow of the first embodiment of the material deterioration diagnosis unit 5 will be described below with reference to FIG. The diagnostic object-specific feature quantity storage means 41 is for an object of a specific shape such as a moving blade,
This is a means for extracting information regarding colors for each predetermined area and storing it together with environmental conditions such as operating time. FIG. 6 shows an example of a specific storage format.

The table of FIG. 6 is an example of the storage format of the feature quantity storage means for each diagnosis object, and is 51a, 51b, 51c.
Is an image of a part operated under certain conditions. The images of these 51a, 51b, 51c correspond to the records 52a, 52b, 52c in the table, respectively,
Each image can be referenced from its corresponding record. 53 to 60 in the table are attribute information regarding each image. Reference numerals 53 to 56 are information relating to a known operating environment, for example, parts type, plant name, material, operating time, and the like. Reference numeral 57 is an estimated or actually measured temperature condition when it is obtained. Reference numerals 58 and 59 are region-specific feature amounts obtained from the image. The feature amount will be described later. 60
Is a diagnostic result for each image, and is, for example, an estimated operating temperature or life in the case of a moving blade of a gas turbine. By accumulating the database using these attribute information as records, the component groups having similar attributes can be specified by the attribute information such as 53 to 56, and the similarity can be extracted from the images of those component groups.

Based on the format of the diagnostic target feature quantity storage means 1 described above, the flow of the processing in FIG. 5 will be described. The process 42 is a process of dividing an area on the surface from the 3D shape. The process 43 is a process of calculating a feature amount for each area. The process 44 is a process of referring to the diagnostic target feature amount storage unit 41 from the feature amount calculated in the process 3, extracting records having similar feature amounts, and determining the state from the contents. Process 45 is a process of registering information regarding the image determined in process 44 in the DB. Process 46 is means for outputting the determination result of process 44. By the above processing, from the images of parts with fixed shapes such as moving blades,
It is possible to extract the feature amount based on the same reference for the surface state and easily search the database for data having a similar surface state to determine the state.

A method of dividing the area on the surface in the process 42 will be described below. FIG. 7 shows an example of a method of dividing the surface area of the rotor blade. 61a, 61b, 61c, 61 in the figure
Each d represents an area boundary line on the surface, and the area surrounded by each boundary line represents each divided area. Since it can be defined based on the 3D shape, the same area can always be defined regardless of the angle at which the image is captured. Three
If there is no D shape information, it is difficult to perform area division in consideration of the curved surface on the surface as shown in FIG. 7. It is necessary to appropriately determine the area division method so that the characteristics of the deterioration of the part are most easily expressed and that the number of areas does not become too large in consideration of convenience such as retrieval. As a criterion for determination, it is appropriate to determine which region on the surface of the component is to be seen and in what order when the inspector visually determines.

If the load of processing such as retrieval is allowed, the size of the area is set to be somewhat small (for example, 3 cm square) and kept constant, and the feature amount of each area is calculated to obtain the similarity. It is also possible.

Next, the contents of the process 43 will be described in more detail. The area-based feature amount is obtained by extracting color information about each area divided in the process 2, and using information such as the average value, variance, and most frequent value of the color information to represent the color characteristics of the area. This is a process of calculating numerical data.

In this processing, assuming that the coordinate range of the corresponding area is (X0, Y0) ... (Xi, Yj), the average and variance of the color information of all the pixels included therein are calculated, and the distribution information is calculated. Is a process for calculating the most frequent value.

Here, for convenience of explanation, an outline will be given of digitization of color information. The representation of colors in a digitized image is usually R (red), G for each pixel.
It is expressed as a value uniquely defined by three values of (green) and B (blue). Each color element of RGB takes a value from 0 to 255, (0,0,0) is black, (255,255,2)
55) is white, red (255,0,0), green (0,255,
0) and blue (0, 0, 255).

Based on this RGB color system, for example, XY
Z system can be expressed by RGB linear conversion formula, Lab, HSB
The display system such as can also be expressed from the XYZ values using a conversion formula.

X = 0.412 * R + 0.358 * G + 0.180 * B Y = 0.213 * R + 0.715 * G + 0.072 * B Z = 0.019 * R + 0.119 * G + 0.950 * B ... ( Formula 3) L = 116.0 * (Y / 100) ^(1/3) -16 a = 500 * (XXn-YYn) b = 200 * (YYn-ZZn) (Formula 4) However, X / 95.04 > 0.008856 XXn = (X / 95.04) ^(1/3) X / 95.04 ≦ 0.008856 XXn = 7.787 * (X / 95.04) +16/116 Y / 100 When 0.0> 0.008856 YYn = (Y / 100) ^{(1/3) When} Y / 100.0 ≦ 0.0088856 YYn = 7.787 * (Y / 100) +16/116 Z / 108.89 > ZZn = (Z / 108.89) when ^{0.008856 (1/3) Z / 108.89 ≦} 0.008856 When ZZn = 7.787 * (Z / 108.89) +16/116 RGB expresses a color uniquely defined by the size of the numerical value of each of the three vectors, while the Lab color system The brightness can be separated and expressed as L, the red-green component of the color as a, and the yellow-blue component of the color as b, so it is relatively resistant to fluctuations in illumination, and the two axes of a and b give the tint. It is easy to express the change of.

As described above, basically any expression system may be used for digitizing the color, but since there are various characteristics depending on the color expression system, the color change characteristics determined by each material can be more accurately determined. It is good to select an expression system that can be expressed in. Here, Lab is used as an example.

In the process 43 of FIG. 5, the average value of colors for each area
The variance or the like is calculated, but since the color is represented by a three-dimensional value as described above, the average, variance, etc. are calculated independently for each numerical value.

Whether the average value is used as the feature amount or the mode value having the largest number of pixels of the corresponding color is used needs to be considered when the two values are far from each other. Since the purpose here is to select the value that best represents the color characteristics of each area, as an example, the threshold of dispersion is determined for each size of the area, and the dispersion is equal to or greater than the threshold. In this case, it is also possible to change the mode value by using a mode value because the reliability of the average value is low.

Next, the process 44 will be described in more detail. The process 44 is based on the feature amount extracted in the process 43.
It is a means for retrieving similar ones from the image feature amount data stored in advance in the feature amount storage means 41 for each diagnosis object. The simplest method for determining the degree of similarity is to select a method that minimizes the sum of the error in the value of the feature quantity for each area. In addition, when an inspector visually inspects the relevant parts, a method of weighting the errors of the respective feature amounts and summing them may be considered depending on which part of the surface is to be seen and in what order. In addition, when calculating the error, three values of three values of color (RGB, Lab, etc.)
There is also a method of performing evaluation using only the error between the values of a and b when using the Lab color system, instead of using all three. In this case, there is an effect that the similarity can be determined only by the tint regardless of the brightness of the color itself.

Further, the number of data having a high degree of similarity is not necessarily limited to one, but N cases having a high degree of similarity or a threshold value is set for the error value for judging the degree of similarity. It is also possible to select all the values which are less than the value as candidates and directly look at the image from them to make the final determination by the user.

As described above, according to the processing of FIG. 5, the luminance noise processing unit 3 projects color information on a three-dimensional shape, and from the image information having the three-dimensional position information with noise removed, the photographing angle of the input image is obtained. It is possible to extract the feature amount of the component surface and compare the similarities with each other regardless of the difference.

Next, the processing flow of the second embodiment of the material deterioration diagnosis unit 5 of FIG. 4 will be described.

In the second embodiment, a test piece of which environmental conditions such as heating temperature and time are known is photographed in advance, and color information is extracted from the image to obtain a database of external environment and material surface color. Created and based on this, estimate the environmental conditions received by the evaluation target from the color of the image that has the three-dimensional position information output from the luminance noise processing unit in FIG. Is the way to do it.

A specific processing flow will be described with reference to FIG. Reference numeral 71 in FIG. 8 is a material surface color data storage means. This means stores the relationship between the physical properties to be evaluated and the surface color in that case for each material to be evaluated in advance for each rank that requires estimation.

An example of the storage format is shown in FIG. For example, assume that the operating time of a certain material X is known and the operating temperature of the environment in which the material is operating is desired to be known. For example, if the range required for temperature identification is divided into three ranks of 500 degrees or less, 500 degrees to 1000 degrees, and 1000 degrees or more, what value is the color evaluation value of the surface color in each rank? Whether or not it is defined is defined as shown in FIG.

From FIG. 9, for example, the material is 400 degrees to 50 degrees.
If it was operated between 0 degrees, the a value would be 8 to 1.
5 and b values take values from 10 to 30. Also from 501 degrees
If it is operated at 1000 degrees, the a value will be 2 to 5, and the b value will be -3 to 10. When it is 1000 degrees or more, the value of L is 40 or less. In this example, there are three ranks for simplification, but each actual material is appropriately classified by the number of ranks for which it is necessary to know the change in the physical characteristics.

The color of an object photographed by a camera is
The color defined by the material surface color data storage means 2 should be defined as a value when the image is captured under a light source that serves as a predetermined reference, because it depends on the type of illumination to be captured. For example, it is preferable to use a white light source or the like that can easily display the color of the object without any bias. In this way, the color to be exhibited in that state is classified into one of the color representation systems (La
b, XYZ, etc.) is the material surface color data storage means 2.

The process 72 is a process for inputting the 3D shape image data output from the luminance noise removal processing unit 3 in FIG. The process 73 is means for extracting an area from the input target image for each physical characteristic rank stored in the material surface color data storage means 3. This process will be described in more detail later. Process 74 is means for estimating the state of the evaluation target from the area information for each color rank extracted in process 73. This process will also be described in detail later.
Process 75 is means for outputting the state of the evaluation target determined in process 74. With the above processing, the second embodiment of the material deterioration diagnosis unit 4 can be realized.

FIG. 10 shows a more detailed processing flow of the processing 73. In this example, it is assumed that the color information of the image is RGB and the physical characteristic rank is stored in the Lab color system.

The processing 91 is to perform the color evaluation value R of the pixel I from the image.
This is a process of reading GB. A process 92 converts the color evaluation value of the pixel I read in the process 1 into the Lab color system using the equation (4). In process 93, the color evaluation value in the Lab color system converted in process 2 is compared with the value stored in the material property storage unit 3, and the color evaluation value of the pixel I is any one of 1, 2, and 3 in FIG. Decide whether to belong to the rank. The process 94 is a process of storing the rank determined in the process 3. The pixel values of x and y are associated with each other and stored. In the process 95, 1 is added to the value of I and the pixel value to be evaluated is advanced by one. The process 96 checks whether the above process has been completed for all pixels. If the value of I is smaller than the total number of pixels of the image, the process returns to step 1, and if it is equal to or larger than the total number of pixels of the image, the above process is terminated.

By the above processing, the target image is ranked. That is, the threshold value is determined for each pixel from the value of the color and the value is converted into multiple values.

FIG. 11 is a view exemplifying an image of an image of a moving blade of a turbine. Reference numeral 1 is a turbine rotor blade, and reference numerals 2, 3 and 4 are areas having different surface colors.
In this example, the area 2 is not uniform in color and has unevenness. However, when the processing of the color extraction by condition 73 described in FIG. 8 is used, the Lab evaluation values of all pixels in the area surrounded by 2 in the figure fall within the area classified into rank 2 in FIG. As a result, the result of ranking the colors on the surface of the turbine by pixel is as shown in FIG.

Next, the process 74 of FIG. 8 will be described in more detail. The state determination here is means for outputting the state determination result regarding the physical information of the material from the color rank area information output from the process 73. The simplest output is information regarding the coordinates of the region most affected by the highest temperature as a result of ranking. In FIG. 12, it corresponds to the tip of the moving blade 114. By presenting the region coordinates in a 3D shape, the user can obtain information regarding the part of the component that is most seriously deteriorated.

Further, as still another state determination method,
This is a method of calculating and presenting area information for each rank. Since the rank of each pixel on the image is known, it is possible to calculate the area for each rank from the surface area information of the 3D shape of the part and the number of pixels for each rank.

The area information for each state of deterioration can be effectively used for determining the degree of deterioration. Information on the surface area of a three-dimensional object cannot be accurately obtained from only the captured two-dimensional image. By using the three-dimensional CAD data, these effects can also be obtained in the deterioration diagnosis.

In the above embodiments, it is assumed that the position and intensity of the illumination in the environment where the image to be evaluated is photographed are known, but in practice there is no means to measure them or the accuracy of the information is low. There is a possibility that

In such a case, it is assumed that a portion where the color boundary on the image is clear is a shadow candidate area, and the position of the illumination is determined from the edge portion of the shadow candidate area and the three-dimensional shape coordinate of the object. It is possible to estimate. For example, 121 in FIG.
There is a three-dimensional shape such as 122, and there is a region such as 123 whose color is different from that of the surrounding area on the surface. Assuming that this area is a shadow projected on the surface, lines 124a and 124b connecting the edge portion of the shadow candidate area and the projection portion estimated from the three-dimensional shape are created from the three-dimensional shape, and the lines are extended. Estimate the lighting position 125.

If the surface of the object to be evaluated is dark in color itself and the influence of light on the shadow is unlikely to occur, a rod of known length is set up vertically with the object to be evaluated, and the shadow is illuminated. It is also possible to use a method of estimating the position of.

As for the intensity of illumination, the shadow area 12
Assuming that the difference between the brightness of the shadow area 6 and the brightness of the area 127 other than the shadow occurs due to the influence of the illumination 5, the color of the shadow area 6 is C
By substituting the color of the area 7 other than s and the shadow as C, the light intensity s can be obtained.

When it is difficult to obtain the position and intensity of illumination in advance, the above method may be used, but another method is to take a picture outdoors. In the case of outdoors, the sun is the illumination light, and the latitude and longitude of the sun are known if the shooting time and the latitude and longitude of the shooting point are known.

<< Third Embodiment >> Next, a third embodiment of the image diagnostic apparatus according to the present invention will be described.

FIG. 14 is a block diagram showing the configuration of the third embodiment. This is the same as the second embodiment except that the individual number collation unit 7
It is a configuration in which.

Parts such as turbine rotor blades are usually engraved with a solid number on the surface for individual management. The individual number collation unit 7 also captures these individual numbers when capturing an image of an evaluation target, and when the image is input, first identifies the individual number of the capturing target by reading the individual number by image processing, This is a process of searching the three-dimensional CAD data using the individual number as a key and extracting the corresponding data.

By using this configuration, it is not necessary to be aware of the individual number when capturing an image or to transmit the individual number information together with the image data, which reduces the amount of work at the image capturing site and saves information. It can be quantified.

<< Fourth Embodiment >> Next, a fourth embodiment of the image diagnostic apparatus according to the present invention will be described.

FIG. 15 is a block diagram showing the structure of the fourth embodiment. This is the same as the second embodiment except that the past image data 8
This is a configuration in which a storage unit of is added. The past image data 8 is image data obtained by projecting an image to be evaluated into a three-dimensional shape by the luminance noise removal processing unit 3 and removing the luminance noise to convert it into the color of the object itself, or the feature amount described in FIG. Data etc.

In the configuration of this embodiment, the material deterioration diagnosis unit 5
Then, by comparing the past image data with the image data newly input this time, it is possible to quantitatively evaluate the change in the surface state in time series. For example, the color difference is calculated for each pixel on the surface of a three-dimensionally projected image, and the distribution of color change amounts is calculated. As a result, if it is assumed that the degree of deterioration is high in the part where the color change is large, the part in which the deterioration progresses quickly can be specified.

<< Fifth Embodiment >> Next, a fifth embodiment of the image diagnostic apparatus according to the present invention will be described.

FIG. 16 is a block diagram showing the structure of the fifth embodiment. This is a configuration in which the illumination effect removal processing unit 9 is added to the configuration of the second embodiment. The illumination effect removal processing unit 9 is a unit that converts the color information of the target captured under the unknown illumination into the color information captured under the reference illumination.

The effect of illumination will be described with reference to FIG. The environment surrounded by a frame 171 in the figure is an environment in which an object is imaged under reference illumination, and the environment surrounded by a frame 172 is an environment in which an object is imaged under illumination under unknown conditions. 1 in the figure
73a and 173b are the same photographing target. 174a,
174b is the same camera, 175 is a reference light source,
Reference numeral 76 is a light source whose condition is unknown. 177a is the reference light source 1
An image of the object 2a taken by the camera 174a under 75,
Reference numeral 177b is an image obtained by photographing the target 173b with the camera 174b under the unknown light source 176.

That is, in environment 171, environment 172, since the object is photographed under different illumination conditions, the color information extracted from the images 177a and 177b photographed under each condition is different due to the influence of illumination. it is conceivable that.
Since the color information is extracted from the image of the imaged object and evaluated by the material deterioration diagnosis unit 5, it is necessary to use the imaged image under the reference illumination.

Therefore, the illumination effect elimination processing section 9 changes the image 177b taken under a light source whose shooting conditions are unknown, such as on-site,
It is means for converting into an image 177a taken under a reference light source.

As a means for correcting the influence of illumination, there can be used, for example, a method for correcting the color of an image using a patch image as described in JP-A-2001-69363. This method extracts the evaluation value of each color sample from an image obtained by shooting a plurality of color samples whose color evaluation values (RGB, etc.) are known in the same environment (lighting conditions etc.) when the evaluation target is photographed. This is a method of estimating the color in the shooting environment and the actual color (under the reference light source) from the deviation between the known color of each color sample and the color on the image.

This method is a method in which a plurality of colors obtained by dividing the values of the three components of the color (CMY) into equal intervals are adopted as color samples, and it is possible to comprehensively correct all color distributions. This is a correction method that takes into consideration.

In this method, the larger the number of colors used as the color sample, the higher the color reproduction accuracy, but there are 256 ³ = 16777216 possible combinations of colors (each RGB value is a value of 0-255). Therefore, the number of colors that can be taken as a color sample will be limited.

In consideration of the above, the first embodiment of the illumination influence removal processing unit used in the present invention is the color sample in the range where the color of the material to be evaluated changes as the color selected for the color sample of the sample. This is a method of increasing the accuracy of color reproduction within that range.

In the present invention, as described with reference to FIG. 9, the range in which the surface color changes can be limited depending on the material. Therefore, it is a method of increasing the color reproduction accuracy of the surface color according to the characteristics of the evaluation target, rather than accurately reproducing the color as a whole as in the known example of Japanese Patent Laid-Open No. 2001-69363.

The second embodiment of the illumination effect removal processing section uses the fact that the physical state of the material to be evaluated is ranked as shown in FIG. 9, and the color evaluation value of each rank is used. By calculating the intermediate value, maximum value, and minimum value of, a color sample that reproduces the average color of each rank is created, and the color sample is photographed together with the evaluation target. Since the numerical values of the colors extracted from the pixels in the color sample area in the captured image are the color evaluation values of each rank under the environment in which the target component is evaluated, the range of colors of each rank is centered around this intermediate value. Assuming that the color is moved in parallel, each rank-based color evaluation value in FIG. 9 can be converted as it is as an evaluation value in an unknown environment in which the target is photographed.

That is, the second embodiment of the illumination effect elimination processing section has the function of adjusting the condition for evaluating the state of the material by converting the color value stored for each characteristic of the material instead of the photographed image. Become.

The above-described two examples have the same idea but different embodiments, and therefore it is preferable to use them properly according to the processing time constraints and the like.

Next, an embodiment of the image diagnostic system according to the present invention will be described with reference to FIG.

Reference numerals 146a, 146b ... In the figure are sites where the parts to be evaluated are used. 151 is
It is a base that holds three-dimensional CAD data of parts operated at the plurality of sites 146a, 146b ... And data related to management and maintenance such as past operation record data and deterioration status data. The sites such as 146a, 146b ... And the 151 bases are communication means 1 such as a telephone line.
It is connected by 50 and is in a state in which data can be transmitted and received.

At the sites 146a, 146b, etc., the cameras 147a, 147b, etc., evaluate objects 148a, 1 such as moving blades.
48b ... Is photographed, and the photographed image and additional information 152a, 152b ... Required for specifying the evaluation target such as the plant name of the site where the evaluation target is operated are input to the data input devices 149a, 149b. The input image is sent to the site 151 via the transmission means 150. Location 15
In the first embodiment, the image diagnosis apparatus described in the above embodiments is provided, and the image data transmitted from the transmission unit 150 is input, and the luminance noise is removed from the three-dimensional CAD data to perform the image diagnosis. As the image diagnosis method at this time, any of the above-described embodiments may be used.

The sites of these 146a, 146b ...
Since the 51 bases are connected by the transmission means 150, even if the sites are distributed in various places around the world, there is an effect that the bases can perform image diagnosis simply by sending an image from the site via the transmission means. .

Next, an embodiment of the technical support system according to the present invention will be described with reference to FIG.

The embodiment shown in FIG. 19 employs the present invention as a part of a technical support system for providing technical support by connecting an unspecified number of sites and a technical support center for providing technical support to those sites via the Internet. It is a thing.

The technical support system of FIG. 19 has a structure in which the technical support centers 153, 154, and 155 are added to the structure of the image diagnosis system of FIG.

A plurality of sites 146a, 146b ...
The 53 technical support centers are connected by 150 communication means. The 153 technical support centers are 146a, 146b ...
This is a department that receives technical inquiries from the site, delivers the inquiries to the corresponding departments and systems, and returns the answers to the inquiries. Examples of technical inquiries include cause analysis related to abnormal operation and equipment performance evaluation.

In the figure, 151, 154, 155 ... Are systems or departments that receive individual inquiries from the technical support center 153 and create an answer to the inquiries. Of these, 151 is positioned as a system for diagnosing inquiries about the degree of deterioration of parts photographed at the site, and image information input from the sites 146a, 146b ... And additional information 152a, 1 related to the image.
In response to 52b ..., Image data is input from the technical support center 153. The diagnosis system 151 removes luminance noise from the three-dimensional CAD data in the CAD data storage unit 2 and performs image diagnosis. As the image diagnosis method at this time, any of the above-described embodiments may be used. The diagnostic result output from the diagnostic system 151 is the technical support center 1
The diagnosis result is sent to 53 and sent to the sites 146a, 146b, ... Inquired by the technical support center 153.

By adopting this diagnostic system as a part of the technical support system for answering inquiries from the site via the Internet, efficient service can be provided. It is also possible to carry out solutions for unspecified large numbers of sites in parallel.

The above are the embodiments of the image processing apparatus and system according to the present invention. In these embodiments, the gas turbine rotor blade has been described as an example, but the subject of the evaluation is applicable to those that perform physical diagnosis from the surface condition such as color. For example, it can be used in remote diagnosis of the skin condition of a patient in the medical field, and also in the case of attaching an object such as internal organs that cannot be seen without an endoscope to a three-dimensional shape for evaluation. However, the number of objects such as the human body is 3
Since it does not have three-dimensional CAD information in advance, it is necessary to use three-dimensional measuring means such as laser measurement and CT scan together to create three-dimensional CAD data.

According to the above, the surface color of the material can be analyzed and the physical characteristics such as the operating temperature can be estimated only by photographing the image of the target material on site. As a result, there is an effect that the condition diagnosis can be performed only by sending the image information even if the inspector does not go to the office directly unlike the conventional case.

[0121]

According to the present invention, it is possible to provide a processing device for processing information on the surface condition of a target by using the information on the three-dimensional shape of the target.

Further, according to the present invention, an image diagnostic apparatus, an image diagnostic system, or an image diagnostic apparatus capable of accurately extracting a surface color distribution from an image of a subject to be diagnosed and diagnosing the target state from the extracted color distribution. A method can be provided.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a first embodiment of an image diagnostic apparatus according to the present invention.

FIG. 2 is a diagram illustrating an example of a flow of a luminance noise removal processing unit.

FIG. 3 is a diagram for explaining an example of a method for calculating the surface color of an object from the position of a three-dimensional object and the position of illumination.

FIG. 4 is a block diagram showing a configuration of a second embodiment of the image diagnostic apparatus according to the present invention.

FIG. 5 is a diagram for explaining an example of the processing flow of the first embodiment of the material deterioration diagnosis unit.

FIG. 6 is an example of a storage format of a diagnostic target feature quantity storage unit.

FIG. 7 is a diagram showing an example of a method of dividing a surface region of a moving blade.

FIG. 8 is a diagram illustrating a processing flow of a second embodiment of the material deterioration diagnosis unit.

FIG. 9 is a diagram showing an example of a storage format of a material surface color data storage means.

FIG. 10 is a diagram for explaining the processing flow of processing 73 of FIG. 7 in more detail.

FIG. 11 is a view exemplifying an image of an image of a moving blade of a turbine.

FIG. 12 is a diagram showing an example of a result of ranking the colors on the surface of the turbine for each pixel.

FIG. 13 is a diagram for explaining an example of a method of conversely estimating the position of the illumination from the edge part of the shadow candidate region and the target three-dimensional shape coordinates.

FIG. 14 is a block diagram showing a configuration of a third exemplary embodiment of the present invention.

FIG. 15 is a block diagram showing a configuration of a fourth embodiment of the present invention.

FIG. 16 is a block diagram showing a configuration of a fifth embodiment of the present invention.

FIG. 17 is a diagram for explaining an example of the influence of illumination.

FIG. 18 is a diagram showing an example of a configuration of an embodiment of an image diagnostic system according to the present invention.

FIG. 19 is a diagram showing an example of a configuration of an embodiment of a technical support system according to the present invention.

[Explanation of symbols]

1 ... Image data storage unit, 2 ... CAD data storage unit, 3 ...
Luminance noise removal processing unit, 4 ... Three-dimensional image display unit, 5 ... Material degradation diagnosis unit, 6 ... Degradation diagnosis result data, 7 ... Individual number collation unit, 8 ... Past image data, 9 ... Illumination influence removal processing unit.

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) A61B 5/107 G01J 3/50 5L096 G01J 3/50 G06T 3/00 300 G06T 3/00 300 7/00 100C 7/00 100 17/40 A 17/40 A61B 5/10 300Q (72) Inventor Masatoshi Takada 7-1, 1-1 Omika-cho, Hitachi-shi, Ibaraki Hitachi Ltd. Hitachi Research Laboratory (72) Inventor Kojima Keikai 7-1-1, Omika-cho, Hitachi-shi, Ibaraki F-term in Hitachi Research Laboratory, Hitachi, Ltd. (reference) 2G020 AA08 DA02 DA03 DA04 DA22 DA31 DA34 DA35 DA52 DA66 4C038 VB22 VC05 4C061 HH51 NN10 SS21 5B050 AA04 BA06 BA09 BA13 CA05 CA08 DA04 DA07 EA04 EA06 EA09 EA15 EA19 EA26 FA02 FA06 5B057 AA01 CA01 CA08 CA13 CB01 CB08 CB13 CE02 CE08 CE16 DA08 DA16 DB03 DB06 DB09 DC 22 DC25 5L096 AA02 AA06 AA09 BA03 CA02 DA01 EA05 FA06 FA14 FA15

Claims (34)

[Claims] 1. A removal processing unit that removes the influence on the information of the surface state of the target from the information of the three-dimensional shape of the target, and a processing unit that superimposes the information of the surface state of the target on the information of the three-dimensional shape. A processing device having. 2. An image processing apparatus for diagnosing the condition of a diagnosis target from image information of the surface condition of the diagnosis target, wherein the surface condition of the diagnosis target is determined from the image information of the surface condition of the diagnosis target and the three-dimensional shape information of the diagnosis target. An image processing apparatus comprising: a removal processing unit that removes the influence on the three-dimensional shape; and a processing unit that superimposes an image of a diagnostic target on the three-dimensional shape. 3. An image diagnostic apparatus for diagnosing a condition of a diagnosis target from a surface condition, a removal processing unit for removing an influence on a surface condition from a three-dimensional shape of the diagnosis target, and a diagnosis target on the three-dimensional shape. An image diagnostic apparatus comprising: a display unit for displaying an image obtained by pasting the captured image. 4. A processing device for diagnosing a condition of a diagnosis target from a surface condition, comprising: a removal processing unit for removing an influence on a surface condition from a three-dimensional shape of the diagnosis target; and a diagnosis unit for estimating the condition of the diagnosis target. A processing device having. 5. An image processing apparatus for diagnosing a condition of a diagnosis target from a surface condition, a removal processing unit for removing an influence on a surface condition from a three-dimensional shape of the diagnosis target, and a diagnosis unit for estimating the condition of the diagnosis target. An image processing apparatus comprising: 6. An image diagnostic apparatus for diagnosing a condition of a diagnosis target from a surface condition, a removal processing unit for removing an influence on a surface condition from a three-dimensional shape of the diagnosis target, and a diagnosis unit for estimating the condition of the diagnosis target. An image diagnostic apparatus comprising: 7. The removal processing unit according to claim 1 or 4, which removes the influence on the surface state from the three-dimensional shape of the diagnostic object for each of the images obtained by photographing the diagnostic object from a plurality of different angles. A processing device characterized by the above. 8. A processing apparatus according to claim 1, claim 4 or claim 7, further comprising a diagnostic section for diagnosing a physical state of an object from a color. 9. The image processing apparatus according to claim 2 or 5, further comprising a diagnostic section for diagnosing a physical state of an object from a color. 10. An image diagnostic apparatus according to claim 3 or 6, further comprising a diagnostic section for diagnosing a physical state of an object from a color. 11. The method according to claim 1, claim 4, claim 7, or claim 8, wherein the three-dimensional shape of the diagnosis target, the position of the illumination of the environment in which the image is captured, and the intensity of the illumination affect the surface of the target. A processing device comprising: an effect removal processing unit that calculates the effect of light and separates the effect of the brightness of light from the color of the object from the color on the image. 12. The effect of light on the surface of an object according to claim 2, claim 5, or claim 9, based on the three-dimensional shape of the object to be diagnosed, the position of the environment illumination where the image is taken, and the intensity of the illumination. An image processing apparatus comprising: an effect removal processing unit that calculates and separates the effect of the brightness of light from the color of a target from the color on the image. 13. The effect of light on the surface of an object according to claim 3, claim 6, or claim 10, based on the three-dimensional shape of the object to be diagnosed, the position of the illumination of the environment where the image is captured, and the intensity of the illumination. An image diagnostic apparatus comprising: an influence removal processing unit that calculates and separates the influence of the brightness of light and the color of a target from the colors on an image. 14. In claim 1, claim 4, claim 7, claim 8 or claim 11, when the position and intensity of the illumination of the environment where the image is captured are unknown,
A diagnostic unit that extracts a clear color boundary from a color distribution on an image of an object, sets a shadow position candidate area from the extracted color boundary, and estimates the position and intensity of illumination from the candidate area. A processing device comprising: 15. In claim 1, claim 4, claim 7, claim 8, claim 11 or claim 14, the object is photographed outdoors, and the position of the sun at the time of photographing is taken at the photographing time,
A processing device having a diagnostic unit that is obtained from the latitude and longitude of a shooting point. 16. In Claim 4, Claim 7, Claim 8, Claim 11, Claim 14 or Claim 15, the surface of the photographed object is divided into regions based on a three-dimensional shape, and each region is divided. A processing device comprising a diagnosis unit that employs a feature amount obtained from an image as a target attribute. 17. In claim 4, claim 7, claim 8, claim 11, claim 14, claim 15 or claim 16, the average of the colors of the image area is used as the feature amount obtained from the image area. A processing device having a diagnostic unit that employs information on any of a value, a variance, a mode value, or a combination thereof. 18. In claim 4, claim 7, claim 8, claim 11, claim 14, claim 15, or claim 16, the surface of the image area is colored as a feature amount obtained from the image area. A processing device including a diagnostic unit that groups the areas close to each other and calculates the surface area of the areas together with the color information of each area as the feature amount of the object. 19. Claim 4, claim 7, claim 8, claim 11, claim 14, claim 15, claim 16, claim 1.
In Claim 7 or 18, an image capturing section that captures the individual number so that the individual number can be read from the image when capturing the object, the individual number displayed on the surface of the component to be diagnosed, and the three-dimensional CAD data. And a collating unit for referencing three-dimensional CAD data corresponding to an individual number. 20. Claim 4, Claim 7, Claim 8, Claim 11, Claim 14, Claim 15, Claim 16, Claim 1.
In claim 7, claim 18 or claim 19, the image capturing unit for projecting an image into a three-dimensional shape is associated with each pixel on the image and each point on the three-dimensional shape, and A processing device comprising: a contour and a position of the contour in a three-dimensional shape, and a diagnosing unit that uses the difference generated therein as a change in the shape of the material. 21. Claim 4, Claim 7, Claim 8, Claim 11, Claim 14, Claim 15, Claim 16, Claim 1.
7. In claim 18 or claim 19, an imaging unit for imaging the target, a measurement unit for measuring the three-dimensional shape of the target by laser measurement, etc., and a diagnostic unit for using the three-dimensional shape to estimate the surface state. A processing device characterized by the above. 22. Claim 4, claim 7, claim 8, claim 11, claim 14, claim 15, claim 16, claim 1.
7, claim 18, claim 19, claim 20 or claim 2
1, the surface state is estimated by comparing the past image of the evaluation target or a storage unit that accumulates information such as the feature amount obtained from the image with the past information and the information of the newly input image. A processing apparatus having a diagnostic unit for performing the processing. 23. Claim 4, Claim 7, Claim 8, Claim 11, Claim 14, Claim 15, Claim 16, Claim 1.
7, claim 18, claim 19, claim 20 or claim 2
1, the surface state is estimated by comparing the past image of the evaluation target or a storage unit that accumulates information such as the feature amount obtained from the image with the past information and the information of the newly input image. An image processing apparatus having a diagnosis unit for performing the image processing. 24. In claim 3, claim 6, claim 10 or claim 13, a storage unit that stores information such as a past image of an evaluation target or a feature amount obtained from the image, and the past An image diagnostic apparatus having a diagnostic unit for estimating a surface state by comparing information with information of a newly input image. 25. A photographing section for photographing an object to be evaluated at a site where the vehicle is actually driven, a transmission section for transmitting the image information using communication means such as the Internet, and the image information is diagnosed. An image diagnostic system comprising: the image diagnostic apparatus according to claim 6, claim 10, claim 13 or claim 24. 26. An image diagnostic method for diagnosing a condition of a diagnosis target from a surface condition, wherein an influence on a surface condition is removed from a three-dimensional shape of the diagnosis target, and the diagnosis target is imaged on the three-dimensional shape. An image diagnostic method characterized by pasting and displaying the created image. 27. The image diagnostic method according to claim 26, wherein, for each of the images obtained by photographing the diagnosis target from a plurality of different angles, the influence on the surface state is removed from the three-dimensional shape of the diagnosis target. 28. The image diagnostic method according to claim 27, wherein the physical state of the object is diagnosed from the color. 29. The claim 26, claim 27, or claim 2.
In 8, the effect of light on the surface of the target is calculated from the three-dimensional shape of the target to be diagnosed, the position of the illumination of the environment in which the image is captured, and the intensity of the illumination, and the color that the target has from the color on the image, An image diagnostic method characterized by separating the influence of the brightness of light. 30. The image of the object to be evaluated is photographed at the actual driving site, the image information is transmitted by using communication means such as the Internet, and the image information is transmitted based on the transmitted image information.
An image diagnostic method comprising performing diagnosis by the image diagnostic method according to any one of claims 6 to 29. 31. In a technical support system for connecting a plurality of sites and a technical support center by communication means such as the Internet, and answering an inquiry about technical information from the site, the evaluation target is actually operating. An on-site image capturing unit is included, and the technical support center includes the image diagnostic apparatus according to any one of claims 3, 6, 10, 13, and 24. A technical support system characterized in that when an image to be evaluated is sent from a device via a communication means, the image diagnostic apparatus makes a diagnosis and creates an answer. 32. In a technical support method for connecting a plurality of sites to a technical support center through a communication means such as the Internet and answering technical information inquiries from the site, the evaluation target is actually operating. An on-site image capturing unit is included, and the technical support center includes the image diagnostic apparatus according to any one of claims 3, 6, 10, 13, and 24. A method for technical assistance, characterized in that, when an image to be evaluated is sent from a device via a communication means, the image diagnostic apparatus makes a diagnosis and creates an answer. 33. In a maintenance support system that connects a plurality of sites to a technical support center by communication means such as the Internet and responds to technical information inquiries from the site, the evaluation target is actually operating. An imaging unit for imaging on-site is provided, and the image diagnostic apparatus according to any one of claims 4 to 24 is provided in the technical support center, and an image to be evaluated is sent from the site via communication means. A maintenance support system characterized in that the image diagnosis apparatus makes a diagnosis when it is received and creates an answer. 34. In a maintenance support method in which a plurality of sites and a technical support center are connected by a communication means such as the Internet, and a response to a technical information inquiry from the site is answered, the evaluation target is actually operating. An on-site image capturing unit is included, and the technical support center includes the image diagnostic apparatus according to any one of claims 3, 6, 10, 13, and 24. A maintenance support method characterized in that, when an image to be evaluated is sent from a device via a communication means, the image diagnostic apparatus makes a diagnosis and creates an answer.