JP7287138B2 - Image processing method, image processing apparatus and information system - Google Patents

Description

The present invention relates to an image processing method, an image processing apparatus and an information system.

In recent years, in applications provided in information terminals such as smartphones, based on images of measurement targets such as fish and shellfish, insects, animals such as mammals, flowers, and plants such as trees, captured by a camera, these measurement targets are determined. An electronic encyclopedia for specifying types is known.

For example, Patent Literature 1 discloses an invention of an electronic picture book that specifies the type of flower to be measured. In this patent document 1, from an acquired image of a flower, "the color, number, shape, and method of gathering flowers" as the feature amount of the flower, and "the shape of the leaf, the edge of the leaf," as the feature amount of the leaf. The shape of the flower, hairs and thorns on the stem, etc. are extracted, and based on these feature amounts, the type of flower is specified. On the basis of quantity, identification of flower types has been made.

However, in such an electronic encyclopedia, the object to be measured in the photographed image is "similar to the pattern of the background", "is oriented in a direction different from the direction indicating the shape for identifying the flower", or "is There is a problem that recognition becomes difficult and the type of the measurement object cannot be specified.

To solve this problem, as described above, the shape of the object to be measured and the feature values such as the size of this shape are used to identify the type of the object to be measured, such as a flower. If the image is "out of the imaging area", the "shape" cannot be cut out. It is caused by being different from "shape". That is, it is difficult to extract the feature amount based on the shape of the measurement target, which is necessary for identifying the type of the measurement target.

In addition to such an electronic picture book, in a detection method (for example, see Patent Document 2) for determining whether an object to be measured as an inspection object is normal or abnormal, an image of the object to be measured is captured. It has been proposed to determine whether an object to be measured is normal or not based on obtained spectral information (spectrum data).

Therefore, in the electronic picture book, the spectral information obtained by imaging the measurement object is used as information for extracting the feature amount for specifying the type of the measurement object, that is, the unique color tone of the measurement object is used. By equipping the electronic picture book with a spectroscopic camera used as a feature value, even if it is difficult to extract the feature value based on the shape of the object to be measured, the type of the object to be measured can be identified. It is considered possible to

Here, this spectroscopic camera converts the spectroscopic information obtained by superimposing the spectroscopic images acquired in the divided regions obtained by dividing the wavelength region to be measured into the tristimulus values specified by the International Commission on Illumination CIE, that is, , X, Y, Z values. After that, the X, Y, and Z values are converted into R, G, and B values using the monitor profile, and then supplied to the display unit, so that the image of the object to be measured is displayed on the display unit. (See Patent Document 3, for example).

JP 2008-152713 A JP 2017-203658 A JP 2009-033222 A

However, as described above, spectral information is acquired by superimposing a plurality of spectral images, and then the display unit is converted into X, Y, Z values and R, G, B values based on this spectral information. In the image processing method for generating an image to be displayed on the display, it takes time to generate the image, particularly to acquire a plurality of spectral images. Therefore, it is considered that there is a problem that the generated image is shifted due to the user's camera shake or the like.

The present invention has been made to solve the above problems, and can be implemented as the following application examples.

An image processing method according to an application example of the present invention includes an input unit for inputting a start signal for instructing the start of acquisition when acquiring spectroscopic information unique to an object to be measured;
an imaging device that captures an image based on the reflected light reflected from the object; and a spectroscopic unit that selectively emits light having a predetermined wavelength range from the incident light and that can change the wavelength range of the emitted light. a spectrometer having
a control unit having a spectroscopic control unit that controls the operation of the spectroscopic unit; and a spectral image acquisition unit that acquires the spectral information by operating the imaging device;
An image processing method using an image processing device comprising a storage unit that stores the spectral information,
When the spectral information is spectral information obtained by superimposing the acquired spectral images in a specific wavelength region obtained by dividing the measurement wavelength region to be measured,
A first acquisition step of acquiring the spectroscopic image in the predetermined specific wavelength region before inputting the start signal to the input unit, and storing the spectroscopic image acquired before inputting the start signal in the storage unit. and,
after inputting the start signal in the input unit, acquiring the spectral image in the specific wavelength region that has not been acquired before the input of the start signal, and storing the spectral image acquired after the input of the start signal in the storage unit; a second acquiring step of storing in
and a superimposing step of obtaining one piece of spectral information by superimposing the spectral image acquired before the input of the start signal and the spectral image acquired after the input of the start signal. .

1 is a plan view showing the front side of an overall image of a smartphone, which is an information terminal to which the first embodiment of the information system of the present invention is applied; FIG. 1 is a plan view showing the back side of an overall image of a smartphone, which is an information terminal to which the first embodiment of the information system of the present invention is applied; FIG. FIG. 3 is a cross-sectional view of the smart phone shown in FIG. 2 taken along the line AA. FIG. 3 is a cross-sectional view of the smartphone shown in FIG. 2 taken along the line BB. 3 is a block diagram showing a schematic configuration of the smartphone shown in FIGS. 1 and 2; FIG. FIG. 3 is a vertical cross-sectional view showing an example in which the wavelength tunable interference filter included in the spectral measurement unit of the smartphone shown in FIGS. 1 and 2 is applied to a Fabry-Perot etalon filter. FIG. 3 is a flow chart showing a method of specifying a type of a measurement target using the smartphone shown in FIGS. 1 and 2; FIG. FIG. 3 is a schematic diagram for explaining specific spectral information possessed by a measurement target obtained by imaging the measurement target using the smartphone shown in FIGS. 1 and 2 ; FIG. FIG. 3 is a flow chart showing a detection method for detecting a measurement target using the smartphone shown in FIGS. 1 and 2. FIG. FIG. 3 is a flow chart showing an appraisal method for appraising a measurement target using the smartphone shown in FIGS. 1 and 2. FIG. FIG. 3 is a schematic diagram for explaining an image processing method for obtaining one piece of spectral information using spectral images acquired by the smartphone shown in FIGS. 1 and 2 before and after a start signal is input; FIG. 3 is a flowchart showing an image processing method for obtaining one piece of spectral information using spectral images acquired by the smartphone shown in FIGS. 1 and 2 before and after a start signal is input; FIG. It is a block diagram which shows schematic structure of the smart phone and spectrometry part to which 2nd Embodiment of the information system of this invention was applied. FIG. 12 is a block diagram showing a schematic configuration of a smartphone and an external display unit to which the information system of the third embodiment of the invention is applied; FIG. 13 is a block diagram showing a schematic configuration of a smartphone, a spectrometry unit, and an external display unit to which the information system according to the fourth embodiment of the present invention is applied; FIG. 12 is a block diagram showing a schematic configuration of a smartphone and a server to which the fifth embodiment of the information system of the present invention is applied; FIG. 12 is a block diagram showing a schematic configuration of a smartphone and a server to which the sixth embodiment of the information system of the present invention is applied;

An image processing method, an image processing apparatus, and an information system according to the present invention will be described in detail below based on preferred embodiments shown in the accompanying drawings.

Before describing the image processing method and image processing apparatus of the present invention, the information system of the present invention will be described below.

<Information system>
<<First Embodiment>>
FIG. 1 is a plan view showing the front side of an overall image of a smartphone, which is an information terminal to which the first embodiment of the information system of the present invention is applied, and FIG. 3 is a cross-sectional view of the smartphone shown in FIG. 2 along the line AA, FIG. 4 is a cross-sectional view of the smartphone shown in FIG. FIG. 5 is a block diagram showing the schematic configuration of the smartphone shown in FIGS. 1 and 2, and FIG. 6 is a Fabry-Perot etalon in which the wavelength tunable interference filter of the spectroscopic unit of the spectroscopic measurement unit of the smartphone shown in FIGS. 1 and 2 is used. FIG. 7 is a flowchart showing an identification method for identifying the type of measurement target using the smartphone shown in FIGS. 1 and 2; FIG. 8 is a smartphone shown in FIGS. 1 and 2; is a schematic diagram for explaining the spectroscopic information unique to the measurement target obtained by imaging the measurement target, and FIG. 9 is a flowchart showing a detection method for detecting the measurement target by the smartphone shown in FIGS. , FIG. 10 is a flowchart showing an appraisal method for appraising an object to be measured using the smartphones shown in FIGS. 1 and 2, and FIG. A schematic diagram for explaining an image processing method for obtaining one piece of spectral information using an acquired spectral image. FIG. 12 is a schematic diagram obtained by the smartphone shown in FIGS. 3 is a flow chart showing an image processing method for obtaining one piece of spectral information using a spectral image that has been processed.

Hereinafter, in this embodiment, when the information system of the present invention is applied to a smartphone 1 (SP) which is one type of information terminal, the information system of the present invention is completed by the smartphone 1 alone which is an information terminal. I will explain the case where

When the smartphone 1 acquires the spectroscopic information unique to the measurement object X to be measured, the input unit 16 to which a start signal for instructing the start of acquisition is input, and the reflected light reflected from the measurement object X and a spectroscopic measurement unit 10 having an imaging device 21 for capturing an image obtained by capturing an image, and a spectroscopic unit 41 capable of selectively emitting light having a predetermined wavelength range from incident light and changing the wavelength range of the emitted light. , a control unit 60 having a spectral control unit 602 that controls the operation of the spectral unit 41, a spectral image acquisition unit 603 that acquires the spectral information by operating the imaging device 21, and a storage unit 17 that stores the spectral information. , wherein the spectral information is spectral information obtained by superimposing acquired spectral images in specific wavelength regions obtained by dividing the measurement wavelength region to be measured, and the control unit 60 controls the input unit 16 Before the input of the start signal in the spectroscopic measurement unit 10 is controlled to acquire the spectroscopic image in a predetermined specific wavelength region, and after the input of the start signal in the input unit 16, the operation of the spectroscopic measurement unit 10 to acquire a spectral image in a specific wavelength region that has not been acquired before the input of the start signal, and the storage unit 17 stores the spectral images acquired before and after the input of the start signal in the input unit 16. One piece of spectral information is configured to be obtained by superimposing a spectral image acquired before the input of the start signal and a spectral image acquired after the input of the start signal. .

As an application included in the smartphone 1, when an electronic picture book for identifying the types of animals and plants as the measurement target X is activated, the display unit 15 displays the types of the identified animals and plants, as well as detailed information thereof. Information is displayed.

Further, when the application starts a detection method for detecting the presence or absence of an animal, a plant, or the like as the measurement object X in the imaged image, that is, the imaged area, or the position of the existence, the detected image is displayed on the display unit 15. The types of animals, plants, etc., the locations of the animals, plants, etc., as well as the probability of their existence, etc. are displayed. In addition, in FIG. 1, an insect such as a rhinoceros beetle, a stag beetle, or the like, which is the measurement object X, is displayed on the display unit 15 by specifying the position at which it perches on a tree.

Furthermore, as the application, when an appraisal method for appraising the authenticity (authenticity) of articles such as bags, wallets, watches, and jewelry as the measurement object X in the captured image and the degree of deterioration over time is started, the display unit In 15, the authenticity of the identified, i.e., appraised item, the degree of deterioration over time, the rate of authenticity, the location of deterioration over time, and the like are displayed.

Note that the wavelength region to be acquired in order to specify the measurement target X by the smartphone 1 is not limited to one selected from the visible light region, and may be selected from the infrared region, the ultraviolet region, etc. .

When the smartphone 1 is used as an electronic encyclopedia, a detection device, and an appraisal device, each part including the spectroscopic camera, that is, the spectroscopic measurement unit 10 is driven in the smartphone 1, and the configuration of each part will be sequentially described below.

[Display unit 15, input unit 16]
In the smartphone 1, the display 70 has both functions of the display unit 15 and the input unit 16, and the display unit 15 is composed of various display devices such as a liquid crystal display and an organic EL display. As shown in FIG. 1, the display unit 15 is provided on the front side of the smartphone 1 and displays various visualized images including information on the specified measurement target X. FIG. In addition, in the present invention, the display unit 15 and the input unit 16 may be provided separately.

The visualized image displayed on the display unit 15, that is, the information of the specified measurement target X includes, for example, a spectroscopic image of the specified measurement target X, information on the characteristics, classification, components, properties, etc. of the measurement target X, and further is the presence or absence and position of the measurement object X in the imaging area, the recognition accuracy (%) and existence probability (%) of the measurement object X, the authenticity probability (%) of the measurement object X, and the degree of deterioration (% ) and the like.

In the present invention, the image obtained by the spectroscopic camera, that is, the spectroscopic measurement unit 10 is displayed on the display unit 15 as the image of the specified measurement target X. It can be prevented and displayed, the details of which will be described later.

The input unit 16 is provided on the surface of the display unit 15, for example, and is configured by a touch panel including a touch-sensitive surface and a sensor for detecting the strength of contact with the touch-sensitive surface. ), i.e., a condition for acquiring the spectroscopic information unique to the measurement target X, and further, a start signal for instructing the smartphone 1 to start acquiring the spectroscopic information when acquiring the spectroscopic information. by doing so.

[Storage unit 17]
The storage unit 17 includes various storage devices (memory) such as ROM and RAM, and stores various data, programs, and the like necessary for controlling the smartphone 1 , particularly for controlling the spectroscopic measurement unit 10 .

The data indicates, for example, an application, a program, etc. for realizing each function of the control unit 60, and the wavelength of transmitted light with respect to the driving voltage applied to the electrostatic actuator 45 provided in the Fabry-Perot etalon filter of the spectroscopic unit 41. Correlation data V-λ data, a database for specifying the type of the object X to be measured based on the spectroscopic information unique to the object X to be measured, and the like. The database referred to here includes the measurement target X to be specified, including animals such as fish and shellfish, insects, and mammals, plants such as flowers and trees, and articles such as bags, wallets, watches, and jewelry. It shows the spectral information for each. Furthermore, specific spectral information possessed by the measurement object X, etc., which is acquired by the operation of the spectroscopic measurement unit 10 or the like, which will be described later, can be used.

[Spectrometry unit 10]
The spectroscopic measurement unit 10 receives and spectroscopically reflects the light reflected by the measurement object X, thereby obtaining a selected specific wavelength or specific wavelength region (hereinafter, representatively described as a “specific wavelength region”). It is a so-called spectroscopic camera that obtains spectral information as an image by capturing an image of light having a specific wavelength region after obtaining light.

In the present embodiment, the spectroscopic measurement unit 10 includes a light source 31 that irradiates light onto a measurement target X, that is, an imaging target, an imaging device 21 that captures an image based on reflected light reflected from the measurement target X, and an incident light beam. and a spectroscopic section 41 that selectively emits light having a predetermined wavelength range from the light source and that can change the wavelength range of the emitted light.

In such a spectroscopic measurement unit 10, as shown in FIG. 2, the light source 31 and the imaging element 21 are arranged on the back side of the smartphone 1 so as to face the same direction. and the object X to be measured. By arranging the light source 31, the spectroscopic unit 41, and the imaging element 21 in such a positional relationship, the spectroscopic measurement unit 10 can be configured as a post-spectroscopic spectroscopic camera. With such a post-spectroscopic spectroscopic camera, by scanning wavelengths in a certain measurement range (predetermined region), it is possible to acquire a specific wavelength region and spectrum shape and grasp the characteristics of the measurement object X. . Therefore, it is an effective method when measuring, ie, capturing an image of a measurement target X whose specific wavelength region is unknown.

Note that the spectroscopic measurement unit 10 may constitute a spectroscopic camera of a pre-spectroscopic method in which the spectroscopic unit 41 is arranged between the light source 31 and the object X to be measured. The spectroscopic camera of the pre-spectroscopic method having such a configuration is a method capable of ascertaining the characteristics of the object X to be measured by irradiating it with light in a specific wavelength region. Therefore, it is an effective method when measuring a measurement object X whose specific wavelength region is known, and has the advantage of reducing the amount of information compared to the post-spectroscopy method, so that the measurement time can be shortened. method.

The configuration of each part included in the spectroscopic measurement unit 10 will be described below.
[Light source 31]
The light source 31 is an optical element that irradiates illumination light toward the object X to be measured.

As shown in FIGS. 2 and 4, the light source 31 is arranged on the back side of the smartphone 1 so as to irradiate illumination light toward the measurement target X on the circuit board 51 arranged inside the housing of the smartphone 1. It is

No spectroscopic unit is arranged between the light source 31 and the object X to be measured, so that the object X to be measured is directly irradiated with the light emitted from the light source 31 .

Such a light source 31 includes, for example, an LED light source, an OLED light source, a xenon lamp, a halogen lamp, and the like. A light source capable of emitting white light having light intensity over the entire visible light region is preferably used. Moreover, the light source 31 may be provided with a light source capable of irradiating light of a predetermined wavelength such as infrared light, for example, other than the white light source.

[Image sensor 21]
The imaging device 21 functions as a detection unit that detects the reflected light reflected from the measurement object X by capturing an image based on the reflected light reflected from the measurement object X. FIG.

As shown in FIGS. 2 and 3, the imaging element 21 is mounted on the back side of the smartphone 1 so as to receive reflected light reflected from the measurement target X on a circuit board 51 arranged inside the housing of the smartphone 1. are placed.

A spectroscopic section 41 is arranged between the imaging device 21 and the object X to be measured. As a result, emitted light having a specific wavelength region is selectively emitted from the incident light incident on the spectroscopic unit 41 from the measurement target X, and the emitted light is captured as a spectral image by the imaging device 21 .
Such an imaging element 21 is configured by, for example, a CCD, a CMOS, or the like.

[Spectroscopic part 41]
The spectroscopic section 41 selectively emits light in a spectroscopic wavelength region, which is a specific wavelength region, from incident light, and can change the specific wavelength region of emitted light. That is, light in a predetermined specific wavelength region is emitted from the incident light toward the imaging device 21 as emitted light.

As shown in FIG. 3 , the spectroscopic section 41 is arranged on a circuit board 52 arranged inside the housing of the smartphone 1 .

The spectroscopic section 41 is arranged between the imaging element 21 and the object X to be measured, that is, on the optical axis between them. As a result, out of the incident light that has entered the spectroscopic section 41 after being reflected from the object X to be measured, the outgoing light having the specific wavelength region is selectively emitted toward the imaging device 21 .

Such a spectroscopic section 41 is composed of a variable wavelength interference filter so as to be able to change the specific wavelength region of emitted light. The wavelength tunable interference filter is not particularly limited, but for example, a wavelength tunable interference filter that controls the wavelength of reflected light to be transmitted by adjusting the size of the gap between two filters (mirrors) with an electrostatic actuator. A Fabry-Perot etalon filter, an acousto-optic tunable filter (AOTF), a linear variable filter (LVF), a liquid crystal tunable filter (LCTF) and the like can be used, and among them, a Fabry-Perot etalon filter is preferable.

A Fabry-Perot etalon filter extracts reflected light of a desired wavelength by utilizing multiple interference between two filters. Therefore, the thickness dimension can be made extremely small, and specifically, it becomes possible to set it to 2.0 mm or less. Therefore, the smartphone 1 including the spectroscopic unit 41 and thus the spectroscopic measurement unit 10 can be made smaller. Therefore, by using the Fabry-Perot etalon filter as the wavelength tunable filter, the spectrometer 10 can be further miniaturized.

The spectroscopic section 41 to which a wavelength-tunable Fabry-Perot etalon filter is applied as a wavelength-tunable interference filter will be described below with reference to FIG.

The Fabry-Perot etalon filter is a rectangular plate-shaped optical member in plan view, and includes a fixed substrate 410, a movable substrate 420, a fixed reflective film 411, a movable reflective film 421, a fixed electrode 412, a movable electrode 422, and a bonding film 414 . The fixed substrate 410 and the movable substrate 420 are laminated and integrally bonded via a bonding film 414 .

The fixed substrate 410 has a groove 413 formed by etching in the thickness direction so as to surround the central portion so that a reflective film installation portion 415 is formed in the central portion. In the fixed substrate 410 having such a configuration, a fixed optical mirror composed of the fixed reflective film 411 is provided on the movable substrate 420 side of the reflective film installation portion 415 , and a fixed electrode 412 is provided on the movable substrate 420 side of the groove 413 . .

In addition, the movable substrate 420 has a groove 423, which is a holding portion, formed by etching in the thickness direction, surrounding the central portion so that a movable portion, which is the reflective film installation portion 425, is formed in the central portion. . In the movable substrate 420 having such a configuration, a movable optical mirror composed of the movable reflective film 421 is provided on the fixed substrate 410 side, i.e., the lower surface side of the reflective film installation portion 425, and a movable electrode 422 is provided on the fixed substrate 410 side. .

The movable substrate 420 is formed such that the thickness of the groove 423 is smaller than that of the reflective film installation portion 425 . It functions as a diaphragm that bends due to electrostatic attraction.

The fixed substrate 410 and the movable substrate 420 can be manufactured as long as they have a thickness of about 0.1 mm or more and 1.0 mm or less. Therefore, since the thickness of the Fabry-Perot etalon filter as a whole can be set to 2.0 mm or less, the size reduction of the spectrometer 10 can be realized.

Between the fixed substrate 410 and the movable substrate 420, the fixed reflective film 411 and the movable reflective film 421 are arranged opposite to each other with a gap at approximately the center of the fixed substrate 410 and the movable substrate 420, respectively. The fixed electrode 412 and the movable electrode 422 are opposed to each other with a gap in the groove surrounding the central portion. Among them, the fixed electrode 412 and the movable electrode 422 constitute an electrostatic actuator 45 that adjusts the size of the gap between the fixed reflective film 411 and the movable reflective film 421 .

Due to the electrostatic attraction generated by applying a voltage between the fixed electrode 412 and the movable electrode 422 that constitute the electrostatic actuator 45 , the groove 423 , which is the holding portion, is bent. As a result, the size of the gap, that is, the distance between the fixed reflective film 411 and the movable reflective film 421 can be changed. By appropriately setting the size of this gap, it is possible to select the wavelength region of light to be transmitted, that is, to selectively emit light in a desired wavelength region from the incident light. Further, by changing the configurations of the fixed reflecting film 411 and the movable reflecting film 421, it is possible to control the half width of the transmitted light, that is, the resolution of the Fabry-Perot etalon filter.

Note that the fixed substrate 410 and the movable substrate 420 are each made of, for example, various glasses such as soda glass, crystalline glass, quartz glass, lead glass, potassium glass, borosilicate glass, alkali-free glass, crystal, etc. The bonding film 414 is composed of, for example, a plasma-polymerized film containing siloxane as a main material, and the fixed reflecting film 411 and the movable reflecting film 421 are composed of, for example, a metal film such as Ag or an alloy film such as Ag alloy. In addition, it is composed of a dielectric multilayer film including TiO ₂ as a high refractive layer and SiO ₂ as a low refractive layer, and the fixed electrode 412 and the movable electrode 422 are composed of various conductive materials.

[Optical systems 81, 83]
Further, in this embodiment, the spectroscopic measurement unit 10 is configured to have optical systems 81 and 83 configured by various optical components, as shown in FIG.

The first spectroscopic unit side optical system 81 is arranged between the measurement object X and the spectroscopic unit 41, and includes an incident lens 811 as an incident optical system and a projection lens 812, and reflects light reflected from the measurement object X. It is guided to the spectroscopic section 41 .

Also, the first imaging element side optical system 83 is arranged between the spectroscopic section 41 and the imaging element 21 , has an input/output lens 831 , and guides the emitted light emitted by the spectroscopic section 41 to the imaging element 21 .

By including at least one of the optical systems 81 and 83 as described above in the spectroscopic measurement unit 10 , it is possible to improve the collection efficiency of the reflected light reflected by the measurement object X by the imaging device 21 .

Note that at least one of the optical systems 81 and 83 may be omitted in consideration of the light collection rate of the imaging device 21 .

In addition to the arrangement as described above (see FIG. 5), the first spectroscopic unit side optical system 81 may be arranged between the spectroscopic unit 41 and the first imaging element side optical system 83. It may be an eggplant.

[Control unit 60]
The control unit 60 is provided in a housing provided in the smartphone 1, and is configured by, for example, a processor in which a CPU, a memory, and the like are combined, and operates each unit such as the light source 31, the imaging device 21, and the spectroscopic unit 41, that is, spectroscopic measurement. It controls the operation of the entire unit 10 or each unit, and also controls the operation of the display unit 15 and the input/output of data to the storage unit 17 . In terms of controlling the operation of the spectroscopic measurement unit 10, the control unit 60 corresponds to or can be said to include a "spectroscopic measurement unit control unit", and controls the operation of the spectroscopic measurement unit 10, that is, the spectroscopic camera. .

More specifically, the control unit 60 stores in the storage unit 17 based on the user's operation instruction input to the input unit 16, that is, the conditions for acquiring the spectroscopic information unique to the measurement target X. The operation of the light source 31, the spectroscopic section 41, and the imaging element 21 is controlled by reading software such as a program. Then, based on the obtained spectral information, for example, the imaged measurement object X is specified, and the display unit 15 displays information such as the type and characteristics of the object and the presence/absence of the object in the imaging area.

In this embodiment, the control unit 60 includes a light source control unit 601, a spectral control unit 602, a spectral image acquisition unit 603, an analysis processing unit 604, and a display control unit 605, as shown in FIG. .

The light source control unit 601 controls the lighting and extinguishing of the light source 31 based on the user's operation instruction input to the input unit 16, specifically, the conditions for acquiring the spectroscopic information unique to the measurement target X. It is something to do.

Based on the V-λ data stored in the storage unit 17, the spectral control unit 602 acquires the voltage value (input value) of the drive voltage corresponding to the spectral wavelength region of emitted light, that is, the specific wavelength region. Then, a command signal is output to apply the acquired voltage value to the electrostatic actuator 45 of the Fabry-Perot etalon filter as the spectroscopic unit 41 . That is, the spectroscopic control section 602 controls the operation of the spectroscopic section 41 to specify the size of the specific wavelength region of the light emitted from the spectroscopic section 41 . Based on various data stored in the storage unit 17, the spectroscopic control unit 602 detects the change timing of the measurement wavelength, changes the measurement wavelength, changes the driving voltage according to the change of the measurement wavelength, and ends the measurement. is determined, and a command signal is output based on the determination.

The spectral image acquisition unit 603 acquires (pictures) the light intensity measurement data (the amount of received light) based on the reflected light reflected by the measurement object X as a spectral image by operating the imaging element 21, and then acquires (images) the acquired spectral image. is stored in the storage unit 17. Note that when storing the spectral image in the storage unit 17, the spectral image acquisition unit 603 stores the measurement wavelength region in which the spectral image was acquired, that is, the specific wavelength region, in the storage unit 17 together.

The analysis processing unit 604 acquires the spectral spectrum as spectral information based on the spectral image of the measurement object X and the measurement wavelength region, that is, the specific wavelength region, and performs analysis processing on them. That is, by performing spectroscopic processing for comparing the spectroscopic spectrum as spectroscopic information with the database stored in the storage unit 17, the imaged measuring object X is specified.

However, acquisition of the spectral image and the measurement wavelength region by the analysis processing unit 604 can also be performed directly from the spectral image acquisition unit 603 without going through the storage unit 17 .

The display control unit 605 causes the display unit 15 to display the image, information, and the like of the measurement object X specified by the analysis processing unit 604 as a visualized image.

In the smartphone 1 having such a configuration, the input unit 16, the spectroscopic measurement unit 10, the control unit 60, and the storage unit 17 constitute the image processing device of the present invention.

In the smartphone 1 as described above, by selecting the type of application to be activated, i.e., by selecting the method of using the smartphone 1, the smartphone 1 can: 2) as a detection device for detecting the presence or absence of animals and plants as the measurement object X in the captured image and the position where they exist; and 3) measurement in the captured image. It can be used as an appraisal device for appraising the authenticity (authenticity) of an article as an object X and the degree of deterioration over time. A method of using the smartphone 1 as a device for implementing 1) to 3) will be described below.

[1) How to use as an electronic picture book]
Hereinafter, a method for specifying the types of animals, plants, and the like as measurement targets X using the smartphone 1 as an electronic picture book will be described in detail below with reference to FIG. 7 and the like.

In the identification method using the smartphone 1 as an electronic picture book, the measurement target X is imaged using the spectroscopic measurement unit 10, and the measurement target X is identified based on the spectral information obtained using the captured spectral image. I do. After that, the image of the specified measurement target X, its type, its detailed description, etc. are displayed on the display 70 .

<1A> First, the user activates an application that uses the smartphone 1 as an electronic picture book by operating the input unit 16, and then selects conditions and the like as necessary according to the instructions of the application (S1A).

The conditions to be input according to the instructions of the application include, for example, the classification of the measurement object X such as flowers, fish, mammals, or the like. By inputting the classification of the measurement object X in advance in this manner, the smartphone 1 can quickly detect the measurement object X, the details of which will be described later.

<2A> Next, the user operates the input unit 16 to give an input instruction to capture an image of the measurement object X using the smartphone 1, that is, the spectroscopic measurement unit 10. Based on the input of the start signal accompanying this input instruction, the control unit 60 obtains unique spectral information of the measurement target X by controlling the operation of the spectroscopic measurement unit 10, that is, the spectroscopic camera, and capturing an image of the measurement target X. FIG.

<2A-1> First, the light source control unit 601 turns on the light source 31 according to the input of the start signal accompanying the user's input instruction to image the measurement object X in the input unit 16 (S2A).

When the light source 31 is turned on, the object X to be measured is irradiated with the illumination light emitted from the light source 31 . The irradiated light is reflected by the object X to be measured, and the reflected light enters the spectroscopic section 41 as incident light.

<2A-2> Next, based on the V-λ data stored in the storage unit 17, the spectral control unit 602 controls the voltage value (input value) of the driving voltage corresponding to the spectral wavelength region to be emitted, that is, the specific wavelength region. to get Then, a command signal is output to apply the acquired voltage value to the electrostatic actuator 45 of the Fabry-Perot etalon filter as the spectroscopic unit 41 (S3A).

As a result, of the light that has entered the spectroscopic section 41 from the object X to be measured as incident light, light having a specific wavelength region is selectively emitted toward the imaging element 21 side as emitted light.

It is preferable that the spectroscopic control unit 602 performs an adjustment process for calibrating the spectroscopic unit 41 before causing the spectroscopic unit 41 to emit light having a specific wavelength region. Thereby, the spectral spectrum s _ref of the light source 31 is obtained.

<2A-3> Next, the spectral image acquisition unit 603 controls the operation of the imaging device 21, so that the light having the specific wavelength region emitted as the emitted light from the spectroscopic unit 41 is captured as a spectral image by the imaging device. 21. That is, the imaging element 21 acquires the light amount measurement data (the amount of received light) of light having a specific wavelength region among the reflected light reflected from the measurement object X as a spectral image. Then, the spectral image acquisition unit 603 causes the storage unit 17 to store the acquired spectral image together with the measurement wavelength region, ie, the specific wavelength region corresponding to the spectral image (S4A).

In such a spectroscopic image acquisition method, the spectroscopic section 41 is arranged on the optical axis of the light received by the imaging device 21 between the measurement object X and the imaging device 21 . As a result, only the light in the specific wavelength region included in the light reflected by the measurement object X is transmitted by the spectroscopic unit 41, and the intensity of the light in this specific wavelength region is spectrally measured by the imaging device 21 as a spectral image.

<2A-4> Next, after acquiring a spectral image with light having a specific wavelength region for the first time, is it necessary to acquire a spectral image for a second time with light having a specific wavelength region different from the first specific wavelength region? Whether or not is determined based on the conditions selected by the user in step <1A>. That is, it is determined whether or not it is necessary to continuously acquire spectral images of light having a second specific wavelength region different from the first specific wavelength region (S5A).

In this determination (S5A), if it is necessary to acquire a spectral image with light having the second specific wavelength region, instead of the light having the first specific wavelength region, the light having the second specific wavelength region , the steps <2A-2> to <2A-4> are repeated. That is, after changing the voltage value applied between the fixed electrode 412 and the movable electrode 422 of the electrostatic actuator 45 to set the second specific wavelength region, the steps <2A-2> to the present step <2A- 4> is repeated. As a result, a second spectroscopic image of light having the specific wavelength region is acquired. Such a second specific wavelength region, that is, acquisition of spectral images in light having a different specific wavelength region is repeated 1st, 2nd to n times. As described above, the steps <2A-2> to this step <2A-4> are repeated, and in the 1st, 2nd to n-th times, the spectrum obtained in the light having each specific wavelength region obtained respectively Overlay the image. Thereby, the spectral information can be obtained as the spectral information s _sam indicating the relationship between each specific wavelength region and the light intensity corresponding to each pixel included in the spectral image.

That is, in FIG. 8, the spectrum information s _sam is a graph showing the relationship between each specific wavelength region divided into n by a uniform wavelength width in the measurement wavelength region of 400 nm or more and 760 nm or less, which is the visible light region, and the light intensity. , and this graph, that is, spectral information s _sam is obtained as spectral information possessed by each pixel of M rows×N columns.

Note that the number n of repetitions of steps <2A-2> to step <2A-4>, that is, the number n of specific wavelength regions obtained by dividing the measurement wavelength region into equal widths is not particularly limited. , preferably 10 or more and 20 or less, more preferably 12. Thereby, spectral information s _sam can be obtained with excellent accuracy.

On the other hand, if it is not necessary to acquire a spectral image of light having the next wavelength, acquisition of a spectral image by the spectroscopic measurement unit 10 is terminated, and the process proceeds to the next step <3A>.

<3A> Next, the analysis processing unit 604 generates a spectral image based on each spectral image of the measurement target X and the specific wavelength region corresponding thereto, that is, based on the spectral information s _sam stored in the storage unit 17. is analyzed (S6A).

In other words, the analysis processing unit 604 acquires the spectral information _{s_sam} stored in the storage unit 17 in step <2A>. After that, the spectral information s _sam as the spectral information is used as a feature amount, and analysis processing is performed for comparison with a database, thereby identifying the imaged measuring object X. FIG.

Specifically, the reflectance r=s _sam /s _ref of the measurement object X is calculated from the spectrum information s _sam and the spectral spectrum s _ref of the light source 31 .

Then, the analysis ^processing unit 604 acquires the data r ⁱ included in those corresponding to the group i=1, . The object X to be measured is determined by determining whether it belongs to which group i. Here, the “group” refers to the classification or subclassification of flowers, fish, mammals, etc. to which the measurement object X belongs. Corresponding groups of data are obtained.

More specifically, first, the spectral information s _sam as a feature is projected onto a discriminant space suitable for discriminating which group it belongs to, that is, a projection function f(·) is generated based on a specific discriminant criterion. do. Note that the discrimination criteria include, for example, the Fisher discrimination criteria and the least squares criteria. Then, the reflectance r of the object X to be measured is projected onto the discriminant space to be y.

y = f(r)

^Similarly , the data r ⁱ of the group i=1, . Then, the distances m ⁱ (i=1, . . . , M) in the discriminant space between the position y of the measurement object X in the discriminant space and the group i=1, .

m ⁱ (i=1,…,M)=g(y,y(r ⁱ ))

where y(r ⁱ ) is the set of positions on the discriminant space of the data belonging to group i, that is, y(r ⁱ )={y(r ⁱ 1),...,y(r ⁱ _N )} ( In the formula, N represents the number of data belonging to group i), and g(a,b) is a function for calculating the distance between a and b in the discriminant space. As the distance, for example, Mahalanobis distance, Euclidean distance, or the like can be used. Among m ⁱ (i=1, .

H=arg _i min m ⁱ

As described above, in this step <3A>, the spectral information s _sam , that is, the spectral information is used as the feature quantity for specifying the measurement target X, and the measurement target X is specified based on the shape of this spectral information s _sam . Therefore, when "the measurement object X resembles the pattern of the background", for example, "Matsutake" in the withered pine needles, "Caterpillar" on the leaf, "Flatfish and flounder" on the sandy beach. , a "stag beetle" or a "beetle" on a branch of a tree can be specified as the measurement object X, even if the measurement object and the background have similar colors, they can be accurately specified. Furthermore, even when "the object X to be measured faces the front or back in the imaging area" and even when "the object X to be measured protrudes from the imaging area", the shape information is not used as the feature amount. can be accurately determined from

<4A> Next, the display control unit 605 creates a visualized image of information on the measurement object X specified by the analysis processing unit 604, and then causes the display 70 including the display unit 15 to display this visualized image (S7A ).

As the information of the measurement object X displayed on the display 70 as this visualized image, when the measurement object X is an animal such as a fish, an insect, or a mammal, or a plant such as a flower or a tree, the specified measurement In addition to the type of the target X, that is, the group H to which the measurement target X belongs, detailed information such as the classification, distribution, morphology, ecology, etc. of the measurement target X can be mentioned.

Further, when the spectral information obtained in step <2A>, that is, the spectral information s _sam is displayed as a visualized image on the display 70, that is, the display unit 15, this spectral information is Convert to tristimulus values, ie, X, Y, Z values, then convert the X, Y, Z values to R, G, B values using the monitor profile, and display these on the display unit 15 By supplying the spectral information, it is possible to display the spectral information as a visualized image.

Information stored in the storage unit 17 is displayed on the display 70, but it is also possible to display information disclosed on the Internet on the display 70 using the communication function of the smartphone 1. .

Through steps <1A> to <4A> using the smartphone 1 as an electronic picture book as described above, the measurement target X is specified.

Note that the spectral processing for comparing the spectral spectrum as spectral information with the database stored in the storage unit 17 is performed using the distances m ⁱ (i=1, . . . , M) in the discriminant space. Although the case has been described, it is not limited to this, and the analysis processing can also be performed by machine learning such as a neural network.

In addition, the identification of the measurement target X using the smartphone 1 as an electronic picture book can be applied to the identification of animals or plants as described above. , constellations, etc. can also be applied.

[2) Usage as a detection device]
Hereinafter, a detection method for detecting the presence or absence of an animal, a plant, or the like as a measurement target X whose presence is to be specified, and the position of the presence using the smartphone 1 described above as a detection device will be described below using FIG. 9 and the like. for details.

In the detection method using the smartphone 1 as a detection device, an area in which the measurement target X, that is, the detection target (target) is presumed to exist is imaged using the spectroscopic measurement unit 10, and based on the captured spectral image, the image is captured. The presence/absence of the object X to be measured, the position of the object to be measured, the probability of existence (%), and the like are specified in the captured image area. After that, the specified contents are displayed on the display 70 .

<1B> First, the user activates an application that uses the smartphone 1 as a detection device by operating the input unit 16, and then selects conditions and the like according to instructions from the application (S1B).

Note that the conditions to be input according to the instructions of the application include the type of the measurement object X to be detected in the captured image. By inputting the type of the measurement object X to be detected in this way, the measurement object X can be quickly detected in the imaged imaging region, the details of which will be described later.

<2B> Next, the user operates the input unit 16 to issue an input instruction to capture an image of a region in which the measurement target X is desired to be detected by the smartphone 1 , that is, the spectroscopic measurement unit 10 . Then, based on this input instruction, the control unit 60 controls the operation of the spectroscopic measurement unit 10, that is, the spectroscopic camera, and performs imaging of an imaging region where the measurement target X is desired to be detected.

<2B-1> First, the light source control unit 601 turns on the light source 31 in accordance with the user's input instruction for imaging from the input unit 16 (S2B).

When the light source 31 is turned on, the illumination light emitted from the light source 31 is applied to the imaging region where the measurement target X is desired to be detected. Then, the irradiated light is reflected in the imaging area, and the reflected light enters the spectroscopic section 41 as incident light.

<2B-2> Next, based on the V-λ data stored in the storage unit 17, the spectral control unit 602 controls the voltage value (input value) of the driving voltage corresponding to the spectral wavelength region to be emitted, that is, the specific wavelength region. to get Then, a command signal is output to apply the acquired voltage value to the electrostatic actuator 45 of the Fabry-Perot etalon filter as the spectroscopic unit 41 (S3B).

As a result, of the light that has entered the spectroscopic section 41 from the object X to be measured as incident light, light having a specific wavelength region is selectively emitted toward the imaging element 21 side as emitted light.

It is preferable that the spectroscopic control unit 602 performs an adjustment process for calibrating the spectroscopic unit 41 before causing the spectroscopic unit 41 to emit light having a specific wavelength region. Thereby, the spectral spectrum s _ref of the light source 31 is obtained.

<2B-3> Next, the spectral image acquisition unit 603 controls the operation of the imaging device 21, so that the light having the specific wavelength region emitted as the emitted light from the spectroscopic unit 41 is captured as a spectral image by the imaging device. 21. That is, the imaging element 21 acquires the light amount measurement data (the amount of received light) of light having a specific wavelength region among the reflected light reflected from the measurement object X as a spectral image. Then, the spectral image acquisition unit 603 causes the storage unit 17 to store the acquired spectral image together with the measurement wavelength region, ie, the specific wavelength region corresponding to the spectral image (S4B).

In such a spectroscopic image acquisition method, the spectroscopic section 41 is arranged on the optical axis of the light received by the imaging device 21 between the imaging region where the object X to be measured is to be detected and the imaging device 21 . As a result, only the light in the specific wavelength region included in the light reflected by the imaging region is transmitted by the spectroscopic unit 41, and the intensity of the light in this specific wavelength region is spectrally measured by the imaging device 21 as a spectral image.

<2B-4> Next, after acquiring a spectral image with light having a specific wavelength region for the first time, is it necessary to acquire a spectral image for a second time with light having a specific wavelength region different from the first specific wavelength region? Whether or not is determined based on the conditions selected by the user in step <1B>. That is, it is determined whether or not it is necessary to continuously acquire spectral images of light having a second specific wavelength region different from the first specific wavelength region (S5B).

In this determination (S5B), if it is necessary to acquire a spectral image with light having the second specific wavelength region, instead of the light having the first specific wavelength region, the light having the second specific wavelength region , the steps <2B-2> to <2B-4> are repeated. That is, after changing the magnitude of the voltage applied between the fixed electrode 412 and the movable electrode 422 of the electrostatic actuator 45 to set the second specific wavelength region, the steps <2B-2> to the present step <2B-4> are repeated. As a result, a second spectroscopic image of light having the specific wavelength region is obtained. Such a second specific wavelength region, that is, acquisition of spectral images in light having a different specific wavelength region is repeated 1st, 2nd to n times. As described above, the steps <2B-2> to <2B-4> are repeated. Then, in the 1st, 2nd to n-th times, the spectral images acquired with the light having the respective acquired specific wavelength regions are superimposed. Thereby, the spectral information can be obtained as the spectral information s _sam indicating the relationship between each specific wavelength region and the light intensity corresponding to each pixel in the imaging region where the object X to be measured is to be detected.

That is, in FIG. 8, the spectrum information s _sam is a graph showing the relationship between each specific wavelength region divided into n by a uniform wavelength width in the measurement wavelength region of 400 nm or more and 760 nm or less, which is the visible light region, and the light intensity. , and this graph, that is, spectral information s _sam is obtained as spectral information possessed by each pixel of M rows×N columns.

Note that the number n of times the steps <2B-2> to the present step <2B-4> are repeated, that is, the number n of specific wavelength regions obtained by dividing the measurement wavelength region into equal widths is not particularly limited. , preferably 10 or more and 20 or less, more preferably 12. Thereby, spectral information s _sam can be obtained with excellent accuracy.

On the other hand, if it is not necessary to acquire a spectral image of light having the next wavelength, acquisition of the spectral image by the spectroscopic measurement unit 10 is terminated, and the process proceeds to the next step <3B>.

<3B> Next, the analysis processing unit 604 analyzes the spectral images based on each spectral image of the measurement object X and the specific wavelength region corresponding thereto, that is, the spectral information s _sam stored in the storage unit 17. Implement (S6B).

In other words, the analysis processing unit 604 acquires the spectral information _{s_sam} stored in the storage unit 17 in step <2B>. Then, by performing analysis processing using the spectral information s _sam as spectral information as a feature amount, the measurement object X, which is the target object, is detected from the imaging region imaged by the spectroscopic measurement unit 10 .

Specifically, from the spectrum information s _sam and the spectral spectrum s _ref of the light source 31, the reflectance r=s _sam /s _ref in the imaging region where the measurement target X is desired to be detected is calculated. Then, by dividing this imaging region into pixels of M rows and N columns, each region (i=1, . . . , M), (j=1, . Calculate the data r ⁱ _j .

Then, the analysis processing unit 604 acquires the reflectance r _base corresponding to the measurement object X from the database prepared in advance in the storage unit 17, and the reflectance r ⁱ _j corresponding to each of the M divided regions is , the position where the measurement object X exists in the imaging region is specified by determining whether or not it belongs to the reflectance r _base corresponding to the measurement object X.

More specifically, first, a projection function f(·) onto a discriminant space suitable for discriminating each group is generated based on a specific discriminant criterion. Note that the discrimination criteria include, for example, the Fisher discrimination criteria and the least squares criteria. Then, the reflectance r _base of the object X to be measured stored in the database is projected onto the discriminant space to be y.

y=f(r _base )

Similarly, the data r ⁱ _j corresponding to each region (i=1,...,M), (j=1,...,N) divided into M rows and N columns is projected onto the discriminant space, and y( r ⁱ _j ). Then, the distance m ⁱ _j (i=1,...,M),(j =1,…,N).

m ⁱ _j (i=1,...,M),(j=1,...,N)=g(y,y(r ⁱ _j ))

Here, g( ) is a function that calculates the distance in the discriminant space. As the distance, for example, Mahalanobis distance, Euclidean distance, or the like can be used.

Then, when the distance m ⁱ _j among m ⁱ _j (i=1,..., M) and (j=1,..., N) is smaller than a certain threshold value set, the measurement target X exists in that area. That is, it is determined that the measurement target X is detected in the area, and the threshold is further subdivided to specify the existence probability (%) in the area. can be done.

As described above, in step <3B>, the spectral information s _sam, that is, the spectral information is used as the feature quantity for detecting the measurement target X, and based on the shape of this spectral information s _sam , the measurement target X in the imaging region Therefore, when ``the object X to be measured resembles the pattern of the background'', for example, ``Matsutake'' in withered pine needles, ``Caterpillar'' on the leaf, and `` Even if the object to be measured and the background have the same color as in the case of detecting a flatfish or a flounder, or a stag beetle or a beetle on a tree branch as the object to be measured X, accurate detection can be made from the imaging area. can be done. Furthermore, even when "the object X to be measured faces the front or back in the imaging area" and even when "the object X to be measured protrudes from the imaging area", the shape information is not used as the feature amount. , the object X to be measured can be accurately detected.

<4B> Next, the display control unit 605 causes the analysis processing unit 604 to display, for example, a red Create a visualized image highlighted by marking such as . After that, as shown in FIG. 1, this visualized image is displayed on the display 70 having the display unit 15 (S7B). However, in FIG. 1, the position where an insect such as a rhinoceros beetle or a stag beetle as the measurement object X is perched on a tree is specified by marking and emphasizing it on the display unit 15 .

Further, when displaying the spectral information acquired in the step <2B>, that is, the spectral information s _sam on the display 70, that is, the display unit 15 as a visualized image, the spectral information is specified by the International Commission on Illumination CIE. Convert to tristimulus values, ie, X, Y, Z values, then convert the X, Y, Z values to R, G, B values using the monitor profile, and display these on the display unit 15 By supplying the spectral information, it is possible to display the spectral information as a visualized image.

In this visualized image, in addition to the marking for the area where the measurement target X is determined to exist, for example, the presence probability (%) of the measurement target X existing in the vicinity of the marking can also be displayed. Alternatively, the color of the marking may be changed according to the existence probability (%) of the measurement object X existing.

Through steps <1B> to <4B> using the smartphone 1 as a detection device as described above, the presence or absence of the measurement target X in the imaging region where the measurement target X is presumed to exist, and , the position where the measurement target X is present can be detected.

Note that the detection of the measurement target X in the imaging area using the smartphone 1 as a detection device can be applied not only to the detection of animals or plants as described above, but also to the detection of minerals such as gemstones, clouds, and constellations. can be applied.

[3) Usage as appraisal device]
Hereinafter, an appraisal method for appraising the authenticity (authenticity) of articles such as bags, wallets, watches, and jewelry as measurement objects X and the degree of deterioration over time using the smartphone 1 described above as an appraisal device will be described with reference to FIG. 10 and the like. will be described in detail below.

In the appraisal method using the smartphone 1 as an appraisal device, the measurement target X to be appraised is imaged using the spectroscopic measurement unit 10, and based on the spectral information obtained using the captured spectroscopic image, the measurement target The authenticity of X or the degree of deterioration over time is appraised. After that, the appraised contents are displayed on the display 70. - 特許庁

<1C> First, the user activates an application that uses the smartphone 1 as an appraisal device by operating the input unit 16, and then selects conditions and the like as necessary according to instructions from the application (S1C).

The conditions to be input according to the instructions of the application include, for example, the type of object X to be measured, that is, the type of article, in other words, the product number of the object X to be measured, and the type of appraisal such as the degree of authenticity or deterioration over time. . By inputting in advance the product number of the measurement object X to be appraised in this way, it is possible to quickly determine the authenticity or the degree of deterioration over time of the measurement object X in the imaging area imaged by the smartphone 1. will be explained later.

<2C> Next, the user operates the input unit 16 to give an input instruction to capture an image of the measurement object X using the smartphone 1, that is, the spectroscopic measurement unit 10. Based on this input instruction, the control unit 60 controls the spectroscopic measurement unit 10 That is, by controlling the operation of the spectroscopic camera and capturing an image of the object X to be measured, spectral information is obtained.

<2C-1> First, the light source control unit 601 turns on the light source 31 according to the input of the start signal accompanying the user's input instruction to image the measurement object X in the input unit 16 (S2C).

When the light source 31 is turned on, the object X to be measured is irradiated with the illumination light emitted from the light source 31 . The irradiated light is reflected by the object X to be measured, and the reflected light enters the spectroscopic section 41 as incident light.

<2C-2> Next, based on the V-λ data stored in the storage unit 17, the spectral control unit 602 controls the voltage value (input value) of the driving voltage corresponding to the spectral wavelength region to be emitted, that is, the specific wavelength region. to get Then, a command signal is output to apply the acquired voltage value to the electrostatic actuator 45 of the Fabry-Perot etalon filter as the spectroscopic unit 41 (S3C).

As a result, of the light that has entered the spectroscopic section 41 from the object X to be measured as incident light, light having a specific wavelength region is selectively emitted toward the imaging element 21 side as emitted light.

It is preferable that the spectroscopic control unit 602 performs an adjustment process for calibrating the spectroscopic unit 41 before causing the spectroscopic unit 41 to emit light having a specific wavelength region. Thereby, the spectral spectrum s _ref of the light source 31 is obtained.

<2C-3> Next, the spectral image acquisition unit 603 controls the operation of the imaging device 21, and converts the light having the specific wavelength region, which is emitted from the spectroscopic unit 41 as emitted light, into the imaging device 21 as a spectral image. Acquired by That is, the imaging element 21 acquires the light amount measurement data (the amount of received light) of light having a specific wavelength region among the reflected light reflected from the measurement object X as a spectral image. Then, the spectral image acquisition unit 603 causes the storage unit 17 to store the acquired spectral image together with the measurement wavelength region, ie, the specific wavelength region corresponding to the spectral image (S4C).

In such a spectroscopic image acquisition method, the spectroscopic section 41 is arranged on the optical axis of the light received by the imaging device 21 between the measurement object X and the imaging device 21 . As a result, only the light in the specific wavelength region included in the light reflected by the measurement object X is transmitted by the spectroscopic unit 41, and the intensity of the light in this specific wavelength region is spectrally measured by the imaging device 21 as a spectral image.

<2C-4> Next, after obtaining a spectral image with light having a specific wavelength region for the first time, is it necessary to obtain a spectral image for a second time with light having a specific wavelength region different from the first specific wavelength region? Whether or not is determined based on the conditions selected by the user in step <1C>. That is, it is determined whether or not it is necessary to continuously acquire spectral images of light having a second specific wavelength region different from the first specific wavelength region (S5C).

In this determination (S5C), if it is necessary to acquire a spectral image in the light having the second specific wavelength region, instead of the light having the first specific wavelength region, the light having the second specific wavelength region , the steps <2C-2> to <2C-4> are repeated. That is, after changing the voltage value applied between the fixed electrode 412 and the movable electrode 422 of the electrostatic actuator 45 to set the second specific wavelength region, the steps <2C-2> to the present step <2C- 4> is repeated. As a result, a second spectroscopic image of light having the specific wavelength region is obtained. Such a second specific wavelength region, that is, acquisition of spectral images in light having a different specific wavelength region is repeated 1st, 2nd to n times. As described above, the steps <2C-2> to <2C-4> are repeated. Then, in the 1st, 2nd to n-th times, the spectral images acquired with the light having the respective acquired specific wavelength regions are superimposed. Thereby, the spectral information can be obtained as the spectral information s _sam indicating the relationship between each specific wavelength region and the light intensity corresponding to each pixel included in the spectral image.

That is, in FIG. 8, the spectrum information s _sam is a graph showing the relationship between each specific wavelength region divided into n by a uniform wavelength width in the measurement wavelength region of 400 nm or more and 760 nm or less, which is the visible light region, and the light intensity. , and this graph, that is, spectral information s _sam is obtained as spectral information possessed by each pixel of M rows×N columns.

Note that the number n of repetitions of steps <2C-2> to this step <2C-4>, that is, the number n of specific wavelength regions obtained by dividing the measurement wavelength region into equal widths is not particularly limited. , preferably 10 or more and 20 or less, more preferably 12. Thereby, spectral information s _sam can be obtained with excellent accuracy.

On the other hand, if it is not necessary to acquire a spectral image for light having the next wavelength, acquisition of a spectral image by the spectroscopic measurement unit 10 is ended, and the process proceeds to the next step <3C>.

<3C> Next, the analysis processing unit 604 analyzes the spectral image based on the spectral image of the measurement target X and the specific wavelength region, that is, the spectral information s _sam , stored in the storage unit 17 (S6C). .

In other words, the analysis processing unit 604 acquires the spectrum information _{s_sam} stored in the storage unit 17 in step <2C>. After that, the spectral information s _sam as spectral information is used as a feature amount, and analysis processing is performed for comparison with a database, thereby performing identification of the imaged measurement object X. FIG.

Specifically, the reflectance r=s _sam /s _ref of the measurement object X is calculated from the spectrum information s _sam and the spectral spectrum s _ref of the light source 31 .

Then, the analysis processing unit 604 selects the data r ⁱ corresponding to the group i=1, . The object ^X to be measured is appraised by obtaining ri and using ^ri to determine whether the reflectance r of the object X is equivalent to the reflectance ^ri of the genuine product.

More specifically, first, a projection function f(·) onto a discriminant space suitable for discriminating each group is generated based on a specific discriminant criterion. Note that the discrimination criteria include, for example, the Fisher discrimination criteria and the least squares criteria.

Then, when the type of appraisal is the authenticity of an article, the reflectance r of the object X to be measured is projected onto the discriminant space to be y.

y = f(r)

Similarly, the data r ⁱ of a brand-new genuine product (i=x) is also projected onto the discriminant space to be y(r ⁱ ). Then, the distance m ⁱ (i=x) in the discriminant space between the position y of the measurement target X in the discriminant space and the genuine product (i=x) is calculated.

m ⁱ (i=x)=g(y,y(r ⁱ ))

Here, g( ) is a function that calculates the distance in the discriminant space. As the distance, for example, Mahalanobis distance, Euclidean distance, or the like can be used.

Then, when the magnitude of m ⁱ (i=x) is smaller than a set threshold, it is determined that the object to be measured is a genuine product. The probability (%) of being

If the type of appraisal is the degree of deterioration over time of an article, the reflectance r of the object X to be measured is projected onto the discriminant space to be y.

y = f(r)

^Similarly , the data r ⁱ of genuine products (i=1, . Then, the distance m ⁱ (i=1,...,M) in the discriminant space between the position y of the measurement object X in the discriminant space and the genuine product (i=1,...,M) according to the degree of deterioration is calculate.

m ⁱ (i=1,…,M)=g(y,y(r ⁱ ))

Here, y(r ⁱ ) is a set of positions on the discriminant space of data belonging to genuine products, that is, y(r ⁱ )={y(r ⁱ 1),...,y(r ⁱ _N )} ( In the formula, N represents the number of data belonging to group i), and g(a,b) is a function for calculating the distance between a and b in the discriminant space. As the distance, for example, Mahalanobis distance, Euclidean distance, or the like can be used.

Then, if the magnitude of m ⁱ (i=1, .

<4C> Next, the display control unit 605 creates a visualized image of information on the measurement object X specified by the analysis processing unit 604, and then causes the display 70 including the display unit 15 to display this visualized image (S7C). ).

Further, when displaying the spectral information acquired in step <2C>, that is, the spectral information s _sam on the display 70, that is, the display unit 15 as a visualized image, the spectral information is Convert to tristimulus values, ie, X, Y, Z values, then convert the X, Y, Z values to R, G, B values using the monitor profile, and display these on the display unit 15 By supplying the spectral information, it is possible to display the spectral information as a visualized image.

In this visualized image, in addition to the image of the measured object X that has been appraised, if the classification of the appraisal is the authenticity of the measured object X, the determination result of whether or not the measured object X is genuine, , information such as the probability (%) of authenticity is displayed, and if the appraisal classification is the degree of deterioration over time of the measurement object X, the degree of deterioration (%) of the measurement object X, , and information such as the assessed price based on it is displayed. If the classification of the appraisal is the authenticity of the object X to be measured, and the appraisal results indicate that the object to be measured X is genuine, the degree of deterioration over time of the object X to be measured is automatically determined. You may make it shift to appraisal of.

The appraisal of the object X to be measured is performed through steps <1C> to <4C> using the smartphone 1 as the appraisal device as described above.

Note that the appraisal of the object X to be measured using the smartphone 1 as an appraisal device can be applied to items such as bags, wallets, watches, and jewelry as described above. It can also be applied to the identification of animals, plants, clouds and constellations.

Furthermore, in the present embodiment, the case of specifying the measurement object X using the reflectance of the measurement object X as the spectral information has been described, but the present invention is not limited to this. Alternatively, the measurement target X can be specified even when Kubelka-Munk transform data or the like is used as the spectral information.

Further, in the present embodiment, the spectroscopic measurement unit 10 is provided with the light source 31, the spectroscopic unit 41, and the imaging element 21 as a spectroscopic camera. may be omitted. In this case, in the steps <2A>, <2B>, and <2C> in which the spectroscopic measurement unit 10 captures a spectroscopic image, the spectroscopic measurement unit 10 irradiates the measurement object X with light from outside such as the sun or indoor lighting. Performed by light.

Furthermore, in the present embodiment, the information system of the present invention is described as being completed by an information terminal alone, and the smartphone 1 is used as an example. However, such an information terminal is not limited to the smartphone 1, For example, it may be a tablet terminal, a notebook computer, a digital camera, a digital video camera, an in-vehicle monitor, a drive recorder, or the like. As in this embodiment, the information system of the present invention can be configured to be completed with only one information terminal, so that it can be used offline. It can be used even in a stable place.

Further, in the present embodiment, a case where the input unit 16 is configured by a touch panel has been described, but the present invention is not limited to this, and the input unit 16 may be operation buttons provided on the housing of the smartphone 1. However, the input may be performed by voice via a microphone provided in the smartphone 1, or a combination of these may be used.

Here, as described above, in order to obtain spectral information, that is, spectral information s _sam using the spectroscopic camera, that is, the spectroscopic measurement unit 10, acquisition of spectral images in light having a plurality of specific wavelength regions is performed by: 1) measuring object X When used as an electronic encyclopedia for identifying the type of, the steps <2A-2> to <2A-4> are used as 2) a detection device for detecting the measurement object X from the imaged imaging area, the above When steps <2B-2> to step <2B-4> are used as 3) an appraisal device for appraising an article as measurement object X, steps <2C-2> to step <2C-4> are performed. As shown in FIG. 8, it is necessary to repeat each operation up to the n-th time and to superimpose the acquired spectral images in the n specific wavelength regions.

Therefore, based on the input of the start signal in the input unit 16 according to the user's input instruction, after sequentially acquiring a plurality of, i.e., n spectral images having each specific wavelength region, these spectral images are superimposed to form one spectral image. In the method of obtaining information, it takes a long time to obtain one frame when one piece of spectral information, that is, spectral information s _sam is defined as one frame. Therefore, there is a problem that image deviation occurs in the acquired spectral information due to camera shake or the like of the user holding the smartphone 1 .

On the other hand, in the present invention, as shown in FIG. 11, the control unit 60 controls the operation of the spectroscopic measurement unit 10 before inputting the start signal in the input unit 16 in accordance with the user's input instruction, and performs a predetermined and after inputting the start signal in the input unit 16, controlling the operation of the spectroscopic measurement unit 10 to obtain the spectroscopic image in the specific wavelength region not acquired before the input of the start signal , and the storage unit 17 is configured to store spectral images acquired before and after the start signal is input to the input unit 16 . As a result, one piece of spectral information, that is, one frame, is obtained by superimposing the spectral image acquired before the input of the start signal and the spectral image acquired after the input of the start signal.

Thus, in the present invention, before inputting the start signal in the input unit 16, the spectroscopic measurement unit 10 is operated in advance to acquire a spectroscopic image in a predetermined specific wavelength region, and furthermore, the spectroscopic image acquired before this input It is configured to store images in the storage unit 17 . Then, after inputting the start signal in the input unit 16, acquire a spectral image in a specific wavelength region that has not been acquired before the input of the start signal, and then acquire the spectral image acquired before the input and after the input of the start signal. One piece of spectral information, that is, one frame is obtained by superimposing the obtained spectral image.

Therefore, among the intervals between the time at which each spectral image is acquired and the time at which the start signal is input, which are used to obtain one piece of spectral information, the longest interval is conventionally , it is possible to set a shorter time than when all spectral images are acquired after the start signal is input. Therefore, it is possible to accurately suppress or prevent image deviation in the obtained spectral information due to camera shake or the like by the user.

Therefore, an image processing method for generating a spectral image using a spectral image acquired before the input of the start signal in the input unit 16 and a spectral image acquired after the input will be described below.

[Image processing method]
This image processing method is a method used when acquiring spectroscopic information unique to the measurement object X using the smartphone 1. Before inputting a start signal in the input unit 16, spectroscopic data in a predetermined specific wavelength region is obtained. a first acquisition step of acquiring an image and storing the spectroscopic image acquired before the input in the storage unit 17; A second acquisition step of acquiring a spectral image in and storing the spectral image acquired after the input in the storage unit 17, the spectral image acquired before the input of the start signal, and the spectral image acquired after the input of the start signal and a superimposition step of obtaining one piece of spectral information by superimposing an image, which will be described in detail below with reference to FIGS.

<1D> First, as shown in Figs. A spectral image in the region is acquired, and then the spectral image acquired before the input of the start signal is stored in the storage unit 17 (first acquisition step; S1D).

Acquisition of the spectroscopic image by the operation of the spectroscopic measurement unit 10 can be performed, for example, by the steps <2A-2> to <2A-4> described in the above-mentioned 1) Method of use as an electronic picture book. .

In addition, the number of spectroscopic images in a predetermined specific wavelength region to be acquired prior to the input of the start signal is not particularly limited, as long as it is determined when the user inputs conditions etc. in advance by operating the input unit 16. Good, but the time required to obtain the spectroscopic image acquired before the input of the start signal in this step <1D> and the time required to obtain the spectroscopic image acquired after the input of the start signal in the post-step <3D> and B is the time required to obtain the spectroscopic image acquired before the input of the start signal in this step <1D>, B/A is preferably 0.25 or more and 0.25 The number of spectroscopic images to be acquired before inputting the start signal in step <1D> is determined so that it is set to 75 or less, more preferably 0.45 or more and 0.55 or less.

By determining the number of spectral images to be acquired before inputting the start signal so that B/A satisfies the above relationship, the time at which each spectral image is acquired, which is used when obtaining one piece of spectral information, and , the longest separation time, which is the longest separation time among the separation times from the time when the start signal is input, can be set to be shorter. Therefore, it is possible to more accurately suppress or prevent image deviation from occurring in the obtained spectral information.

<2D> Next, the control unit 60 determines whether or not there is an input instruction for image acquisition by the spectroscopic camera by the user operating the input unit 16, in other words, whether or not the shutter of the spectroscopic camera has been pressed (S2D).

In this determination (S2D), if the shutter of the spectroscopic camera is not pressed by the user's operation of the input unit 16, the above step <1D> is repeated. Thereby, acquisition of spectral images in a predetermined specific wavelength region is continuously performed.

In addition, when performing the step <1D> repeatedly, it is preferable to delete the spectroscopic image stored in the storage unit 17 in the previous step <1D>. As a result, unnecessary spectral images can be reliably prevented from remaining in the storage unit 17 .

On the other hand, when the user presses the shutter of the spectroscopic camera by operating the input unit 16, the step <1D> is repeated based on the input of the start signal accompanying the input instruction by operating the input unit 16. , and proceed to the next step <3D>.

<3D> Next, the control unit 60 controls the operation of the spectroscopic measurement unit 10, that is, the spectroscopic camera, acquires a spectroscopic image in a specific wavelength region that has not been acquired before the input of the start signal, and then inputs the start signal. A spectroscopic image acquired later is stored in the storage unit 17 (second acquisition step; S3D).

Acquisition of the spectroscopic image by the operation of the spectroscopic measurement unit 10 can be performed, for example, by the steps <2A-2> to <2A-4> described in the above-mentioned 1) Method of use as an electronic picture book. .

Further, in the present embodiment, in FIG. 11, the specific wavelength region of the spectral image obtained in step <1D> is on the short wavelength side, and the specific wavelength region of the spectral image obtained in step <3D> is on the long wavelength side. In the case of a combination on the wavelength side, that is, in the step <1D> and the present step <3D>, spectral images in a plurality of specific wavelength regions are sequentially acquired from those having a low wavelength region to those having a high wavelength region. Although the case of doing is exemplified, if the specific wavelength regions of the spectroscopic images respectively acquired in the step <1D> and the main step <3D> do not overlap in the step <1D> and the main step <3D>, It is not particularly limited. Specifically, for example, the specific wavelength region of the spectral image obtained in step <1D> is on the long wavelength side, and the specific wavelength region of the spectral image obtained in step <3D> is on the short wavelength side. In addition, the specific wavelength region of the spectral image obtained in step <1D> is near the center wavelength, and the specific wavelength region of the spectral image obtained in step <3D> is on the short wavelength side and the long wavelength side A certain combination, further, a combination having an inverse relationship with this combination may be used, and the specific wavelength regions of the spectral images acquired in the step <1D> and the present step <3D> are random. Any combination may be used.

<4D> Next, the control unit 60 controls the operation of the spectroscopic measurement unit 10, that is, the spectroscopic camera, and superimposes the spectroscopic image acquired before the input of the start signal and the spectroscopic image acquired after the input of the start signal. (superimposing step; S4D).

Thereby, one piece of spectral information can be obtained as spectral information s _sam indicating the relationship between each specific wavelength region and light intensity.

In addition, since one piece of spectral information is generated by superimposing the spectral image acquired before the input of the start signal and the spectral image acquired after the input of the start signal, the time at which each spectral image is acquired and , the longest separation time, which is the longest separation time from the time when the start signal is input, compared to the conventional case where all spectral images are acquired after the start signal is inputted. , can be set shorter. Therefore, it can be said that the obtained spectral information is accurately suppressed or prevented from causing image deviation due to hand shake or the like by the user.

Through steps <1D> to <4D> as described above, it is possible to acquire spectral information, that is, spectral information s _sam in which the occurrence of image shift is accurately suppressed or prevented.

<<Second Embodiment>>
Next, a second embodiment of the information system of the present invention will be described.

FIG. 13 is a block diagram showing a schematic configuration of a smartphone to which the second embodiment of the information system of the present invention is applied and a spectrometer.

In the following, the information system of the second embodiment will be described with a focus on differences from the information system of the first embodiment, and the description of the same items will be omitted.

The information system of the second embodiment shown in FIG. 13 includes a spectroscopic measurement unit 10 independently of the smartphone 1 as an information terminal, and the spectroscopic measurement unit 10 functions as a spectroscopic camera. are the same as those of the information system of the first embodiment. That is, in the information system of the second embodiment, the information system of the present invention does not end with the smartphone 1 alone as the information terminal, but has the smartphone 1 as the information terminal and the spectroscopic measurement unit 10 as the spectroscopic camera. are doing.

In the information system of the second embodiment, as shown in FIG. 13 , the smartphone 1 does not include the spectroscopic measurement unit 10 inside the smartphone 1 . is provided independently on the outside of the smartphone 1, and its operation is configured to be controllable by the smartphone 1. Note that the control of the spectroscopic measurement unit 10 by the smartphone 1 through the electrical connection between the smartphone 1 and the spectroscopic measurement unit 10 may be wired or wireless.

In the information system of the second embodiment having such a configuration, acquisition of an image including the measurement target X by the spectroscopic measurement unit 10 and operation such as setting of conditions for specifying the measurement target X by the smartphone 1 are performed independently. can be implemented. Therefore, it is possible to improve the operability of the information system.

In addition, by installing an application on the smartphone 1 and connecting the spectroscopic measurement unit 10, the information system of the present invention can be used, and the smartphone 1 without the spectroscopic measurement unit 10 can be used, which improves versatility. Excellent.

Furthermore, by configuring the information system of the present invention to have a configuration in which the smartphone 1 and the spectroscopic measurement unit 10 are complete, off-line use is possible except for communication between the smartphone 1 and the spectroscopic measurement unit 10. Therefore, it can be used even in places where communication conditions are unstable.

The same effects as those of the first embodiment can be obtained by the information system of the second embodiment as described above.

Further, in the present embodiment, the information system of the present invention has the smartphone 1 as the information terminal and the spectroscopic measurement unit 10 as the spectroscopic camera. Therefore, the information terminal may be configured by a tablet terminal or the like, or the smartphone 1, that is, the information terminal may be configured by a server or the like.

<<Third Embodiment>>
Next, a third embodiment of the information system of the present invention will be described.

FIG. 14 is a block diagram showing a schematic configuration of a smartphone to which the third embodiment of the information system of the invention is applied and an external display unit.

In the following, the information system of the third embodiment will be described with a focus on differences from the information system of the first embodiment, and descriptions of the same items will be omitted.

The information system of the third embodiment shown in FIG. 14 includes an external display unit 18 independently of the smartphone 1 as an information terminal, and the external display unit 18 functions as an external display unit of the smartphone 1. The information system is the same as the information system of the first embodiment except that the That is, in the information system of the third embodiment, the information system of the present invention is not completed solely by the smartphone 1 as the information terminal, but has the smartphone 1 as the information terminal and the external display unit 18 .

As shown in FIG. 14, the information system of the third embodiment includes, in addition to the display unit 15 provided in the smartphone 1, an external display unit 18 provided independently outside the smartphone 1. The operation of the display unit 18 is configured to be controllable by the smartphone 1 . Note that the control of the external display unit 18 by the smartphone 1 through the electrical connection between the smartphone 1 and the external display unit 18 may be wired or wireless.

In the information system of the third embodiment having such a configuration, confirmation by the operator of the image on the external display unit 18 and operation such as setting conditions for specifying the measurement target X by the smartphone 1 are performed independently. Therefore, it is possible to improve the operability of the information system.

In addition, by configuring the information system of the present invention to be completed with the smartphone 1 and the external display unit 18, offline use is possible except for communication between the smartphone 1 and the external display unit 18. Therefore, it can be used even in places where communication conditions are unstable.

Examples of the external display unit 18 include a mobile notebook PC, a tablet terminal, a head-up display (HUD), a head-mounted display (HMD), etc. Among them, the head-mounted display is preferable. According to the head-mounted display, in augmented reality (AR), for example, it is possible to display the result of specifying the measurement target X, the position where the measurement target X is assumed to exist, and the like. Since it can be used, it is possible to further improve the operability of the information system.

The same effects as those of the first embodiment can be obtained by the information system of the third embodiment as described above.

<<Fourth Embodiment>>
Next, a fourth embodiment of the information system of the present invention will be described.

FIG. 15 is a block diagram showing a schematic configuration of a smartphone, a spectroscopic measurement section, and an external display section to which the fourth embodiment of the information system of the present invention is applied.

In the following, the information system of the fourth embodiment will be described with a focus on differences from the information system of the first embodiment, and the description of the same items will be omitted.

The information system of the fourth embodiment shown in FIG. 15 further includes a spectroscopic measurement unit 10 and an external display unit 18 independently in addition to the smartphone 1 as an information terminal, and the spectroscopic measurement unit 10 functions as a spectroscopic camera. , and the external display unit 18 functions as an external display unit of the smartphone 1, the information system is the same as the information system of the first embodiment. That is, in the information system of the third embodiment, the information system of the present invention is not completed by the smartphone 1 alone as the information terminal, but the smartphone 1 as the information terminal, the spectrometry unit 10, and the external display unit 18. have.

In the information system of the fourth embodiment, as shown in FIG. 15, the smartphone 1 omits the arrangement of the spectroscopic measurement unit 10 inside. is provided independently on the outside of the smartphone 1, and its operation is configured to be controllable by the smartphone 1. Furthermore, in addition to the display unit 15 provided in the smartphone 1, an external display unit 18 is provided independently outside the smartphone 1, and the operation of the external display unit 18 can be controlled by the smartphone 1. It is configured.

Control of the spectroscopic measurement unit 10 by the smartphone 1 through electrical connection between the smartphone 1 and the spectroscopic measurement unit 10, and an external display unit by the smartphone 1 through electrical connection between the smartphone 1 and the external display unit 18 Each of the 18 controls can be either wired or wireless.

In the information system of the fourth embodiment having such a configuration, acquisition of an image including the measurement object X by the spectroscopic measurement unit 10, confirmation by the operator of the image on the external display unit 18, and identification of the measurement object X by the smartphone 1 Since the operation such as setting of the conditions of (1) can be performed independently, it is possible to improve the operability of the information system.

Further, by installing an application on the smartphone 1 and connecting the spectroscopic measurement unit 10 and the external display unit 18, the information system of the present invention can be used, and the smartphone 1 without the spectroscopic measurement unit 10 can be used. Since it can be used, it is excellent in versatility.

Furthermore, by configuring the information system of the present invention to have a configuration in which the smartphone 1, the spectroscopic measurement unit 10, and the external display unit 18 are completed, communication between the smartphone 1, the spectroscopic measurement unit 10, and the external display unit 18 Other than that, it can be used offline, so it can be used even in places where the communication situation is unstable.

Examples of the external display unit 18 include a mobile notebook PC, a tablet terminal, a head-up display (HUD), a head-mounted display (HMD), etc. Among them, the head-mounted display is preferable. According to the head-mounted display, in augmented reality (AR), for example, it is possible to display the result of specifying the measurement target X, the position where the measurement target X is assumed to exist, and the like. Since it can be used, it is possible to further improve the operability of the information system.

The same effects as those of the first embodiment can be obtained by the information system of the fourth embodiment as described above.

Further, in the present embodiment, the case where the information system of the present invention includes the smartphone 1 as an information terminal, the spectroscopic measurement unit 10 as a spectroscopic camera, and the external display unit 18 such as a head-mounted display has been described. , the information system of the present invention may include a tablet terminal or the like as an information terminal instead of the smartphone 1, or may include a server or the like instead of the smartphone 1, that is, an information terminal.

In addition, in the present embodiment, the information system of the present invention has the spectroscopic measurement unit 10 and the external display unit 18 as separate bodies, as shown in FIG. 15, but it is not limited to this. , the spectroscopic measurement unit 10 and the external display unit 18 may be integrally formed.

<<Fifth Embodiment>>
Next, a fifth embodiment of the information system of the present invention will be described.

FIG. 16 is a block diagram showing a schematic configuration of a smartphone and a server to which the fifth embodiment of the information system of the invention is applied.

In the following, the information system of the fifth embodiment will be described with a focus on differences from the information system of the first embodiment, and descriptions of the same items will be omitted.

The information system of the fifth embodiment shown in FIG. 16 is the same as the information system of the first embodiment, except that a server 100 is independently provided in addition to the smartphone 1 as an information terminal. That is, in the information system of the fifth embodiment, the information system of the present invention is not completed solely by the smartphone 1 as the information terminal, but has the smartphone 1 as the information terminal and the server 100 .

In the information system of the fifth embodiment, as shown in FIG. 16, the smartphone 1 further includes a transmission/reception unit 19, and a control unit 60 includes a transmission/reception control unit that controls the operation of the transmission/reception unit 19. 606.

The server 100 also includes a storage unit 117, a transmission/reception unit 119, and a control unit 160. The control unit 160 controls the operations of the storage unit 117 and the transmission/reception unit 119, respectively. It has a control unit 162 .

In the information system of this embodiment, the storage unit 17 provided in the smartphone 1 does not store a database for specifying the measurement target X. Instead, the storage unit 117 provided in the server 100 stores , where this database is stored.

The transmitting/receiving unit 19 included in the smartphone 1 and the transmitting/receiving unit 119 included in the server 100 exchange databases. That is, the transmitting/receiving unit 19 transmits a database acquisition request from the storage unit 117 to the transmitting/receiving unit 119 and receives the database from the server 100 via the transmitting/receiving unit 119 . Further, the transmitting/receiving unit 119 receives a database acquisition request from the storage unit 117 by the transmitting/receiving unit 19 and transmits the database to the smartphone 1 via the transmitting/receiving unit 19 . The transfer of the database between the transmitting/receiving unit 19 and the transmitting/receiving unit 119 may be carried out either by wire or wirelessly, and in the case of wireless, may be carried out via the Internet.

In addition, the transmission/reception control unit 606 provided in the control unit 60 of the smartphone 1 controls the operation of the transmission/reception unit 19, receives the database from the server 100 via the transmission/reception unit 119, and the analysis processing unit 604 controls the transmission/reception control unit 606. , this database is received, and the object X to be measured is specified based on this.

Furthermore, the data acquisition unit 161 provided in the control unit 160 of the server 100 acquires the database stored in the storage unit 117 in response to the acquisition request, and then passes it to the transmission/reception control unit 162 . Then, the transmission/reception control unit 162 controls the operation of the transmission/reception unit 119 to transfer the database from the server 100 to the smartphone 1 via the transmission/reception unit 19 .

In the information system of the fifth embodiment having such a configuration, the information system is not completed solely by the smartphone 1, which is an information terminal, and further includes a server 100, and a database is stored in the storage unit 117 of the server 100, When this database specifies the measurement object X, it is transmitted to the smartphone 1 via the transmitter/receiver 19 and 119 . This provides an advantage that it is not necessary to store a large amount of data in the storage unit 17 included in the smartphone 1 . Moreover, when the information system includes a plurality of smartphones 1, sharing of the database can be achieved. Furthermore, simply by updating the database provided in the storage unit 117 of the server 100, the database used in each smartphone 1 can be updated as a result.

Furthermore, in the information system of the present invention, when specifying the measurement target X by the smartphone 1 offline, the necessary database is transferred from the storage unit 117 provided in the server 100 to the storage unit 17 provided in the smartphone 1 when offline. It should be stored in memory. This allows the smartphone 1 to identify the measurement target X even when offline.

The same effects as those of the first embodiment can be obtained by the information system of the fifth embodiment as described above.

Further, in the present embodiment, the information system of the present invention has the smartphone 1 as an information terminal and the server 100 . The smart phone 1, that is, the information terminal may be configured by a digital camera, a digital video camera, a head-up display (HUD), a head-mounted display (HMD), or the like.

Furthermore, in the present embodiment, the control unit 60 included in the smartphone 1 includes a light source control unit 601, a spectral control unit 602, a spectral image acquisition unit 603, an analysis processing unit 604, and a Although the display control unit 605 is provided, at least one of these may be included in the control unit 160 of the server 100 .

<<Sixth Embodiment>>
Next, a sixth embodiment of the information system of the present invention will be described.

FIG. 17 is a block diagram showing a schematic configuration of a smartphone and a server to which the sixth embodiment of the information system of the invention is applied.

In the following, the information system of the sixth embodiment will be described with a focus on differences from the information system of the first embodiment, and the description of the same items will be omitted.

The information system of the sixth embodiment shown in FIG. 17 is the same as the information system of the first embodiment, except that a server 100 is independently provided in addition to the smart phone 1 as an information terminal. That is, in the information system of the sixth embodiment, the information system of the present invention does not consist solely of the smartphone 1 as an information terminal, but has the smartphone 1 as an information terminal and the server 100 .

In the information system of the sixth embodiment, as shown in FIG. 17, the smartphone 1 does not have an analysis processing unit in the storage unit and the control unit 60, whereas the smartphone 1 further includes a transmission/reception unit 19 , and the control unit 60 has a transmission/reception control unit 606 for controlling the operation of the transmission/reception unit 19 .

The server 100 also includes a storage unit 117, a transmission/reception unit 119, and a control unit 160. The control unit 160 controls the operations of the storage unit 117 and the transmission/reception unit 119, respectively. It has a control section 162 and an analysis processing section 163 for specifying the object X to be measured.

In such an information system of the present embodiment, the arrangement of the storage unit in the smartphone 1 is omitted, and the server 100 includes the storage unit 117 instead. The storage unit 117 stores various data, like the storage unit 17 included in the smartphone 1 of the first embodiment.

Also, the arrangement of the analysis processing unit in the control unit 60 included in the smartphone 1 is omitted, and instead, the control unit 160 included in the server 100 includes the analysis processing unit 163 . Similar to the analysis processing unit 604 included in the smartphone 1 of the first embodiment, the analysis processing unit 163 identifies the measurement target X by comparing the spectroscopic spectrum, that is, the spectral information of the measurement target X with the database. implement.

The transmitting/receiving unit 19 included in the smartphone 1 and the transmitting/receiving unit 119 included in the server 100 exchange various data. Specifically, for example, the transmitting/receiving unit 19 transmits to the transmitting/receiving unit 119 the spectroscopic information specific to the measurement target X acquired by the smartphone 1, i.e., the spectral spectrum, and the transmission/reception unit 119 transmits the measurement target X from the server 100 via the transmission/reception unit 119. Receive the specified result. Further, the transmitting/receiving unit 119 receives the spectroscopic information, that is, the spectroscopic spectrum of the measurement object X acquired by the smartphone 1 from the smartphone 1 via the transmitting/receiving unit 19, and transmits the spectroscopic information to the smartphone 1 via the transmitting/receiving unit 19 for measurement. Send target X identified results. The transfer of various data between the transmitting/receiving unit 19 and the transmitting/receiving unit 119 may be carried out either by wire or wirelessly, and in the case of wireless, may be carried out via the Internet. .

Further, the transmission/reception control unit 606 provided in the control unit 60 of the smartphone 1 controls the operation of the transmission/reception unit 19 to transmit the spectral spectrum of the measurement target X, that is, the spectral information acquired by the spectral image acquisition unit 603 to the transmission/reception unit 119. to the analysis processing unit 163 included in the server 100 via the server 100 . Then, the analysis processing unit 163 acquires the database stored in the storage unit 117 by controlling the operation of the data acquisition unit 161, and then identifies the measurement target X by comparing the spectral information with the database. conduct.

Furthermore, the transmission/reception control unit 162 provided in the control unit 160 of the server 100 controls the operation of the transmission/reception unit 119 to transmit data from the server 100 to the smartphone 1 via the transmission/reception unit 19 to specify the measurement target X by the analysis processing unit 163. pass the result.

In the information system of the sixth embodiment having such a configuration, the information system is not completed solely by the smartphone 1, which is an information terminal, and further includes a server 100, and a database is stored in the storage unit 117 of the server 100. . Then, the analysis processing unit 163 of the server 100 identifies the measurement object X based on this database, and then transmits the result of the identification of the measurement object X to the smartphone 1 via the transmission/reception units 19 and 119. It is designed to be This eliminates the need to store a large amount of data in the storage unit provided in the smartphone 1, and furthermore, it eliminates the need to impose a large computational load on the smartphone 1, which is an advantage. Further, by making the analysis processing unit 163 excellent in processing capability and by making the information amount of the database stored in the storage unit 117 more detailed, it is possible to specify the measurement target X with higher accuracy. It becomes possible.

Moreover, when the information system includes a plurality of smartphones 1, sharing of the database can be achieved. Furthermore, simply by updating the database provided in the storage unit 117 of the server 100, the database used in each smartphone 1 can be updated as a result.

The same effects as those of the first embodiment can be obtained by the information system of the sixth embodiment as described above.

Further, in the present embodiment, the information system of the present invention has the smartphone 1 as an information terminal and the server 100 . The smart phone 1, that is, the information terminal may be configured by a digital camera, a digital video camera, a head-up display (HUD), a head-mounted display (HMD), or the like.

Furthermore, in the present embodiment, the control unit 60 included in the smartphone 1 includes a light source control unit 601, a spectral control unit 602, a spectral image acquisition unit 603, and a display control unit 605, as in the first embodiment. However, at least one of these may be included in the control unit 160 of the server 100 .

Although the image processing method, image processing apparatus, and information system of the present invention have been described above based on the illustrated embodiments, the present invention is not limited to these.

For example, in the image processing apparatus and information system of the present invention, each component can be replaced with any component that can exhibit similar functions, or any component can be added.

Further, in the information system of the present invention, any two or more configurations shown in the first to sixth embodiments may be combined.
Moreover, in the image processing method of the present invention, an optional step may be added.

REFERENCE SIGNS LIST 1 smart phone 10 spectral measurement unit 15 display unit 16 input unit 17 storage unit 18 external display unit 19 transmission/reception unit 21 imaging device 31 light source 41 spectroscopic unit 45... Electrostatic actuator 51... Circuit board 52... Circuit board 60... Control unit 70... Display 81... First spectroscopic unit side optical system 83... First imaging element side optical system 100... Server 117 Memory unit 119 Transmission/reception unit 160 Control unit 161 Data acquisition unit 162 Transmission/reception control unit 163 Analysis processing unit 410 Fixed substrate 411 Fixed reflection film 412 Fixed electrode 413 Groove 414 Bonding film 415 Reflective film installation unit 420 Movable substrate 421 Movable reflective film 422 Movable electrode 423 Groove 425 Reflective film installation unit 601 Light source control unit 602 Spectrum Control unit 603 Spectral image acquisition unit 604 Analysis processing unit 605 Display control unit 606 Transmission/reception control unit 811 Incident lens 812 Projection lens 831 Incident/emission lens X Measurement object

Claims (10)

an input unit for receiving a start signal for instructing the start of acquisition when acquiring spectroscopic information unique to an object to be measured;
an imaging device that captures an image based on the reflected light reflected from the object; and a spectroscopic unit that selectively emits light having a predetermined wavelength range from the incident light and that can change the wavelength range of the emitted light. a spectrometer having
a control unit having a spectroscopic control unit that controls the operation of the spectroscopic unit; and a spectral image acquisition unit that acquires the spectral information by operating the imaging device;
An image processing method using an image processing device comprising a storage unit that stores the spectral information,
When the spectral information is spectral information obtained by superimposing the acquired spectral images in a specific wavelength region obtained by dividing the measurement wavelength region to be measured,
A first acquisition step of acquiring the spectroscopic image in the predetermined specific wavelength region before inputting the start signal to the input unit, and storing the spectroscopic image acquired before inputting the start signal in the storage unit. and,
after inputting the start signal in the input unit, acquiring the spectral image in the specific wavelength region that has not been acquired before the input of the start signal, and storing the spectral image acquired after the input of the start signal in the storage unit; a second acquiring step of storing in
and a superimposing step of obtaining one piece of spectral information by superimposing the spectral image acquired before the input of the start signal and the spectral image acquired after the input of the start signal. Image processing method. Time required to obtain the spectroscopic image acquired before input of the start signal in the first acquisition step and time required to obtain the spectroscopic image acquired after input of the start signal in the second acquisition step When A is the total time and time, and B is the time required to obtain the spectroscopic image acquired before the input of the start signal in the first acquisition step, B/A is 0.25 or more. 2. The image processing method according to claim 1, wherein the value is 0.75 or less. 3. The image processing method according to claim 1, wherein the measurement wavelength range is a visible light range. 4. The image processing method according to claim 3, wherein the visible light region is from 400 nm to 760 nm. 2. The image processing method according to claim 1 , wherein the plurality of specific wavelength regions are obtained by dividing the measurement wavelength region into equal widths. 6. The image processing method according to claim 5, wherein the number of even divisions of the measurement wavelength region is 10 or more and 20 or less. 2. The image according to claim 1, wherein in the first acquisition step and the second acquisition step, the spectroscopic images in the plurality of specific wavelength regions are sequentially acquired from one having a low wavelength region to one having a high wavelength region. Processing method. an input unit for receiving a start signal for instructing the start of acquisition when acquiring spectroscopic information unique to an object to be measured;
an imaging device that captures an image based on the reflected light reflected from the object; and a spectroscopic unit that selectively emits light having a predetermined wavelength range from the incident light and that can change the wavelength range of the emitted light. a spectrometer having
a control unit having a spectroscopic control unit that controls the operation of the spectroscopic unit; and a spectral image acquisition unit that acquires the spectral information by operating the imaging device;
a storage unit that stores the spectral information,
The spectral information is spectral information obtained by superimposing the acquired spectral images in specific wavelength regions obtained by dividing the measurement wavelength region to be measured,
The control unit acquires the spectroscopic image in the predetermined specific wavelength region by controlling the operation of the spectroscopic measurement unit before the start signal is input to the input unit, and the start signal is input to the input unit. After inputting the signal, controlling the operation of the spectroscopic measurement unit to acquire the spectroscopic image in the specific wavelength region that has not been acquired before the input of the start signal;
The storage unit is configured to store the spectral images acquired before and after the input of the start signal in the input unit,
The image processing apparatus, wherein the one piece of spectral information is obtained by superimposing the spectral image acquired before the input of the start signal and the spectral image acquired after the input of the start signal. . 9. The image processing apparatus according to claim 8, further comprising a display section for displaying said spectral information. An information system comprising the image processing apparatus according to claim 8 .

Images