JP7182103B2 - IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, COMPUTER PROGRAM - Google Patents

Description

The present invention relates to an image processing technique for extracting a pupil range from an image of an eyeball.

It has long been known in the medical field that the pupil is under the dual innervation of the sympathetic and parasympathetic nerves. Enlargement of the pupil (mydriasis) is achieved by the action of the sympathetic nerve, and contraction of the pupil (miosis) is achieved by the action of the parasympathetic nerve.
Therefore, it is possible to determine which of the sympathetic nerves and the parasympathetic nerves is working predominantly by irradiating the subject's eyes with stimulating light and observing how the pupils react. When the sympathetic nerve is dominant, the subject is generally in a tense or excited state, and when the parasympathetic nerve is predominant, the subject is generally in a relaxed or sedated state. Therefore, by observing the reaction of the pupil, it is possible to determine not only which of the sympathetic nerves and the parasympathetic nerves is working predominantly, but also, for example, the subject's current state of mind and body. Moreover, as is well known, both the sympathetic nerves and the parasympathetic nerves belong to the autonomic nerves, which work independently of the subject's will. Therefore, the result of such determination is highly reliable in that it cannot be deceived by the intention of the subject.
Determination of mental and physical conditions by observing pupils is performed for both medical and non-medical purposes. Moreover, subjects are not limited to humans, and can be mammals other than humans, for example.

In order to achieve the above purposes, an eyeball may be imaged. An eyeball imaging device for that purpose has also been put into practical use and is in widespread use.

However, such an eyeball imaging device also has a problem.
In particular, when the eyeball is imaged for the purpose described above, it is necessary to accurately capture the expansion and contraction of the pupil. For this purpose, a technique is required for accurately extracting the range of the pupil from the moving image generated by imaging the eyeball with a camera.
However, doing it automatically is not trivial. The pupil is generally in the center of the iris, which extends circularly around it. However, since the boundary line between the iris and the pupil is not a smooth curve and there are fine irregularities, image processing is performed on the moving image generated by imaging the eyeball, and the pupil in the moving image is corrected. Extracting the range of is difficult, at least when trying to do it accurately.

An object of the present invention is to provide a technique for automatically and accurately extracting the range of the pupil in the moving image by performing image processing on the moving image data generated by imaging the eyeball. do.

In order to solve the above problems, the inventor of the present application proposes the following technique.
The invention of the present application is a sequence of a large number of still image data, which is data about a still image including the eyeball of one eye of a subject, which is generated by a predetermined camera capable of capturing moving images. a moving image data receiving unit that receives moving image data that is moving image data; Binary moving image, which is moving image data, which is a sequence of a large number of binary still image data, which is binary still image data, which is an image obtained by binarizing the still image by performing a value conversion process. and a binarization processing unit that generates image data.
The binarization processing unit in the image processing apparatus binarizes each of the still images based on the plurality of still image data to be binarized using a plurality of different thresholds, which are provisional thresholds. a first processing unit for generating a plurality of provisional threshold still images in which the density of each pixel is maximized or minimized, and based on the plurality of provisional threshold still images generated by the first processing unit specifying a range of the provisional threshold in which the area of the range in which the density in the provisional threshold still image is maximum does not change even if the provisional threshold changes, and a threshold included in the range is set as the specific threshold. and a second processing unit that determines, as the binary still image, one of the plurality of temporary threshold still images that has been binarized using the specific threshold.

The image processing apparatus according to the present application is used to extract the range of the pupil from the moving image based on the moving image data, more specifically, from the eyeball reflected in the moving image. However, the purpose of extracting the range of the pupil from the eyeball, or the purpose of using the range of the pupil extracted from the image of the eyeball, is to determine which of the sympathetic nerves and parasympathetic nerves is dominant, It may be to determine the mental and physical state of the person, or it may be other than that.
An image processing apparatus according to the present application includes a moving image data reception unit and receives moving image data from a camera. The moving image data is a sequence of a number of still image data, which are data about still images. Such moving image data is extremely common. A still image based on still image data that constitutes moving image data reflects the eyeball of one eye of the subject.
As described above, the image processing apparatus includes the moving image data receiving section, and the moving image data receiving section receives moving image data from the camera. At this time, the image processing device may be configured separately from the camera or may be configured integrally with the camera. The image processing device configured separately from the camera is configured by, for example, a publicly known computer device such as a desktop or laptop computer, tablet, or smart phone. When the image processing device is configured separately from the camera, a means for connecting the camera and the computer device, for example, a connection terminal in the computer device when the camera and the computer device are wired. However, when the camera and the computer are wirelessly connected, the communication mechanism in the computer (for example, the communication mechanism for communicating by Bluetooth) serves as the moving image data reception section in the present invention. When the image processing device is integrated with the camera, the image processing device is, for example, built in the camera. In this case, the camera is an imaging device (for example, CCD or CMOS) that generates moving image data. The interface for connecting the elements is the moving image data reception section in the present invention.
Note that the moving image data reception unit may receive the moving image data directly from the camera (for example, without passing through other devices or devices). On the other hand, the moving image data receiving section may receive the moving image data from the camera via a predetermined network. In this case, the image processing apparatus uses so-called cloud computing technology.
In this case, the image processing apparatus receives moving image data via a network from a computer separate from the camera or from a computer integrated with the camera, and performs image processing on the moving image data as will be described later. The binary moving image data generated by the operation is returned to the computer that sent it, or is transferred to another device.

An image processing apparatus according to the present invention includes a binarization processing section. This binarization processing unit performs binarization processing on moving image data using a specific threshold value that is a threshold value. The binarization process itself performed by it is known or well-known. The binarization processing unit generates binary moving image data by binarizing at least a plurality of still image data included in the moving image data received by the moving image data receiving unit. do. Performing binarization processing on at least a plurality of pieces of still image data means that it is not necessary to perform binarization processing on all portions of moving images based on moving image data. showing. For example, when the moving image data receiving unit receives moving image data for a moving image for 10 seconds, the finally generated binary moving image data is for the latter half of the moving image for 7 seconds. It is an example of what is meant by the statement that binarization processing is performed on at least a plurality of pieces of still image data. In addition, by binarizing a large number of still image data included in the moving image data every other one, the number of frames of the binary moving image data finally generated can be reduced to the number of frames of the moving image data. Halving a number is another example of what is meant by the above phrase. If the computing power of the image processing device is insufficient and it is difficult to binarize all the still image data contained in the moving image data, such a method can be adopted. can.

As is well known or well-known, a threshold value is essential for binarization processing, and in the case of binarization processing performed by a binarization processing unit, a specific threshold value is used as the threshold value. The image processing apparatus according to the present invention determines the specific threshold as described later. The p-tile method, the mode method, etc. are known as methods for determining the threshold value when performing the binarization process. and iris boundaries cannot be accurately determined. Therefore, the binarization processing unit in the present invention binarizes still images to be binarized out of still images based on still image data in moving image data as follows. It is designed to be a binary still image.
The binarization processing section in the image processing apparatus of the present application includes a first processing section and a second processing section. The first processing unit binarizes each of the still images based on the plurality of still image data to be binarized with a plurality of different thresholds, that is, provisional thresholds, thereby obtaining the density of each pixel from one piece of still image data. A plurality of provisional threshold still images are generated in which is maximized or minimized. That is, the first processing unit repeats the process of binarizing a certain still image while changing the threshold. The resulting binarized image is the temporary threshold still image. The number of provisional threshold still images to be created, in other words, how many thresholds are used to binarize the still images can be determined as appropriate. If (or lightness) is 256 steps from 0 to 255, the number can be 256, for example. However, the specific threshold does not need to be set at all stages of the density. For example, if the density has 256 levels from 0 to 255, only the odd-numbered density is used as the threshold, or the specific threshold is included. can be used as a threshold only in the range of densities in which is approximately expected;
On the other hand, the second processing unit determines, based on the plurality of temporary threshold still images generated by the first processing unit, that even if the temporary threshold is changed, the area of the range in which the density in the temporary threshold still image is maximum is A range of unchanging provisional thresholds is specified, a threshold included in the range is determined as a specific threshold, and a plurality of provisional threshold still images binarized using the specific threshold are binarized still images. It is determined as an image. The reason why the specific threshold value and thus the binary still image can be determined by such a method will be described below. For example, the human pupil reflected in a still image has a lower brightness (higher density) than other parts of the eyeball, including the iris, regardless of race. This is based on the research result of the inventor of the present application that it is possible to extract only the pupil from the image.
When a still image in which an eyeball is reflected is binarized with various temporary threshold values that gradually decrease from maximum density to minimum density to obtain a temporary threshold still image, as the temporary threshold value gradually decreases, The area of the range in which the density in the temporary threshold still image is maximum gradually increases. When the area of the range in which the density is maximum does not change, and the provisional threshold becomes smaller than a certain size, as the threshold becomes smaller, the area of the range in which the density in the provisional threshold still image is maximized. Temporary threshold when the area of the range where the density in the still image is maximum does not change even if the size of the threshold changes. It has been found that the range of the maximum density in the still image matches well with the range of the pupil in the original still image. According to these, based on the plurality of temporary threshold still images generated by the first processing unit, even if the temporary threshold is changed, the area of the range in which the density in the temporary threshold still image is maximum does not change. Binary moving image data obtained as a result of the binarization processing performed by the binarization processing unit is the binary moving image Each binary still image based on the binary still image data constituting the data is an accurate extraction of only the pupil.
The first processing unit and the second processing unit in the binarization processing unit repeat the above-described processing for a plurality of still image data to be binarized. As a result, binary moving image data composed of continuous binary still image data is generated. In this case, since the processes performed by the first processing section and the second processing section are standardized, the processes performed by them can be automatically performed.

Therefore, by using the image processing apparatus of the present invention, the range of the pupil in the moving image can be automatically and accurately extracted by performing image processing on the moving image data generated by imaging the eyeball. becomes possible.
Moreover, the extraction of the pupil by this image processing apparatus is performed when it is necessary to accurately extract the pupil even when a change in the imaging environment that affects the imaging result occurs during the imaging of the moving image. Especially effective. For example, a change in the environment between the presence and absence of illumination light may occur during imaging by a camera. Alternatively, although imaging is basically performed in the absence of illumination light, there are cases where illumination light temporarily exists while the camera is performing imaging. In this case, there will be three states: no illumination, illumination, and no illumination. When there is such a change in imaging environment, in general, there is a case where a specific threshold suitable for each still image in the moving image captured in each environment is required.
In the present invention, the first processing unit and the second processing unit repeat the above-described processing for all still images to be binarized. An appropriate specific threshold is selected in binarizing all still images such that . As a result, pupil extraction by this image processing apparatus can be performed accurately even when a change in imaging environment that affects imaging results occurs during imaging of a moving image.

As described above, the second processing unit in the binarization processing unit, based on the plurality of temporary threshold still images generated by the first processing unit, determines whether the density in the temporary threshold still image is changed even if the temporary threshold changes. A range of temporary thresholds in which the area of the maximum range does not change is specified, and the specific threshold is determined from the temporary thresholds in that range.
For example, the second processing unit may determine the specific threshold so as to be included in a 50% range including the center of the range of the provisional threshold. By doing so, extraction of the range of the pupil in the moving image based on the moving image data generated by imaging the eyeball becomes more accurate. More preferably, the 50% range mentioned above is 30%. Of course, by doing so, the extraction of the range of the pupil in the moving image based on the moving image data generated by imaging the eyeball becomes more accurate.
The relative position (or size) of the specific threshold in the range of the provisional threshold in which the area of the range where the density in the provisional threshold still image is maximum does not change even if the provisional threshold changes, all still images It is preferable to make it constant when binarizing.

The inventor of the present application also proposes a method executed by an image processing apparatus as one aspect of the present invention. The effect of such a method is equivalent to that of the image processing apparatus according to the present invention.
An example of the method is to collect a large number of still image data, which is data about a still image including a single eyeball of a subject, generated by a predetermined camera capable of capturing moving images. It is a method executed by a computer having a moving image data reception unit that receives moving image data, which is data of moving images, which is a sequence of moving image data.
Then, in this method, at least a plurality of the still image data included in the moving image data is binarized using a specific threshold value, which is executed by the computer. A binarization process for generating binary moving image data, which is moving image data, which is obtained by connecting a large number of binary still image data, which is binary still image data, which is an image obtained by binarizing the still image. wherein, in the binarization process, each of the still images based on the plurality of still image data to be binarized is binarized with a plurality of temporary thresholds that are different thresholds. a first processing step for generating a plurality of provisional threshold still images in which the density of each pixel is maximized or minimized; and based on the plurality of provisional threshold still images generated in the first processing step, specifying a range of the provisional threshold in which the area of the range in which the density in the provisional threshold still image is maximum does not change even if the provisional threshold changes, and a threshold included in the range is set as the specific threshold. and a second processing step of determining, as the binary still image, one of the plurality of temporary threshold still images binarized using the specific threshold.

The inventor of the present application also proposes, as one aspect of the present invention, a computer program for causing a predetermined, for example, general-purpose computer to function as an image processing apparatus. The effect of such a computer program is equivalent to the effect of the image processing apparatus according to the present invention, and another effect is that a predetermined computer can be made to function as the image processing apparatus according to the present application.
The computer program, which is an example, generates still image data, which is data about a still image including the eyeball of one eye of a subject, generated by a predetermined camera capable of capturing moving images. A computer having a moving image data receiving unit that receives moving image data, which is data of moving images, which is a sequence of a large number of moving image data, is subjected to a threshold value for at least a plurality of the still image data included in the moving image data. By performing binarization processing using a specific threshold value, a sequence of a large number of binary still image data, which is binary still image data that is an image obtained by binarizing the still image, a binarization processing unit that generates binary moving image data that is image data, wherein the binarization processing unit converts the still image based on a plurality of the still image data to be binarized. a first processing unit that generates a plurality of temporary threshold still images in which the density of each pixel is maximized or minimized by binarizing each of the above with a temporary threshold that is a plurality of different thresholds; Based on a plurality of the provisional threshold still images generated by one processing unit, the provisional threshold is such that even if the provisional threshold is changed, the area of the range in which the density in the provisional threshold still image is maximum does not change. A range is specified, a threshold included in the range is determined as the specific threshold, and one of the plurality of temporary threshold still images binarized using the specific threshold is set as the binary still image. A computer program for functioning as an image processing device, comprising a second processing unit that determines.

1 is a diagram showing the overall configuration of a mental and physical state determination system according to a first embodiment; FIG. FIG. 2 is a horizontal cross-sectional view of the head of the eye imaging device included in the mental and physical condition determination system shown in FIG. 1 ; FIG. 2 is a vertical cross-sectional view of an eye imaging device included in the mental and physical state determination system shown in FIG. 1 ; FIG. 2 is a diagram showing the hardware configuration of the computer device shown in FIG. 1; FIG. 3 is a block diagram showing functional blocks generated inside the computer device shown in FIG. 2; FIG. 6 is a block diagram showing an example of functional blocks generated inside the binarization processing unit shown in FIG. 5A; FIG. 2 is a diagram for explaining the behavior of illumination light and reflected light when imaging is performed by the eyeball imaging device shown in FIG. 1; 6 is a graph showing the relationship between the provisional threshold values in a large number of provisional threshold value still images generated by the first processing unit shown in FIG. FIG. 2 is a diagram schematically showing an example of an image displayed on a display based on graph data generated by the computer device shown in FIG. 1; The figure which shows roughly the whole structure of the determination system of a state of mind and body of the subject in 2nd Embodiment. FIG. 10 is a functional block diagram showing functional blocks generated in the computer device shown in FIG. 9; FIG. FIG. 10 is a functional block diagram showing functional blocks generated in the server shown in FIG. 9; FIG.

Preferred first and second embodiments and modifications of the present invention will be described below with reference to the drawings.
In the explanations of both the embodiments and the modifications, the same objects are denoted by the same reference numerals, and overlapping explanations are omitted as the case may be. In addition, the technical content described in each embodiment and modified examples can be combined with each other as long as there is no particular contradiction.

<<First Embodiment>>
FIG. 1 shows an overview of the mental and physical condition determination system (hereinafter, sometimes simply referred to as “determination system”) in this embodiment.
As shown in FIG. 1, the determination system is configured by a combination of an eye imaging device 1 and a computer device 100. FIG.
The eyeball imaging device 1 includes a camera. The computer device 100 has the function of the image processing device in the present invention. In other words, the mental and physical state determination system according to this embodiment is an example of the case where the image processing apparatus and the camera according to the present invention are separate bodies.
The judging system in this embodiment is for judging the mental and physical condition of a person. In other words, the subject is human. Of course, the subject to be determined is not limited to humans, and may be mammals other than humans. In that case, the configuration of the eyeball imaging device 1 described below (for example, the size and shape of the head of the eyeball imaging device 1 described below) should be appropriate for the mammal to be imaged. Transformed as needed.

The eyeball imaging device 1 in this embodiment picks up an image of the eyeball of a human subject, and is moving image data, which is data about a moving image in which the image of the subject's eyeball, more specifically, the state of expansion and contraction of the pupil in the eyeball is included in the image. has a function to generate and output As far as that is concerned, the eyeball imaging device 1 may be publicly known or well-known.
An eyeball imaging device 1 in this embodiment includes a grip 10 that can be held by hand, and a head 20 provided on the upper front side of the grip 10 .
The grip part 10 and the head part 20 are made of, for example, an opaque resin, although this is not a limitation. At least the head 20 should be made of an opaque material. Their interiors are hollow, and various components are housed or attached to their interiors as described below. Since the grip part 10 and the head part 20 contain parts, they function as a virtual case in which the parts are contained.
The grip part 10 has a shape that can be held with one hand, and is in the shape of a rod or a cylinder in this embodiment, although the shape is not limited to this.
The head portion 20 is a tube that can be called a hood with a substantially rectangular cross section that widens slightly toward the tip, and is made of an opaque material such as an opaque resin. An opening 21 is provided at the tip of the head 20 (the side facing the subject's face during use, the front side in FIG. 1). Although not limited to this, the opening 21 in this embodiment has a horizontally long substantially rectangular shape with four rounded corners. This eyeball imaging device 1 is configured such that the subject himself/herself or a person other than the subject, such as a doctor, holds the grip portion 10 and abuts the edge of the opening 21 at the tip of the head 20 around one of the eyes of the subject. It is used with the By pressing the edges of the openings 21 around the subject's eyes without any gaps, the resin forming the head 20 is opaque, and it is possible to create a state in which external light does not enter the inside of the head 20. there is
A switch 15 is provided at an appropriate position on the grip portion 10 or the head portion 20 , for example, on the front side of the grip portion 10 . The switch 15 is an operator for performing an input that triggers lighting of a stimulating light source, which will be described later. However, the switch 15 need not be of the push-button type if it is possible to generate an input signal that triggers lighting of the stimulating light source. More specifically, by using a sensor that detects that the edge of the opening 21 of the head 20 is pressed against the eyes of the subject, the stimulation light source is turned on regardless of the intention of the subject or the doctor. is also possible. When such a configuration is employed, it becomes unnecessary to provide the switch 15 in the eyeball imaging device 1 .

FIG. 2 shows a horizontal sectional view of the head 20 in the eye imaging device 1, and FIG. 3 shows a vertical sectional view of the entire eye imaging device 1. As shown in FIG.
A lens 11, an illumination light source 31a, a stimulation light source 31b, and a first polarizing plate 32 are provided inside the head 20 on the front side in FIG. A second polarizing plate 33 and an imaging device 12 are provided inside the head 20 on the far side in FIG.

In this embodiment, there are a plurality of illumination light sources 31a. The illumination light source 31a emits natural light as illumination light. As long as this is possible, the illumination light source 31a can be composed of an appropriate light bulb, LED, or the like. The illumination light source 31a in this embodiment is an LED, although not limited to this. The illumination light source 31a emits light with a certain degree of directivity toward the surface of the eyeball, which is an object. As will be described later, since the eyeball is positioned substantially in the center of the opening 21 during imaging, the optical axis of the light emitted from each illumination light source 31a is directed substantially toward the center of the opening 21. there is As a result, the certain range in which the eyeball is expected to be positioned at the time of imaging is illuminated with a uniform brightness to some extent by the illumination light from the illumination light source 31a. The illumination light source 31a is fixed to a board fixed to the head 20, but illustration of the board is omitted.
The wavelength of the illumination light emitted by the illumination light source 31a is not particularly limited. The illumination light source 31a may emit general white light. However, if it is important not to have a physiological effect on the subject's eyes, for example, to prevent pupil dilation caused by illumination light irradiation, the illumination light should be of an invisible wavelength. The light should be, for example, light of wavelengths in the infrared region, but is not limited to that in this embodiment. More specifically, the illumination light source 31a of this embodiment is composed of an LED that selectively emits light having a wavelength within a specific wavelength range in the infrared region. The wavelength of the light emitted by the illumination light source 31a of this embodiment is set to 900 nm or longer mainly because the light is invisible to the subject. Even if the wavelength is lengthened, the linear polarizers (for example, wire grid polarizers) used for the first polarizer 32 and the second polarizer 33 in this embodiment are used to obtain a matte image as described later. No effect. However, when the wavelength is long, it becomes difficult to perform imaging with a general imaging device 12 . Considering these points, the wavelength of illumination light should be appropriately selected in a range longer than 900 nm. Although not limited to this, the illumination light source 31a in this embodiment is always lit when the power is turned on by a switch (not shown).
As described above, the number of illumination light sources 31a in this embodiment is, but not limited to, plural. Each illumination light source 31a is located near the opening 21 of the head 20, and in this embodiment is located slightly inside the wall of the head 20 on both sides of the opening 21 in the horizontal direction in FIG. . A plurality of illumination light sources 31a located on the right and left sides of the opening 21 are arranged linearly, more precisely, in the vertical direction in FIG. 1 in this embodiment. As shown in FIG. 3, the number of illumination light sources 31a arranged in the vertical direction on the right side of the opening 21 is five, and the number on the left side of the opening 21 is the same. Of course, the number of illumination light sources 31a arranged in the vertical direction on the right side and the left side of the opening 21 need not be five, and moreover, it need not be plural.
On the other hand, the stimulation light source 31b irradiates the eye of the subject with light having a wavelength that allows the subject to perceive brightness, in other words, stimulating light that is light with a wavelength that prompts the pupil of the subject's eye to expand and contract. It is a light source for Although not limited to this, in this embodiment, they are provided above the five illumination light sources 31a arranged vertically on the right and left sides of the opening 21, respectively, as described above. However, the position of the stimulating light source 31b is not limited to this as long as the subject's eyes can be irradiated with the stimulating light. Moreover, since the stimulating light emitted from the stimulating light source 31b does not need to pass through the first polarizing plate 32, it may be positioned on the front side of the first polarizing plate 32 in FIG.
The stimulation light source 31b in this embodiment is normally not lit, but is lit when the switch 15 is pressed as described above. The time period during which the stimulating light source 31b is turned on when the switch 15 is pressed may be short. The time during which the stimulation light source 31b is turned on when the switch 15 is pressed can be determined, for example, from 0.1 seconds to 1 second. In this embodiment, although not limited to this, the time is 0.5 seconds.

In front of five illumination light sources 31a and five stimulating light sources 31b arranged in the vertical direction near the left and right ends of the head 20, there are polarized light sources that linearly polarize the illumination light and stimulating light, which are natural light that passes through them, respectively. The first polarizing plates 32, which are plates, are arranged one by one. The first polarizing plate 32 in this embodiment, as shown in FIGS. 1 and 3, is a vertically long rectangle. It is arranged near the tip of the opening 21 . In this embodiment, of the light emitted from the illumination light source 31a, all of the illumination light that contributes to imaging performed by the imaging element 12 as described later passes through the first polarizing plate 32. The width of one polarizing plate 32 is designed from that point of view. In this embodiment, the inner edge of the lens 11 is provided with a gap around the lens 11, although not limited to this, so that all illumination light that contributes to imaging by the imaging device 12 passes through the first polarizing plate 32. A donut-shaped partition wall 19 which does not transmit light is provided in the head portion 20. The partition wall 19 is in contact with the inner peripheral surface of the head portion 20 without gaps. The partition wall 19 divides the space inside the head 20 into a space in front of the lens 11 and a space behind the lens 11 . The presence of the partition wall 19 prevents light emitted from each illumination light source 31a that has not passed through the first polarizing plate 32 from reaching the space behind the lens 11 where the imaging element 12 exists. Become. In this embodiment, the stimulating light emitted from the stimulating light source 31b also passes through the first polarizing plate 32, but as described above, the stimulating light emitted from the stimulating light source 31b is not necessarily the first polarized light. It does not have to pass through the plate 32 .
The first polarizing plate 32 in this embodiment functions as a polarizing plate for light with wavelengths in the infrared region. Although not necessarily limited to this, the first polarizing plate 32 in this embodiment is a wire grid polarizing plate. A wire grid polarizer is made by arranging very thin metal wires parallel to a resin plate at predetermined intervals. act as a board. An example of the wire grid polarizer is "Asahi Kasei WGF (trademark)" manufactured and sold by Asahi Kasei E-Materials Corporation. Note that the second polarizing plate 33 is the same as above.
As described above, the illumination light that has passed through the first polarizing plate 32 becomes linearly polarized light. For example, the plane of polarization of illumination light, which is linearly polarized light that has passed through the first polarizing plate 32, is horizontal in FIG.

The lens 11 is for forming an image on the imaging element 12 of the reflected light generated by the illumination light being reflected by the object, which is the eyeball. As long as it is possible, the lens 11 does not have to be a single piece, and may include necessary optical parts other than the lens 11 . Further, the lens 11 may have the function of enlarging the image, or may have other functions.
The imaging device 12 captures the reflected light and performs imaging. The imaging device 12 of this embodiment may be of any type as long as it can perform imaging with light having a wavelength in the infrared region. The imaging device 12 can be configured by, for example, a CCD or a CMOS. The imaging device 12 generates image data obtained by imaging. An image captured by the imaging device 12 is a moving image. The imaging device images an eyeball and generates moving image data, which is moving image data. The subject's eyeball is reflected in the moving image based on the moving image data, and the enlargement and contraction of the pupil can be understood. Moving image data is a sequence of a large number of still image data, which are data about still images.
The imaging element 12 is connected to the circuit 13 by a connection line 12a. The circuit 13 receives image data generated by the imaging device 12 from the imaging device 12 via a connection line 12a. Prior to outputting the video signal to the outside, the circuit 13 performs necessary processing such as brightness adjustment and analog/digital conversion if necessary.
Circuit 13 is connected to output terminal 14 via connection line 13a. The output terminal 14 connects to the computer device 100 via a cable 16 (not shown). The cable 16 and the output terminal 14 may be connected in any way, but it would be convenient to use a standardized connection method such as USB, for example. It should be noted that the moving image data generated by the eye imaging device 1 does not need to be output to the computer device 100 by wire as in this embodiment. When moving image data is output wirelessly, the eye imaging device 1 is provided with a known or well-known communication mechanism for communicating with the computer device 100 by Bluetooth (trademark), for example, instead of the output terminal 14. It will be. Of course, it is also possible to make the connection with the computer device 100 both wired and wireless.
Circuit 13 is also connected by connection 15a to switch 15 described above. The circuit 13 that receives the input signal from the switch 15 turns on the stimulation light source 31b.

Although the second polarizing plate 33 is made of the same material as the first polarizing plate 32 as described above, its function is different from that of the first polarizing plate 32 . The second polarizing plate 33 is the light reflected by the surface of the eyeball out of the reflected light generated by the reflection of the illumination light linearly polarized by the first polarizing plate 32 on the surface of the object, which is the eyeball. It has the function of blocking the linearly polarized light component contained in the surface reflected light.
When viewed with the optical axes of the illumination light and the reflected light as axes, the first polarizing plate 32 and the second polarizing plate 33 are oriented such that the planes of polarization of the linearly polarized light passing through them are orthogonal to each other. ing. In this embodiment, when the natural light passes through the second polarizing plate 33 in the same direction as the reflected light, the plane of polarization of the linearly polarized light generated by passing through the second polarizing plate 33 is the vertical direction in FIG. It is designed to be
Reflected light from the eyeball passes through the lens 11 and further through the second polarizing plate 33 and is imaged by the imaging element 12 . Therefore, the light that contributes to imaging by the imaging element 12 is only the component of the reflected light that can pass through the second polarizing plate 33 . The second polarizing plate 33 may be present on the optical path of the reflected light between the imaging element 12 and the object. is composed of a plurality of lenses, it may be positioned between the plurality of lenses.

Next, the computer device 100 will be described.
The computer device 100 is a general computer, and may be a commercially available one. Computing device 100 in this embodiment is, but is not limited to, a commercially available tablet. Note that the computer device 100 does not necessarily have to be a tablet as long as it has the configuration and functions described below, and may be a smartphone, a notebook computer, a desktop computer, or the like. Even in the case of a smart phone or a personal computer, the computer device 100 may be commercially available. Examples of tablets include the iPad (trademark) series manufactured and sold by Apple Japan LLC. Examples of smartphones include the iPhone (trademark) series manufactured and sold by the company.

The appearance of computer device 100 is shown in FIG.
The computer device 100 has a display 101 . The display 101 is for displaying still images or moving images, or generally both of them, and a publicly known one can be used. The display 101 is, for example, a liquid crystal display. Computing device 100 also includes an input device 102 . The input device 102 is used by the user to make desired inputs to the computer device 100 . The input device 102 can use a publicly known or well-known device. Although the input device 102 of the computer device 100 in this embodiment is of a button type, it is not limited to this, and it is also possible to use a numeric keypad, keyboard, trackball, mouse, or the like. In particular, if the computer device 100 is a notebook computer or desktop computer, the input device 102 may be a keyboard, mouse, or the like. Also, when the display 101 is a touch panel, the display 101 also functions as the input device 102, which is the case in this embodiment.

A hardware configuration of the computer device 100 is shown in FIG.
The hardware includes a CPU (central processing unit) 111 , a ROM (read only memory) 112 , a RAM (random access memory) 113 and an interface 114 , which are interconnected by a bus 115 .
The CPU 111 is an arithmetic device that performs arithmetic operations. The CPU 111 executes a process described later by executing a computer program recorded in the ROM 112 or the RAM 113, for example. Although not shown, the hardware may include a HDD (hard disk drive) or other large-capacity recording device, and the computer program may be recorded on the large-capacity recording device.
The computer program referred to here includes at least a computer program for causing this computer device 100 to function as one of the determination devices of the present invention. This computer program may be pre-installed in the computer device 100 or may be installed after the computer device 100 is shipped. The installation of this computer program in the computer device 100 may be performed via a predetermined recording medium such as a memory card, or may be performed via a network such as a LAN or the Internet.
The ROM 112 records computer programs and data necessary for the CPU 111 to execute processing described later. The computer program recorded in the ROM 112 is not limited to this, and if the computer device 100 is a tablet, a computer program and data necessary for making the computer device 100 function as a tablet, for example, for executing e-mail. is recorded. The computer device 100 is also capable of browsing homepages on the Internet, and is equipped with a known or well-known web browser for enabling this.
A RAM 113 provides a work area necessary for the CPU 111 to perform processing. In some cases, the computer programs and data described above may be recorded.
The interface 114 exchanges data between the CPU 111, the RAM 113, and the like connected via the bus 115 and the outside. The display 101 and the input device 102 are connected to the interface 114 . Data about the operation content input from the input device 102 is input to the bus 115 from the interface 114 . Also, as is well known, image data for displaying an image on the display 101 is output from the interface 114 to the display 101 . The interface 114 also receives moving image data from the cable 16 described above (more precisely, from an input terminal (not shown) of the computer device 100 connected to the cable 16). Moving image data input from the cable 16 is sent from the interface 114 to the bus 115 .

As the CPU 111 executes the computer program, functional blocks as shown in FIG. 5A are generated inside the computer device 100 . Note that the functional blocks described below may be generated by the function of the above-described computer program alone for causing the computer device 100 to function as one of the image processing devices of the present invention, but the above-described computer program, It may be generated in cooperation with an OS installed in the computer device 100 or other computer programs.
In the computer device 100, an input unit 121, a binarization processing unit 122, a graph generation unit 123, and an output unit 124 are generated in relation to the functions of the present invention.

Input unit 121 receives data from interface 114 . Data received by the input unit 121 are operation content data input from the input device 102 and moving image data input from the cable 16 . When the input unit 121 receives the data via the interface 114 , it sends the data to the binarization processing unit 122 .
The binarization processing unit 122 generates binary moving image data based on the moving image data received from the input unit 121 . The binarization processing unit 122 generates binary moving image data by performing binarization processing on moving image data or part thereof. In the binarization processing section 122, a first processing section 122B1 and a second processing section 122B2 are generated. Details of the functions of the binarization processing unit 122 and the binary moving image data generated by the binarization processing unit 122, including the functions of the first processing unit 122B1 and the second processing unit 122B2, will be described later. The binarization processing unit 122 sends the generated binary moving image data to the graph generation unit 123 . Also, in some cases, binary moving image data can be sent from the binarization processing unit 122 to the output unit 124 .
When the binary moving image data is received, the graph generation unit 123 generates graph data, which is data of a graph showing the expansion and contraction of the pupil, based on the received binary moving image data. Details of the functions of the graph generation unit 123 and the graph data generated by the graph generation unit 123 will be described later. The graph generator 123 sends the generated graph data to the output unit 124 .
The output unit 124 outputs image data of an image based on graph data or, in some cases, an image based on binary moving image data to the display 101 via the interface 114 . The output unit 124 has a function of generating image data for displaying an image on the display 101 based on graph data or binary moving image data. An image corresponding to the graph data or the binary moving image data will be displayed on the display 101 that has received the image data.

Next, the method of use and operation of the determination system described above will be described.
When using the determination system, first, as described above, the eyeball imaging device 1 and the computer device 100 are connected with the cable 16 . When both are connected, in this embodiment, the illumination light source 31a of the eyeball imaging device 1 lights up. A power switch may be provided in the eyeball imaging device 1, and the illumination light source 31a may be turned on by turning on the power switch.
In this state, the subject, a doctor, or the like grips the grip portion 10 and presses the edge of the opening 21 in the head 20 around one eye of the subject. At this time, care should be taken not to leave a gap between the periphery of the subject's eyes and the edge of the opening 21 so that external light does not enter the inside of the head 20 from the periphery of the opening 21 .

Illumination light emitted from the illumination light source 31 a passes through the first polarizing plate 32 , is reflected by the eyeball positioned substantially in the center of the opening 21 , passes through the lens 11 and the second polarizing plate 33 , and reaches the image sensor 12 . up to. The imaging device 12 picks up reflected light from the eyeball and takes an image. As a result, the imaging device 12 generates moving image data, which is data about moving images in which the subject's eyeballs are captured in a state in which the state of dilation and contraction of the pupil can be seen. A large number of still image data, which are data of still images including the eyeball of one eye of the subject, are connected. As described above, the wavelength of the illumination light used in this embodiment is in the infrared region, which is invisible to the subject's eyes.
The moving image data is sent to the circuit 13 via the connection line 12a, and after being subjected to appropriate processing (such as brightness adjustment) in the circuit 13, reaches the output terminal 14 via the connection line 13a. The output terminal 14 is connected to the computer device 100 via the cable 16 . The moving image data may be constantly sent from the eyeball imaging device 1 to the computer device 100, but in this embodiment, the moving image data is not always sent from the eyeball imaging device 1 to the computer device 100. do not have. As will be described later, the moving image data is sent from the eyeball imaging device 1 to the computer device 100 only for the necessary time period.

FIG. 6 conceptually shows what kind of reflected light is imaged by the imaging element 12 by the illumination light from the illumination light source 31a.
FIG. 6(A) shows surface reflected light, which is reflected light from the outermost surface of the eyeball, and FIG. 6(B) shows the behavior of internally reflected light, which is reflected light that enters the eyeball slightly from the surface of the eyeball and is reflected. showing. In addition, the straight line drawn in the thick ○ mark conceptually indicates the direction of the plane of polarization of the illumination light or the reflected light in the relevant part, and the radial lines drawn in the ○ mark indicate This indicates that the linear polarization of the illumination light or the reflected light in the portion is disturbed (for example, it is natural light).
The illumination light emitted from the illumination light source 31 a passes through the first polarizing plate 32 . The illumination light that has passed through the first polarizing plate 32 becomes linearly polarized light. The plane of polarization of the linearly polarized light, which is the illumination light in that case, is the horizontal direction in FIG. The steps up to this point are common to FIGS. 6A and 6B.
The illumination light, which is linearly polarized light that has passed through the first polarizing plate 32, hits the eyeball X and becomes reflected light from the eyeball X. As shown in FIG. Of the light reflected by the surface of the eyeball X, the surface-reflected light ideally maintains its polarization state. The surface-reflected light, which is linearly polarized light, is blocked by the second polarizing plate 33 , which is set so that the direction of the plane of polarization of the linearly polarized light generated when natural light passes through is orthogonal to that of the first polarizing plate 32 . (Fig. 6(A)).
Internally reflected light, on the other hand, has its polarization state disturbed. Of the light contained in the internally reflected light, the part that oscillates in a direction perpendicular to the plane of polarization of the linearly polarized light contained in the surface reflected light passes through the second polarizing plate 33, so about half of it is captured. It reaches the element 12 (FIG. 6(B)).
As a result, only the internally reflected light is used for the imaging element 12 to capture an image using the illumination light from the illumination light source 31a. What this means is that the image generated by the imaging device 12 is a matte image. In other words, the moving image captured by the imaging device 12 does not reflect the illumination light source 31a itself, and the state of the eyeball itself is accurately captured. This is very useful for grasping the size of the pupil from the moving image, for example, by image processing.

The doctor or the like presses the switch 15 provided on the grip part 10 while pressing the edge of the opening 21 in the head 20 of the eye imaging device 1 around one eye of the subject. Even in this state, the subject's eyes are not dark-adapted. When the switch 15 is pressed, the input signal generated by the switch 15 is sent to the circuit 13, and the circuit 13 causes the stimulating light source 31b to irradiate the stimulating light. The stimulating light is applied for 0.5 seconds, although in this embodiment it is not limited to this, as described above.
The stimulating light emitted from the stimulating light source 31b passes through the first polarizing plate 32, is reflected by the eyeball positioned substantially in the center of the opening 21, passes through the lens 11 and the second polarizing plate 33, and reaches the imaging element 12. up to. Of the stimulating light emitted from the stimulating light source 31b and reflected by the eyeball, only the internally reflected light reaches the imaging element 12 as in the case of the illumination light. However, since the imaging element 12 does not pick up light of the wavelength of the stimulating light in the first place, the imaging element 12 does not pick up the stimulating light or an image based on the stimulating light.
The imaging device 12 is adapted in this embodiment to always generate moving image data as described above, for example whenever the eye imaging device 1 is connected to the computer device 100 by the cable 16 . This moving image data is always sent to the circuit 13 . The circuit 13 incorporates an overwrite recording unit (not shown), and the overwrite recording unit always overwrites moving image data while storing it. Such an overwrite recording unit can be configured by, for example, a well-known ring buffer. As described above, of the moving image data generated by the imaging device 12 , the moving image data corresponding to the amount necessary for the computer device 100 to perform the determination described later is sent to the computer device 100 . In the overwrite recording section, moving image data corresponding to a longer time moving image than the moving image corresponding to the necessary amount of moving image data sent to the computer device 100 is recorded. For example, if the moving image corresponding to the moving image data sent to the computer device 100 has a maximum length of 6 seconds, the moving image data recorded in the overwrite recording unit may be changed to a length longer than the past 6 seconds. 10 seconds worth of moving image data. Of course, it is not limited to this, but for the time being, in this embodiment, the moving image corresponding to the moving image data sent from the eye imaging device 1 to the computer device 100 shall be six seconds at the longest.
As described above, when switch 15 is pressed, the input signal generated by switch 15 is sent to circuit 13 . The circuit 13 receiving this turns on the stimulating light source 31b as described above, and at the moment when the switch 15 is pressed, the moving image data of the moving image for the past one second recorded in the overwrite recording unit, Then, the moving image data of the moving image for 5 seconds after the moment when the switch 15 is pressed is read from the overwrite recording section as the moving image data for 6 seconds.
In other words, in this manner, when the switch 15 is pressed, the circuit 13, with reference to the moment when the stimulating light source 31b starts irradiating the stimulating light, which is substantially the same as the moment when the switch 15 is pressed, Moving image data for a total of 6 seconds, i.e., 1 second before and 5 seconds after, is generated and sent to the output terminal 14 via the connection line 13a. The moving image data for 6 seconds is sent from the eye imaging device 1 to the computer device 100 via the cable 16 .
In order to cause the circuit 13 to generate moving image data for a total of 6 seconds, i.e., 1 second before and 5 seconds after the moment when the stimulating light source 31b starts irradiating stimulating light, the above-described An overwrite record part is not required. For example, the circuit 13 causes the stimulating light source 31b to start irradiating the stimulating light one second after the moment when the above-described input signal is input from the switch 15 to the circuit 13, and the circuit 13 receives the input signal from the switch 15. If the moving image data of the moving image for 6 seconds from the moment the stimulating light source 31b starts emitting the stimulating light is used as the reference, Moving image data for a moving image for a total of 6 seconds, i.e., 1 second followed by 5 seconds, is generated.

In any case, the moving image data of the moving image having a length of 6 seconds in this embodiment is sent from the eye imaging device 1 to the computer device 100 via the cable 16 . The moving image data is sent from the interface 114 to the input unit 121 in the computer device 100 .
The input unit 121 sends the received moving image data to the binarization processing unit 122 . Upon receiving the moving image data, the binarization processing unit 122 generates binary moving image data based on the moving image data.

Processing when the binarization processing unit 122 generates binary moving image data from moving image data is as follows.
When the binarization processing unit 122 receives the moving image data, among a large number of still image data included in the moving image data, those to be binarized are selected by the first binarization processing unit 122 . It is sent to the processing unit 122B1. It is possible to appropriately determine which of the large number of still image data included in the moving image data is to be sent to the first processing unit 122B1. is at least plural.
When receiving the still image data, the first processing unit 122B1 binarizes the still image based on the still image data using a plurality of mutually different provisional threshold values as threshold values to obtain a plurality of provisional threshold value still images. Each of the plurality of provisional threshold still images is made such that the density of each pixel is maximized or minimized. There are multiple temporary thresholds as described above. If the density that can be assigned to each pixel is 256 steps from 0 to 255 (although not limited to this, in this embodiment, the density assigned to each pixel of a still image is 256 steps from 0 to 255). ), and a maximum of 256 temporary threshold values from density 0 to density 255 can be adopted. However, even in this case, if the accuracy of the specific threshold value to be determined later is not so high, only even density values such as 2, 4, 6, . . . It is also possible to use a threshold value, or to use only density values that are multiples of 3 as provisional threshold values. If the range of density values in which the specific threshold is generally expected to be included (the range that can include the range of S described later) is clear, the range, for example, the density is 256 steps from 0 to 255 It is also possible to use only densities in the range of 50 to 200 as thresholds. In this case, the number of provisional threshold still images is less than 256. In this embodiment, 256 values from density 0 to density 255 are used as provisional thresholds. Then, the first processing unit 122B1 generates 256 provisional threshold still images. The first processing unit 122B1 transfers the provisional threshold still image data for the 256 provisional threshold still images to the second processing unit 122B2.

Based on a plurality of (256 in this embodiment) provisional threshold still images generated by the first processing unit 122B1, the second processing unit 122B2 determines whether the density in the provisional threshold still image is maximum even if the provisional threshold changes. A temporary threshold range in which the area of the range does not change is specified, and a specific threshold is determined so as to be included in the range. Then, the second processing unit 122B1 determines a threshold included in the range as a specific threshold, and uses the specific threshold among a plurality of temporary threshold still images created from still images based on the same still image data. Then, the binary still image is decided as the binary still image.
In the provisional threshold value still image generated by the first processing unit 122B1 as described above with the threshold value set to 255, among the pixels of the original still image, only the pixels having the density value of 255 have the maximum density value (255 ), and the density values of all other pixels are minimal (0). In the temporary threshold still image with a threshold of 254, among the pixels of the original still image, only the pixels with density values of 255 and 254 have the maximum density value (255), and all other pixels have density values of 255 and 254. is the minimum (0). In the temporary threshold still image with a threshold of 253, among the pixels of the original still image, only the pixels with density values between 255 and 253 have the maximum density value (255), and all other pixels have density values of 255 and 253. is the minimum (0). In other words, when the temporary threshold value is lowered from the maximum density value, the range (area) where the density value is maximum gradually increases.
However, this relationship is not maintained even if the provisional threshold value is continuously reduced, and even if the provisional threshold value is further reduced, the range (area) where the density value becomes maximum does not increase. This state is a state in which the density value of only the pixels existing in the pupil portion in the original still image is maximized.
This state continues for a while, but when the temporary threshold value is reduced to a certain extent or more, the range (area) where the density value becomes maximum begins to increase.
FIG. 7 schematically shows the size of the provisional threshold value and the area of the range where the density value in the provisional threshold still image is maximum. A range indicated by S in FIG. 7 is a range in which the area of the range where the density value in the temporary threshold still image is maximum does not change even if the temporary threshold changes. The second processing unit 122B2 identifies the range of S and determines a specific threshold within the range of S. Although any specific threshold may be determined within the range of S, the second processing unit 122B2 in this embodiment sets the , more preferably, the specific threshold is determined so as to be included in a 30% range (centered at the center) including the center of the range of S . In this embodiment, the threshold in the middle of the range of S is determined as the provisional threshold. If there is a fraction in the specific threshold value, appropriate processing such as rounding up or rounding down the fraction may be performed. Which threshold in the range of S is the specific threshold (for example, the threshold corresponding to the middle value of the range of S is the specific threshold, or the threshold corresponding to the value 10% below the middle of the range of S is specified) ) is the same for all cases of obtaining a specific threshold from still image data to be binarized, or converting still image data into binary still image data. should.
As described above, the specific threshold value is determined by the second processing unit 122B2.
Next, the second processing unit 122B2 selects a provisional threshold still image binarized with a specific threshold among a large number of provisional threshold still images created from one still image as an image obtained by binarizing the still image. Binary still image is determined.

By repeating the above processing by the first processing unit 122B1 and the second processing unit 122B2, still images to be binarized are converted into binary still images one after another. That is, the still image to be binarized is changed into a large number of provisional threshold still images by the first processing unit 122B1, and one of them is selected as one binary still image by the second processing unit 122B2. of.
The binarization processing unit 122 may binarize all still images included in the moving image data, or may binarize a plurality of still images included in the moving image data. You may perform processing of conversion. For example, of the moving image data for 6 seconds, the still image for the first 0.5 seconds may not be binarized. In this embodiment, the binarization processing unit 122 performs binarization processing on still images corresponding to all six seconds of moving image data. A binarized still image is a binarized still image, and data of the binarized still image is binarized still image data. Binary moving image data is a series of binary still image data.
The binarization processing unit 122 sends the generated binary moving image data to the graph generation unit 123 . The binarization processing unit 122 may send the binary still image data to the graph generation unit 123 after all the binary still image data is prepared, or may send the binary still image data to the graph generation unit 123 each time the binary still image data is generated. You may send to the graph generation part 123 one by one. In any case, the graph generator 123 receives the binary moving image data.

When the binary moving image data is received, the graph generation unit 123 generates graph data, which is data of a graph showing the expansion and contraction of the pupil, based on the received binary moving image data. As described above, each still image has already been binarized, and the range where the density is maximum in each still image corresponds to the range of the pupil reflected in the original still image. there is
Therefore, when the area of the range in which the density is maximized in each binary still image is obtained, it corresponds to the size of the pupil in each still image. For example, the graph generation unit 123 generates graph data, which is graph data in which the horizontal axis is time and the vertical axis is the pupil size (for example, area) at that time.
The graph generator 123 sends the generated graph data to the output unit 124 .
The output unit 124 outputs image data for an image based on graph data to the display 101 via the interface 114 . Output unit 124 generates image data for displaying an image on display 101 based on the graph data, and outputs the image data to display 101 . An image corresponding to the graph data is displayed on the display 101 that receives the image data. An example of the image is shown in FIG. In FIG. 8, the horizontal axis corresponds to time, and the vertical axis corresponds to the size (area) of the pupil. The state of expansion and contraction of the pupil is expressed as a line graph. Although not limited to this, in this embodiment, the graph generation unit 123 generates graph data representing a state of expansion and contraction of the pupil as a line graph. One division on the horizontal axis is one second. There are shaded time zones on the horizontal axis. The time zone is a time zone during which stimulation light is emitted from the stimulation light source 31b. When the eyeball receives the stimulating light, the pupil shrinks and then gradually approaches its original size. Based on this graph, it is possible to determine which of the sympathetic nerves and parasympathetic nerves is dominant in the subject, and determine the state of mind and body of the subject.
For example, it is possible to display a binary moving image based on binary moving image data on the display 101 based on an input from the input device 102 . In that case, the binary moving image data is sent from the binarization processing unit 122 to the output unit 124, and the output unit 124 outputs image data for displaying the moving image based on the binary moving image data on the display 101. Generate. When the image data is output to the display 101 via the interface 114, the display 101 displays a binary moving image. As is clear from the above description, in both this embodiment and the present invention, the output of binary moving image data from the image processing device to another device (for example, a display) may be performed, but is not essential.
Although this system for judging the mental and physical conditions of a subject is called a judging system, it only displays graphs necessary for judging on the display 101 . For example, the computing device 100 can be further provided with a function for determining the subject's mental and physical condition from the graph without human intervention.

<Modification 1>
The psychosomatic state determination system of the subject of Modification 1 is generally the same as that of the first embodiment.
The determination system of Modification 1 also includes the eye imaging device 1 and the computer device 100, as in the case of the first embodiment.
The difference between the determination system of Modification 1 and the determination system of the first embodiment is that the binarization processing unit 122 that existed in the computer device 100 in the first embodiment is replaced with It exists in the eyeball imaging device 1 . Naturally, in Modification 1, the binarization processing unit 122 included in the computer device 100 in the first embodiment is omitted.
In the first embodiment, from the moving image data generated by the imaging device 12, the circuit 13 generates moving image data corresponding to the required length of the moving image, which is six seconds in the first embodiment. The binarization processing unit 122 in the computer device 100 that receives the moving image data generates binary moving image data based on the moving image data. Graph data is generated based on the value moving image data.
On the other hand, in Modification 1, the circuit 13 generates, from the moving image data generated by the imaging device 12, moving image data corresponding to the moving image for 6 seconds in the first embodiment, and this Binary moving image data is generated based on the moving image data by the binarization processing unit 122 that is also present in the eyeball imaging device 1 . That is, in Modification 1, processing up to the generation of binary video data is performed within the eye imaging device 1 . Then, the binary moving image data generated by the eyeball imaging device 1 is sent from the eyeball imaging device 1 to the computer device 100 via the cable 16, for example. In the computer device 100, the binary moving image data received by the computer device 100 is sent to the graph generator 123 via the interface 114, and the graph generator 123 generates graph data based on the pupil dilation data. become. Subsequent processing is the same as in the first embodiment.
The functions of the binarization processing unit 122 and the graph generation unit 123, the contents of the binary moving image data, and the graph data need not be changed between the first embodiment and the first modification.
Note that the binarization processing unit 122 that exists in the eyeball imaging device 1 in Modification 1 may exist in the circuit 13 or may be provided in the latter stage of the circuit 13 . The binarization processing unit 122 may be, for example, a functional block generated by a computer program, as in the first embodiment. In that case, the circuit 13 may have a hardware configuration as shown in FIG.
Furthermore, the graph generator 123 can also be provided in the eye imaging device 1 instead of in the computer device 100 . In this case, graph data is output from the eyeball imaging device 1 . In that case, the graph data would be sent to the display 101 via the computing device 100 or directly without the computing device 100 .

<<Second embodiment>>
The psychosomatic state determination system of the subject of the second embodiment is basically the same as that of the first embodiment.
The determination system of the second embodiment also includes the eye imaging device 1 and the computer device 100 as in the case of the first embodiment. However, the determination system of the second embodiment includes a server 200 capable of communicating with the computer device 100 via the network 300, as shown in FIG. Network 300 is, for example, the Internet.

The eye imaging device 1 in the second embodiment can be exactly the same as the eye imaging device 1 in the first embodiment, but is not limited to this in this embodiment. In other words, what the eyeball imaging device 1 of the second embodiment outputs is moving image data for 6 seconds, although not limited to this, as described above.

On the other hand, a part of the computer device 100 in the second embodiment is clearly different from the computer device 100 in the first embodiment.
First, the computer device 100 in the second embodiment has a communication function. More specifically, the computer device 100 in the second embodiment can communicate with the server 200 via a network 300 such as the Internet.
The hardware configuration of the computer device 100 in the second embodiment is the same as that shown in FIG. It is connected to a transmitting/receiving mechanism (not shown) which is a known means for communicating with. The functionality of the transceiver mechanism allows computing device 100 to transmit data over network 300 and to receive data over network 300 . Transmission and reception of data via the network 300 may be performed by wire or wirelessly. For example, if computing device 100 is a smart phone or tablet, such communication would typically be wireless. To the extent that is possible, the configuration of the transceiver mechanism may be known or known. Data received by the transceiver from the network 300 is received by the interface 114, and data passed from the interface 114 to the transceiver is sent by the transceiver to the server 200 via the network 300. It's becoming
Also, functional blocks are generated in the computer device 100 of the second embodiment in the same manner as in the first embodiment, and the functional blocks include the binarization processing unit 122 as shown in FIG. not exist. In this point as well, the computer device 100 of the second embodiment differs from that of the first embodiment.

The server 200 may be any known or known or commercially available server device. The hardware configuration may be very general, and may be, for example, as shown in FIG. 4, which is the case in this embodiment. However, the server 200 is normally equipped with a HDD or other large-capacity recording medium.
The server 200 can also communicate via the network 300 . The server 200 has a known or well-known transmission/reception mechanism similar to that of the computer device 100 , and can transmit and receive data to and from the computer device 100 via the network 300 . An interface provided in the server 200 is connected to a transmission/reception mechanism (not shown). The functionality of the transceiver mechanism allows the server 200 to transmit data over the network 300 and receive data over the network 300 . Data received by the transceiver mechanism from the network 300 is received by the interface, and data passed from the interface to the transceiver mechanism is sent by the transceiver mechanism to the outside via the network 300 to the computer device 100. It has become.
By executing a computer program in the server 200, functional blocks as shown in FIG. 11 are generated. Functional blocks generated in the server 200 in the second embodiment are an input unit 221 , a binarization processing unit 222 , and an output unit 224 .
The input unit 221 receives moving image data received from the computer device 100 via the network 300 by the transmitting/receiving mechanism of the server 200 via the interface. The input unit 221 sends the received moving image data to the binarization processing unit 222 .
The binarization processing unit 222 has the same function as the binarization processing unit 122 in the first embodiment, and converts the received moving image data into binary moving image data as in the first embodiment. It is designed to generate The binarization processing unit 222 includes a first processing unit and a second processing unit ( (both are not shown), and these functions enable binary moving image data to be generated from moving image data. The binarization processing unit 222 sends the generated binary moving image data to the output unit 224 .
The output unit 224 sends the binary moving image data to the transmission/reception mechanism of the server 200 via an interface. The transmitting/receiving mechanism of the server 200 that has received it returns the binary moving image data to the computer device 100 via the network 300 .

The usage method and operation of the determination system of the second embodiment will be described.
First, the usage and operation of the eye imaging device 1 are the same as in the first embodiment. In the eyeball imaging device 1, the circuit 13 generates moving image data corresponding to the required length of the moving image, which is 6 seconds in the first embodiment, based on the moving image data generated by the imaging device 12. , the eyeball imaging device 1 sends the generated moving image data to the computer device 100 via the cable 16, for example.
Such moving image data is sent to the input section 121 via the interface 114 and sent from the input section 121 to the output section 124 .
The output unit 124 sends the moving image data to the transmission/reception mechanism of the computer device 100 via the interface 114 . The transmission/reception mechanism of computer device 100 sends moving image data to server 200 via network 300 .

The moving image data sent from the computer device 100 is received by the transmission/reception mechanism of the server 200 . The transmission/reception mechanism of the server 200 sends the moving image data to the input unit 221 via the interface.
The input unit 221 sends the received moving image data to the binarization processing unit 222 .
The binarization processing unit 222 processes the received moving image data in the same manner as the binarization processing unit 122 of the first embodiment has performed to generate binary moving image data. The binarization processing unit 222 sends the generated binary moving image data to the output unit 224 .
The output unit 224 sends the binary moving image data to the transmission/reception mechanism of the server 200 via the interface. The transmitting/receiving mechanism of the server 200 that has received it returns the binary moving image data to the computer device 100 via the network 300 .

The computer device 100 receives the binary moving image data through its transmission/reception mechanism.
The binary moving image data sent from the server 200 is received by the transmission/reception mechanism of the computer device 100 . The transmission/reception mechanism of the computer device 100 sends the binary moving image data to the input unit 121 via the interface 114 .
The input unit 121 sends the received binary moving image data to the graph generation unit 123 .
Subsequent processing is the same as in the first embodiment, and an image having content corresponding to the graph data is displayed on the display 101 of the computer device 100 .

In short, in the second embodiment, instead of omitting the binarization processing unit 122 that existed in the computer device 100 in the first embodiment, the binarization processing unit 122 having the same function as the binarization processing unit 122 is installed in the server 200. A processing unit 222 is provided so that the server 200 is provided with the function of converting moving image data into binary moving image data, which was covered by the binarization processing unit 122 of the computer device 100 in the first embodiment. be.

It should be noted that the server 200 in the second embodiment may serve not only the function of converting moving image data into binary moving image data, but also the function of generating graph data based on the binary moving image data. In that case, the graph generator 123 in the computer device 100 in the first embodiment will be omitted, and a graph generator corresponding to the graph generator 123 will be generated in the server 200 .

1 eyeball imaging device 10 grip part 11 lens 12 imaging element 13 circuit 16 cable 19 partition 20 head 31a illumination light source 31b stimulation light source 32 first polarizing plate 33 second polarizing plate 100 computer device 121 input unit 122 binarization processing Section 122B1 First Processing Section 122B2 Second Processing Section 123 Graph Generation Section 124 Output Section 200 Server 221 Input Section 222 Binary Processing Section 224 Output Section 300 Network

Claims (6)

A moving image, which is a sequence of a large number of still image data, which is data about a still image including the eyeball of one eye of a subject, generated by a predetermined camera capable of capturing moving images. a moving image data reception unit that receives moving image data that is image data;
binarizing at least a plurality of the still image data included in the moving image data using a specific threshold value to obtain a second image obtained by binarizing the still image; a binarization processing unit for generating binary moving image data, which is moving image data, which is obtained by connecting a large number of binary still image data, which is value still image data;
An image processing device comprising
The binarization processing unit
Each of the still images based on the plurality of still image data to be binarized is binarized with a plurality of different thresholds, which are provisional thresholds, so that the density of each pixel is maximized or minimized. a first processing unit that generates a plurality of temporary threshold still images;
Based on the plurality of the provisional threshold still images generated by the first processing unit, the provisional threshold still image does not change the area of the range where the density in the provisional threshold still image is maximum even if the provisional threshold is changed. A threshold range is specified, a threshold included in the range is determined as the specific threshold, and one of the plurality of temporary threshold still images binarized using the specific threshold is the binary still image. a second processing unit for determining as an image;
is equipped with
Image processing device. The second processing unit is configured to determine the specific threshold so as to be included in a 50% range including the center of the range of the temporary threshold,
The image processing apparatus according to claim 1. integral with the camera;
The image processing apparatus according to claim 1. The moving image data reception unit receives the moving image data from the camera via a predetermined network,
The image processing apparatus according to claim 1. A moving image, which is a sequence of a large number of still image data, which is data about a still image including the eyeball of one eye of a subject, generated by a predetermined camera capable of capturing moving images. A method executed by a computer including a moving image data receiving unit that receives moving image data that is image data,
the computer executes
binarizing at least a plurality of the still image data included in the moving image data using a specific threshold value to obtain a second image obtained by binarizing the still image; a binarization process for generating binary moving image data, which is moving image data, which is a sequence of a large number of binary still image data, which is value still image data;
contains
In the binarization process,
Each of the still images based on the plurality of still image data to be binarized is binarized with a plurality of different thresholds, which are provisional thresholds, so that the density of each pixel is maximized or minimized. a first process for generating a plurality of temporary threshold still images;
Based on the plurality of provisional threshold still images generated in the first processing step, the provisional threshold still image does not change the area of the range where the density in the provisional threshold still image is maximum even if the provisional threshold is changed. A threshold range is specified, a threshold included in the range is determined as the specific threshold, and one of the plurality of temporary threshold still images binarized using the specific threshold is the binary still image. a second process of determining as an image;
run the
Image processing method. A moving image, which is a sequence of a large number of still image data, which is data about a still image including the eyeball of one eye of a subject, generated by a predetermined camera capable of capturing moving images. A computer equipped with a moving image data receiving unit for receiving moving image data, which is image data,
binarizing at least a plurality of the still image data included in the moving image data using a specific threshold value to obtain a second image obtained by binarizing the still image; a binarization processing unit that generates binary moving image data, which is moving image data, which is a sequence of a large number of binary still image data, which is value still image data;
and
The binarization processing unit
Each of the still images based on the plurality of still image data to be binarized is binarized with a plurality of different thresholds, which are provisional thresholds, so that the density of each pixel is maximized or minimized. a first processing unit that generates a plurality of temporary threshold still images;
Based on the plurality of the provisional threshold still images generated by the first processing unit, the provisional threshold still image does not change the area of the range where the density in the provisional threshold still image is maximum even if the provisional threshold is changed. A threshold range is specified, a threshold included in the range is determined as the specific threshold, and one of the plurality of temporary threshold still images binarized using the specific threshold is the binary still image. a second processing unit for determining as an image;
is equipped with
A computer program for functioning as an image processing device.

Images