JP2020072901A - Stress evaluation program for portable terminal, and portable terminal having the program - Google Patents

Abstract

To provide a technique for measuring a state of stress by another objective symptom correlated with stress in a portable terminal device.SOLUTION: A program is developed in order to solve the problem so as to record an animation ON step of turning on an animation photographing function of a portable terminal, a light ON step of turning on light provided on a photographing side of a portable terminal, a pupil recognition step of recognizing pupils of eyes of an animal including a human from among images being photographed, a pupil change computation step for computing a change in a pupil as a time change of the degree of opening of a recognized pupil, in a portable terminal device so as to be readable.SELECTED DRAWING: Figure 49

Description

  The present invention relates to a program for evaluating stress using a mobile terminal.

  Stress is the cause of various illnesses. However, in general, stress is not visually recognizable, and it is not easy to measure it using a device or the like.

  Therefore, a technique for measuring the state of stress based on objective symptoms having a correlation with stress has been desired.

  As such a technique, for example, there is a technique as disclosed in Patent Document 1.

Japanese Patent Laid-Open No. 2005-143684

  This patent document takes advantage of the fact that the degree of stress and relaxation has a correlation in the relationship with the degree of pupillary light reaction, and the pupillary reaction when light is applied to the eyeball is photographed and analyzed. Then, the idea is to measure the degree of stress. With this idea, the purpose of measuring the state of stress is objectively achieved by the objective symptom correlated with the stress.

  However, in the patent literature, it is premised that photographing is performed by wearing a three-dimensional mask that covers the eyes and is worn on the face. Therefore, it is not easy for the user to use because it requires a dedicated three-dimensional mask.

  In modern society, mobile terminal devices such as smartphones have a high prevalence rate. Therefore, if an objective stress measurement can be performed using a mobile terminal device, it will be easy for users to use. In such a mobile terminal device, for example, in the case of shooting with a smartphone or the like, there is no device for recognizing the pupil, and it is a problem that the change in the size of the pupil cannot be grasped.

  Therefore, it is an object of the present invention to provide a technique for measuring the state of stress in a mobile terminal device according to an objective symptom having a correlation with stress.

  In order to solve the above problems, the present invention provides a moving image ON step of turning on a moving image shooting function of a mobile terminal, a light ON step of turning on a light provided on a shooting side of the portable terminal, and an image being shot. A pupil recognition step for recognizing the pupil of the eyes of animals including humans, and a pupil change calculation step for calculating a pupil change which is a time change of the degree of opening of the recognized pupil can be read and executed by the mobile terminal device. The recorded program was developed.

  Further, in addition to the above characteristics, the present invention has developed a program that further includes an illuminance changing step of changing the illuminance of the light turned on in the light ON step with the passage of time.

  In addition to the above characteristics, the present invention has developed a program that further includes a body composition data acquisition step of acquiring body composition data received from a body composition monitor.

  In addition to the above characteristics, the present invention further develops a program having a stress evaluation step for evaluating a stress level and obtaining a stress evaluation result based on the pupil change calculated by the pupil change calculation step. did.

  Further, in addition to the above characteristics, the present invention has developed a program that further includes a pupil change transmitting step of transmitting the pupil change calculated by the pupil change calculating step to a predetermined address. Then, a pupil change receiving step of receiving a pupil change transmitted as a result of executing the program, a pupil change accumulating step of accumulating the received pupil change, and a pupil change statistical processing step of statistically processing the accumulated pupil change. We have developed a program that can be read and executed on a computer.

  Further, in addition to the above characteristics, the present invention has developed a program further including a stress evaluation result transmission step of transmitting the stress evaluation result obtained in the stress evaluation step to a predetermined address. A program having a pupil change receiving step of receiving a pupil change transmitted as a result of executing the program having the pupil change transmitting step, a pupil change accumulating step of accumulating the received pupil change, and a stress evaluation result transmitting step. The stress evaluation result receiving step of receiving the stress evaluation result transmitted as a result of executing, the stress evaluation result accumulating step of accumulating the received stress evaluation result, and the stress of statistically processing the accumulated stress evaluation result and pupil change We have developed a program in which the statistical processing steps and the computer can be read and executed.

  Further, in addition to the above-mentioned features, the present invention provides a program which is a continuous ON step for turning on the continuous still image shooting function of the mobile terminal, instead of the moving image ON step for turning on the moving picture shooting function of the mobile terminal. developed.

  The present invention also includes a moving image ON step for turning on the moving image shooting function of the mobile terminal, a light ON step for turning on the light provided on the shooting side of the portable terminal, and a person from the image being taken. A portable terminal including a pupil recognition step of recognizing a pupil of an eye of an animal including a moving picture, a moving picture holding step of holding a moving picture taken in the moving picture ON step, and a moving picture sending step of sending the held moving picture to a predetermined address. We have developed a program that can be read and executed on the device. In addition to that feature, instead of the moving picture ON step for turning on the moving picture shooting function of the mobile terminal, a continuous ON step for turning on the continuous still picture shooting function of the mobile terminal is set as the moving picture ON step. Instead of the moving picture holding step of holding the moving picture, a continuous still picture holding step of holding the continuous still picture taken by the mobile terminal, and instead of the moving picture sending step of sending the held moving picture to a predetermined address, continuous A program has been developed in which the continuous still image transmission step of transmitting the continuous still images held in the still image holding step is recorded in a portable terminal device so that it can be read and executed.

  Further, the present invention has recognized a moving image receiving step of receiving a moving image including eyes of an animal including a person from a mobile terminal, and a pupil recognition step of recognizing a pupil of an eye of an animal including a person from the received moving image. We have developed a program in which the pupil change calculation step for calculating the pupil change, which is the change in the degree of pupil opening over time, is recorded in a computer so that it can be read and executed. Then, a continuous still image receiving step of receiving a continuous still image including the eyes of an animal including a person from the mobile terminal, a pupil recognition step of recognizing a pupil of an eye of the animal including a person from the received continuous still images, We have developed a program that records the pupil change calculation step that calculates the pupil change, which is the time change of the recognized pupil opening, in a computer so that it can be read and executed.

  Further, the present invention has developed a portable terminal device or a statistical processing device in which the above program is recorded so that it can be read and executed.

  According to the above invention, it is possible to provide a technique for measuring the state of stress in a mobile terminal device according to an objective symptom having a correlation with stress.

FIG. 3 is a diagram illustrating an example of functional blocks of the mobile terminal device according to the first embodiment. FIG. 3 is a diagram showing an example of a hardware configuration of the mobile terminal device according to the first embodiment. The flowchart which shows an example of the flow of the process of Example 1. FIG. 6 is a diagram showing an example of functional blocks of a mobile terminal device according to a second embodiment. FIG. 3 is a diagram showing an example of a hardware configuration of a mobile terminal device of a second embodiment. The flowchart which shows an example of the flow of the process of Example 2. FIG. 6 is a diagram showing an example of functional blocks of a mobile terminal device of a third embodiment. FIG. 6 is a diagram showing an example of a hardware configuration of a mobile terminal device of a third embodiment. The flowchart which shows an example of the flow of the process of Example 3. The figure which shows an example of the functional block of the portable terminal device of Example 4. FIG. 10 is a diagram showing an example of a hardware configuration of a mobile terminal device of a fourth embodiment. The flowchart which shows an example of the flow of the process of Example 4. The figure which shows an example of the functional block of the portable terminal device of Example 5. FIG. 13 is a diagram showing an example of a hardware configuration of a mobile terminal device according to a fifth embodiment. The flowchart which shows an example of the flow of the process of Example 5. The figure which shows an example of the functional block of the portable terminal device of Example 6. FIG. 10 is a diagram showing an example of a hardware configuration of a mobile terminal device according to a sixth embodiment. The flowchart which shows an example of the flow of the process of Example 6. The figure which shows an example of the functional block of the portable terminal device of Example 7. The figure which shows an example of the hardware constitutions of the portable terminal device of Example 7. The flowchart which shows an example of the flow of the process of Example 7. The figure which shows an example of the functional block of the portable terminal device of Example 8. FIG. 16 is a diagram showing an example of a hardware configuration of a mobile terminal device of an eighth embodiment. The flowchart which shows an example of the flow of the process of Example 8. The figure which shows an example of the functional block of the portable terminal device of Example 9. FIG. 13 is a diagram showing an example of a hardware configuration of a mobile terminal device of a ninth embodiment. The flowchart which shows an example of the flow of the process of Example 9. The figure which shows an example of the functional block of the portable terminal device of Example 10. The figure which shows an example of the hardware constitutions of the portable terminal device of Example 10. The flowchart which shows an example of the flow of the process of Example 10. FIG. 1 showing an example of a mobile terminal device. FIG. 2 showing an example of a mobile terminal device FIG. 3 showing an example of a mobile terminal device The figure which shows an example of the screen of a portable terminal device when implementing this invention. Figure 1 showing the contents of statistical processing Figure 2 showing the contents of statistical processing Figure 3 showing the contents of statistical processing Diagram for explaining the contents of pupil recognition Diagram for explaining the contents of the pupil change calculation The figure which shows an example at the time of making the processing flow of Example 1 into a time chart. Diagram with miosis speed on the horizontal axis and stress level on the vertical axis Diagram showing stress change over time during specific work when the miosis speed is plotted on the horizontal axis and the stress level is plotted on the vertical axis The figure which shows an example at the time of making the processing flow of Example 2 into a time chart. FIG. 3 is a diagram for explaining a mode of pupil calculation. Diagram showing human eyes only with contour lines Figure of mobile terminal device when a part of the screen is a white light image with strong illumination Graph showing stress evaluation value on the vertical axis and pupil change rate on the horizontal axis Graph with stress evaluation value on the vertical axis and body composition on the horizontal axis Diagram showing the method of stress evaluation Diagram for explaining the mechanism for tracing and continuing identification even if slight movements occur in the pupil part Diagram showing replay image after calculation The figure which shows an example which displayed the data regarding the pupil contraction on the screen The figure which shows the flow of a process when it has a light OFF step. The figure which shows the time chart for enabling the structure which complete | finishes a process when it returns to the size of the pupil at the time of recognition. The figure which shows an example of the functional block of the portable terminal device of Example 11. The figure which shows an example of the hardware constitutions of the portable terminal device of Example 11. The flowchart which shows an example of the flow of the process of Example 11. The figure which shows an example of the functional block of the portable terminal device of Example 12. FIG. 16 is a diagram showing an example of the hardware configuration of a mobile terminal device of Example 12; The flowchart which shows an example of the flow of the process of Example 12.

  Embodiments of the present invention will be described below with reference to the drawings. In the following description, the first embodiment is described in claims 1, 13 and 25, the second embodiment is described in claims 2 and 14, the third embodiment is described in claims 3 and 15, and the fourth embodiment is described in claims 4 and 16. Example 5 is claimed in claims 5 and 17, Example 6 is claimed in claims 8 and 18, Example 7 is claimed in claims 6 and 19, Example 8 is claimed in claims 7 and 20, and Example 8 is claimed. Items 9 and 21 correspond to claims 9 and 21, example 9 corresponds to claims 10 and 22, example 11 corresponds to claims 11 and 23, and example 12 corresponds to claims 12 and 24. The contents of the present invention are not limited to the following examples, and various modifications can be made without departing from the gist of the present invention.

<Outline>
The invention of this embodiment is a program in which a moving image of a change in the pupil when light is applied and a function for calculating the change in the pupil of a captured image in a time unit are readably and executably recorded in a portable terminal device. .. In addition, the program is a portable terminal device in which the program is readable and executable. These can be realized not only by the mobile terminal but also by a terminal such as a desktop personal computer, but the case where the mobile terminal is used will be described below as an example.

  The functions and hardware contents of the mobile terminal device of this embodiment and the flow of processing will be described in detail below.

<Functional configuration>
FIG. 1 is an example of a functional block of the mobile terminal device of the present embodiment. As shown in this figure, the mobile terminal device (0101) of this embodiment has a camera unit (0102), a moving image ON unit (0103), a light unit (0104), a light ON unit (0105), and a pupil recognition unit (0106). ) And a pupil change calculation unit (0107).

  It should be noted that the functional blocks configuring the present mobile terminal device described below can be realized as hardware, software, or both hardware and software. Specifically, if a computer is used, a CPU, a main memory, a bus, or a secondary storage device (a hard disk, a non-volatile memory, a storage medium such as a CD-ROM or a DVD-ROM, and a read drive for those media). Etc.), hardware components such as printers, display devices, and other external peripheral devices, I / O ports for those external peripheral devices, driver programs for controlling those hardware, other application programs, and information input. Examples include the user interface used.

  In addition, these hardware and software perform arithmetic processing on the program expanded on the main memory by the CPU, process and accumulate data stored in the memory and hard disk, data input through the interface, etc. It is used for output processing or controlling each hardware component. The present invention can be realized not only as an apparatus but also as a method. Further, a part of such an invention can be configured as software. Further, a software product used for causing a computer to execute such software and a recording medium having the product fixed to a recording medium are naturally included in the technical scope of the present invention (the same applies throughout the present specification). Is).

<< Functions of camera section and light section >>
The “camera unit” has a function for shooting a moving image and / or a still image. The “light unit” has a function of a light provided on the photographing side surface of the mobile terminal. The "camera section" and the "light" section may be present on the front surface and the back surface, respectively, but the object of the present invention can be achieved in any case.

  31 to 33 are schematic diagrams of so-called smartphones among the mobile terminal devices. (A) is the front surface and (b) is the back surface, and there is actually a microphone (not shown).

  On the surface of the mobile terminal device (4601) of FIG. 31, there are a camera (4604), a light (4605), and a screen (4606) in addition to the operation button (4602) and the earpiece (4603), but on the back surface. There are no cameras or lights.

  Although the operation button (4702), the earpiece (4703) and the screen (4706) are present on the front surface of the portable terminal device (4701) in FIG. 32, neither the camera nor the light is present, but on the back surface the camera (4704) and There is a light (4705).

  On the surface of the mobile terminal device (4801) of FIG. 33, in addition to the operation buttons (4802) and the earpiece (4803), there are a camera (4804), a light (4805), and a screen (4806), and also on the back surface. The camera (4804) also has a light (4805).

  For example, when the mobile terminal as shown in FIG. 31 or FIG. 33 is used, the user can shoot a moving image while looking at the screen. Therefore, as shown in FIG. 31, it is preferable to use a mobile terminal having a camera and a light on the front side with a screen.

  On the other hand, in the case of using the mobile terminal as shown in FIG. 32, in order to apply the light to the pupil, the back side of the screen comes to the user's side. Then, the user cannot grasp which part of the screen his / her eyes or pupil is present, and thus has a problem in ease of operation. Therefore, for example, when the pupil is not near the center of the screen, an instruction voice such as "right" or "left" is notified, while when the pupil is near the center of the screen, a user is notified by a beep sound. It may be provided with a pupil position grasping means such as. Besides the sound, the pupil position grasping means may notify the user that the pupil is near the center of the screen by changing the illuminance of the light.

  The size of the pupil can be easily recognized by using infrared rays. Therefore, it is desirable that the camera has an infrared camera function.

  Further, in the case where the screen is facing the user side, in the case of a mobile terminal device in which only the camera is present on the surface of the screen, a part of the screen may be replaced with a light by making a white light image with high illuminance. .. For a white light image with high illuminance, it may be in the upper half of the screen, in the lower half, or in the upper and lower quarters. Further, the ratio of the portion where the captured image is displayed and the portion of the white light image where the illuminance is high is not limited to 1: 1 and may be any ratio sufficient for measuring the pupillary light reaction.

  FIG. 46 is a diagram of a mobile terminal device when a part of the screen is a white light image with high illuminance. Since the mobile terminal device (6101) has a camera (6102) but no light, the lower half of the screen (6103) is a white light image (6104) with high illuminance. As a result, it can be replaced with a light.

<< Functions of video ON section and light ON section >>
The "video ON section" functions to turn ON the video shooting function of the mobile terminal. Further, the “light ON section” functions to turn on the light section including the light provided on the photographing side surface of the mobile terminal. As the most basic configuration of the present invention, it is necessary to take a moving image of the pupil when light is applied. The user can shoot a moving image of the pupil portion by the function of the moving image ON unit. Further, the user can apply light to the pupil by the function of the light ON section.

  The "shooting" in the moving image ON section includes the case of simultaneously recording, but the recording is not essential. For example, this embodiment includes a mode in which the moving image ON unit only shoots a moving image and the moving image being shot is calculated in real time by the “pupil recognition unit” and the “pupil change calculation unit” described later. Is.

  In addition, it is desirable that the moving images that are shot include not only moving images that are being illuminated, but also moving images before and after being illuminated. By including the moving images before and after the light irradiation, the pupillary light reaction can be accurately measured.

  It is premised that the moving image being photographed here includes the pupil, but it is preferable that the moving image during photography includes not only the pupil but also a part of the body around the pupil other than the pupil. The health condition of a person's eyes appears in various forms on the human eyeball. For example, in allergic conjunctivitis, infectious conjunctivitis, uveitis, dry eye, and the like, hyperemia of the eye may be caused. Therefore, there is an effect that the above-mentioned symptom can be known by the fact that, for example, red blood in the conjunctiva occurs.

  Next, "mobile terminals" include, for example, smartphones, mobile phones, laptop computers, tablet PCs, clock computers, wearable computers, digital cameras, PDAs, PocketPCs, Palms, dedicated terminals with lights and cameras, etc. It includes. Normally, it is assumed that the device has a camera function and a light function, but in the present invention, even if the unit sold does not have a camera function or a light function, an external camera is mounted. In this case, or when a light is mounted externally, the device and the external camera or the external light are regarded as a “portable terminal” as a unit.

  The intensity of the light emitted by the light-ON unit may be sufficient to cause the pupil to undergo a light reaction. In addition to this, in view of the fact that the fundus examination is performed by illuminating the pupil with light, for example, it is conceivable to set the light intensity suitable for performing the fundus examination. By doing so, the fundus examination can be performed simultaneously with the measurement of the stress intensity.

  Furthermore, the mobile terminal device may include a moving image shooting function and a control unit for controlling ON / OFF of the light. With this control unit, only by the user turning the switch ON, the moving image shooting function and the ON / OFF control of the light are automatically controlled at appropriate timing.

  Further, in the control unit, the moving image shooting function and the light may be automatically turned on at a preset predetermined time. By doing so, the pupillary light reaction at a specific time can be measured, and for example, it can be automatically measured during work.

  With these, it is possible to take a moving image of the pupil when the light is applied.

≪Pupil recognition section≫
The "pupil recognition unit" functions to recognize the pupils of the eyes of animals including humans from the image being photographed. The point of the problem of the present invention is that it is not possible to grasp the change in the size of the pupil in the case of photographing with a mobile terminal device such as a smartphone. Specifically, in the moving image capturing function of the mobile terminal, for example, although the face recognition function has permeated, the function for recognizing the pupil part of the eyeball is not general. In order to solve the problems of the present invention, it is necessary to calculate the size of the pupil based on the moving image information, but if the size of the pupil cannot be recognized, such calculation is also impossible. The pupil recognition unit enables recognition of the size of the pupil and is an important function in solving the problems of the present invention.

  As a function for recognizing the pupil, after the pupil is photographed, the color or lightness of the pupil is registered based on the moving image or still image information of the pupil, and the same color or lightness as the registered color or lightness is registered. It is conceivable to recognize the pupil by identifying the part with.

  Further, the mobile terminal device may include means for automatically recognizing the pupil portion even if the color or brightness of the pupil is not registered. Specifically, referring to FIG. 38, first, there is a color lightness recognition means for recognizing the dark color part (A), the light color part (B), the white part (C) and the skin part (D) of the eyeball by color or brightness. Then, the iris recognition means for finding the boundary E between the light-colored part (B) and the white part (C) of the eyeball, and finally the boundary which is the boundary between the dark-colored part (A) and the light-colored part (B) of the eyeball. A method of having a pupil finding means for finding F may be used.

  Even if slight movements occur in the pupil part, it is desirable to trace and continue the identification. Further, in order to prevent the slight movement of the pupil part, for example, as shown in FIG. 50, the portable terminal device (6801) is turned sideways and on the screen (6804) near the camera (6802) or the light (6803). It is conceivable that the pupil part is fixed by displaying an image (6805) such as a moving image on the screen so that one point is seen. In this case, if white light is displayed on the screen (6804), the light can be replaced with the light. It is desirable that the place for displaying an image such as a moving image be positioned so that the center line of the moving image is on the center line of the camera (6802). In addition, an attention point (6806) to be noticed in the image may be clearly indicated.

  The circumference of the pupil part may be darker in color. In such a case, the inside of the darker part may be the boundary F, the outside may be the boundary F, or the intermediate point may be the boundary F. However, in order to improve the accuracy of information, it is not desirable that the measurement be different for each measurement, but that the settings be unified.

  This point will be described with reference to FIG. FIG. 45 is a diagram showing only the outline of the human eyes. The human eye (6001) has an iris boundary portion (6002) and a pupil boundary portion (6003), and there is a portion (6004) inside the pupil that has a darker color up to the portion indicated by the dotted line. There are cases. In such a case, the darker color portion (6004) may be the boundary F, the pupil boundary portion (6003) may be the boundary F, or the intermediate point may be the boundary F.

  FIG. 34 is a diagram showing a screen when a pupil is recognized. When the eye portion including the pupil is photographed using the mobile terminal device (4901) and the pupil portion is successfully recognized, it indicates that the pupil has been recognized as shown in the right figure. As described above, the pupil portion (4902) may be raised. Thereby, it is possible to inform the user that the pupil recognition has been successful. In addition, as a means for notifying that the pupil recognition has been successful, for example, a method of notifying through a notification sound of “pee” or a change in illuminance may be used, but is not limited thereto.

  In order to automatically recognize the pupil, it is desirable that the camera has an autofocus function for automatically focusing. Further, since there is a possibility that camera shake will occur when the user operates it, it is desirable to have a camera shake prevention function. This allows accurate pupil recognition.

  Further, since the user does not blink basically during moving image shooting, the user suffers from symptoms such as dry eyes. Therefore, it is convenient for the user to understand the elapsed seconds or the remaining seconds. Therefore, it is desirable to have a device to inform the remaining time of shooting by sound even during shooting of a moving image.

  In addition, when the user blinks, the accuracy of the calculation decreases, but the user may not notice that the user has blinked. Therefore, when a blink is recognized, a function may be provided to notify the user by a notification sound, ON / OFF of a light, color, illuminance, or the like, in order to reshoot a moving image. Further, when the blink is recognized, the moving image shooting may not be restarted from the beginning, and the moving image shooting may have an extension function for extending the shooting time (measurement time).

≪Pupil change calculation unit≫
The “pupil change calculation unit” has a function of calculating a pupil change, which is a change with time of the recognized opening degree of the pupil. As described above, the degree of stress or relaxation has a correlation with the degree of pupillary light reaction. Therefore, the pupil change calculation unit calculates the pupil light reaction. Then, the pupillary light reaction is, specifically, a temporal change in the size of the pupil caused by shining light on the pupil.

  The data of the pupil to be calculated may be the diameter of the pupil or the area of the pupil. For example, if it is the diameter of the pupil, it is possible to measure more accurately even when the entire image of the pupil is blocked by eyelashes or the eyelids are not completely opened. desirable. On the other hand, if it is the area of the pupil, it is desirable in that the size of the pupil can be accurately grasped under the above circumstances.

  Further, as the calculation content, in addition to the diameter value and the area value, it is conceivable to obtain the time required until the pupil is minimized, the miosis ratio, the miosis speed, the miosis acceleration, and the like. The user can obtain information on his / her pupillary light reaction based on the value obtained by this calculation. By doing so, the user can objectively know the degree of stress and relaxation of the user by performing analysis based on the correlation between the stress and the degree of relaxation.

  Further, as the content of the calculation, it is conceivable to obtain the time required for the pupil with reduced pupil to become normal (original state), the mydriatic ratio, the mydriatic velocity, and the mydriatic acceleration. For that purpose, a “light OFF section” for turning off the light provided on the photographing side of the mobile terminal may be further provided. According to this, a more accurate stress level can be known.

  Regarding the specific content of the pupil calculation, considering that the distance to the imaged portion is not constant, it is impossible to accurately measure the pupil change even if the size of the pupil portion in the moving image is measured aimlessly. In order to realize accurate measurement, it is desirable to have a configuration in which the influence of the distance and angle with the imaged portion is small. Specifically, for example, with reference to FIG. 39, if the value is obtained by comparing the diameter α of the iris portion (5401) and the diameter β of the pupil portion (5402) by division, the distance to the imaging portion or It is not affected by the angle. For example, in the example of FIG. 39, the calculation result is “α / β” or “β / α”. Therefore, it is desirable that the diameter α of the iris portion be compared with the diameter β of the pupil portion.

  In order to realize such a comparison, based on the iris finding means and the pupil finding means described in FIG. 38, the diameter measuring means for measuring the diameters of the iris portion and the pupil portion on the moving image, and the measurement. It can be realized by having a dividing unit that performs the division on the diameter.

≪Other≫
In addition, a registration information accepting unit that accepts registration information such as name, age and date of birth, and eating habits may be provided as identification information for identifying the user. Then, the iris information of the user may be accepted in association with this registration information. Thereby, by performing iris recognition by the camera of the mobile terminal device, reliable user management can be performed regardless of the user ID and the like.

  Further, by attaching an external fan to the mobile terminal device, the fan can be activated to simultaneously perform the intraocular pressure test. In the intraocular pressure test, since the intraocular pressure tends to be measured high when blinking, the notification function when blinking is useful for the fundus examination. Fundus examination reveals diseases such as glaucoma, retinal detachment, choroidal detachment, and iridocyclitis.

  In addition, in order to increase the value of information on pupil changes, name, sex, date of birth, place of origin, measurement time, measurement location (for example, residence, university, workplace, izakaya, train, park, etc.), before meal, It is desirable to store additional information such as food composition, occupation, presence / absence of work on measurement day, and strength, and body composition data and treatment history in addition to after meal, in the case of after meal. In particular, the date of birth is necessary for realizing the stress evaluation that takes into account biorhythms that may become noise in stress evaluation. When the mobile terminal device is equipped with GPS, the GPS function may be used to recognize the longitude / latitude of the measurement location and to calculate the altitude from the information. Since stress can be influenced by oxygen concentration, it has a meaning as additional information.

  Furthermore, it is desirable to have a pupil change list output unit that outputs a list of pupil changes. As a method of displaying the list, a graph, a table, a calendar, or the like can be considered.

  FIG. 35 is a diagram showing the contents of statistical information processed by the pupil change list output unit. For the user A, the pupil change for each date is shown in a list. In addition to this, for example, information such as measurement conditions may be written in the date column portion. For example, if it is displayed together with information such as the brightness of the measured place, the measurement time, whether before or after meal, the sleeping time on the previous day, the presence or absence of work on the measurement day, and the intensity, the user can You can know the cause and help you think about life to reduce stress.

  Further, when the boundary of the pupil is difficult to understand, a device may be devised to make the pupil portion recognized by the pupil recognition unit stand out as shown in FIG. In FIG. 36, a device for making the outline of the pupil clear is devised.

  In addition, it is preferable that the mobile terminal device includes a replay unit for replaying the captured moving image after the calculation is completed. In this replay, in order to confirm whether or not the pupil recognition by the pupil recognition unit is accurate, the part recognized as the pupil is outlined or the part used for the measurement as the diameter is clearly shown in FIG. 51. It is desirable to adopt a configuration that does.

  Further, data regarding pupil contraction may be displayed on the screen so that the pupil change can be confirmed immediately after the photographing. FIG. 52 is an example in which data relating to pupil contraction is displayed on the screen. The timing of when the light becomes bright, the illuminance of the light, the transition of the change of the pupil, and the like are listed so that the user can see the calculation result at a glance.

≪Summary≫
With the above-described functions, it is possible to provide a technique for measuring the state of stress in a mobile terminal device according to an objective symptom having a correlation with stress.

<Hardware configuration>
FIG. 2 is a diagram showing an example of the configuration of the mobile terminal device when the functional components are implemented as hardware. The operation of each hardware configuration unit in each processing in this device will be described with reference to this drawing.

  As shown in this figure, the mobile terminal device of this embodiment includes a "CPU (central processing unit)" (0201) for performing various kinds of arithmetic processing and a "main memory" (0202). Further, a “HDD” (0203) and a “network I / F (network interface)” (0204) for transmitting / receiving information to / from an “I / F (interface)” (0207) of an external device (0206), It also has an “I / O (input / output)” (0208) for transmitting and receiving information to and from a camera, a light, a display and the like. Then, they are mutually connected by a data communication path such as a “system bus” (0205) and perform transmission / reception and processing of information. The “external device” here usually refers to an administrator terminal or a statistical processing device, but includes, for example, a user terminal or a terminal of a medical institution.

  Further, the "main memory" provides a work area which is a work area of the program at the same time as the program for performing various processes is read for execution by the "CPU". Further, a plurality of addresses are assigned to each of the "main memory", "HDD", and "flash memory" (not shown), and the program executed by the "CPU" identifies each address and accesses each other. It is possible to exchange data with and process. The programs stored in the present embodiment are a moving picture ON program for turning on the moving picture shooting function of the mobile terminal, a light ON program for turning on the light provided on the shooting side of the mobile terminal, and an image being shot. A pupil recognition program for recognizing the pupils of eyes of animals including humans, and a pupil change calculation program for calculating a pupil change which is a temporal change in the degree of opening of the recognized pupils.

  The pupil image of the user acquired by the moving image ON program of the mobile terminal device is stored in the addresses of the “main memory” and the “HDD”. Then, in the "CPU" of the mobile terminal device, the temporal change of the pupil recognized by the logical operation processing is calculated using the moving image information stored in the "main memory". It should be noted that when recording a captured moving image, the moving image information is stored in the “HDD” in this way, while on the other hand, when recognizing and calculating the pupil of the recorded recording in real time, the moving image information is stored in the “HDD”. It will not be stored.

<Process flow>
FIG. 3 is a flowchart showing an example of the flow of processing in the mobile terminal device of this embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded in a medium.

  As shown in (a) of this figure, first, the moving image capturing function of the mobile terminal is turned on (S0301). Next, the light provided on the photographing side of the mobile terminal is turned on (S0302). Then, the pupils of the eyes of animals including humans are recognized from the image being photographed (S0303). Finally, a pupil change, which is a change with time of the recognized pupil opening degree, is calculated (S0304).

  As shown in (b) of the figure, the moving image ON step (S0301) for turning on the moving image shooting function of the mobile terminal and the light ON step (S0302) for turning on the light provided on the shooting side of the portable terminal are: The order may be reversed. That is, first, the light provided on the photographing side of the mobile terminal is turned on. Next, the video recording function of the mobile terminal is turned on. Then, the pupils of the eyes of animals including humans are recognized from the image being photographed. Finally, the flow of processing of calculating a change in the pupil, which is a change with time of the recognized degree of opening of the pupil, may be used.

  Further, as shown in FIG. 7C, a light ON step (S0302) of turning on a light provided on the photographing side of the mobile terminal and recognizing the pupils of eyes of animals including a person from the image being photographed. In the pupil recognition step (S0303), the order may be reversed. That is, first, the moving image capturing function of the mobile terminal is turned on. Next, the pupils of the eyes of animals including humans are recognized from the image being photographed. Then, the light provided on the photographing side of the mobile terminal is turned on. Finally, the flow of processing of calculating a change in the pupil, which is a change with time of the recognized degree of opening of the pupil, may be used.

  The order of the processing flow as described above is the same for the embodiments other than this embodiment.

  In addition, the light provided on the photographing side of the mobile terminal is turned off in order to obtain the time required for the pupil with reduced pupil to become normal (original state), the mydriasis rate, the mydriasis speed, and the mydriasis acceleration. It may have a “light OFF step”. According to this, a more accurate stress level can be known.

  FIG. 53 is a diagram showing the flow of processing in the case of having a light OFF step. First, there is a moving image ON step (S7101) and a light ON step (S7102). Next, recognition is started as a pupil recognition step (S7103), it is confirmed whether or not the pupil recognition is successful (S7104), and the recognition step is repeated until the pupil recognition is successful. This pupil recognition continues until the whole processing is completed. Next, there is a pupil change calculation step (S7105). This pupil change calculation continues until the entire processing is completed. Next, as the content of the light OFF step, it is confirmed whether the light should be turned off (S7106). If YES, the light is turned off (S7108), and if NO, the process is continued until YES. The confirmation of whether to turn off the light here can be considered to turn off the light when a predetermined time elapses, when the contraction stops, or when the speed reaches a certain speed or less. Considering that there are individual differences in biological reactions such as the miosis speed, it is desirable that the timing of turning off the light is not uniform, but when the contraction is stopped or when the speed becomes a certain speed or less. Finally, there is a confirmation as to whether or not the whole process is completed. If NO, the process is continued, and if YES, the whole process is ended. Whether or not the whole process is completed is set to YES when a predetermined time has elapsed, when the motion is stopped or when the velocity is below a certain level, or when the size of the pupil at the time of recognition is returned. It is possible that

  FIG. 40 is a diagram showing an example in which the above steps are made into a time chart. When the user turns on the switch (SW), moving image shooting and pupil recognition start. Then, the pupil calculation is started at the same time as the recognition is completed and the pupil recognition function is turned ON (high) or after a slight delay. After that, the light is turned off when a predetermined time elapses, when the contraction is stopped, or when the speed becomes a certain speed or less (the range indicated by α in the drawing). Then, when a predetermined time elapses, when the scatter stops, or when the velocity becomes lower than a certain level (range indicated by β in the drawing), or when the size of the pupil at the time of recognition returns. The whole process ends.

  When the transition of the miosis is examined using FIG. 40, it is found that the miosis occurs while the light is on, the mydriasis starts after the light is turned off, and the mydriasis stops after a certain degree of mydriasis. I understand.

  In this time chart, for example, in moving image shooting, there may be a stage (High) of automatically tracking only the iris portion and the pupil portion, which are important portions in the pupil calculation. As a result, the pupil calculation can be performed accurately.

  FIG. 44 is a diagram for explaining the aspect of pupil calculation. Assuming that there are A and B data as model data, the miosis speed (a1) when the light of the line A is applied, the miosis ratio (ha) at the time of the most miosis, and the light after turning off the light The mydriatic speed (a2), the time from turning off the light until the mydriasis stops (ta), etc. are measured. Also for the line B, the miosis speed (b1) when the light is applied, the miosis ratio (hb) when the light is most shrunk, the mydriatic speed after the light is turned off (b2), and the light is turned off. After that, the time (tb) until the mydriasis stops is measured. In this way, the miosis starts after turning on the light, and the mydriasis starts after turning off the light. If the two model data are analyzed on the assumption that the stress is weaker for both the miosis and the mydriasis, the model data B is less stressed than the model data A. I understand.

  Through the above steps, the state of stress can be measured by using the mobile terminal device according to the objective symptom correlated with the stress.

<Outline>
As a more desirable embodiment, the present embodiment provides a mobile terminal device having a function of changing the illuminance of a light with time in addition to the configuration of the first embodiment.

  Hereinafter, the functions and hardware contents of the apparatus and the flow of processing in this embodiment will be described in detail.

<Functional configuration>
FIG. 4 is a diagram showing an example of functional blocks of the mobile terminal device of the present embodiment. As shown in this figure, the portable terminal device (0401) of this embodiment has a camera unit (0402), a moving image ON unit (0403), a light unit (0404), a light ON unit (0405), and a pupil recognition unit (0406). ), A pupil change calculation unit (0407), and an illuminance change unit (0408). The feature of this embodiment lies in the contents of the illuminance changing unit. Therefore, the functional configuration of the illuminance changing unit will be mainly described. Since the other functions are the same as those described above, the description thereof will be omitted.

  The "illuminance changing unit" (0406) functions to change the illuminance of the light turned on by the light ON unit with the passage of time. For example, although it is necessary to apply light to the pupil in order to recognize the pupil, on the other hand, if the illuminance of the light is too strong, the pupil-light reaction occurs. Then, the change starts before the pupil is recognized by the pupil recognition unit, and the accuracy of the calculation is impaired. Therefore, by the function of the illuminance changing unit, for example, light with weak illuminance is applied until pupil recognition, and from the time when the pupil is recognized thereby, light with illuminance sufficient to cause a pupillary light reaction is applied. It is possible to obtain a high calculation result.

  Further, where it is desirable to change the pupil in a dark room, the operation of the illuminance changing unit enables operation using a minimum amount of light, which enables use in a dark room using a mobile terminal.

<Hardware configuration>
FIG. 5 is a diagram showing an example of the configuration of the mobile terminal device when the functional components are implemented as hardware. The mobile terminal device includes a "CPU (central processing unit)" (0501), "main memory" (0502), "HDD" (0503), "network I / F" (0504), and "I / O" (0508). , And "system bus" (0505), and the external device (0506) has "I / F" (0507).

  With reference to these figures, the characteristic parts of the present embodiment out of the respective hardware components in each processing in this device will be described, and other parts will be the same as the description of the above-mentioned embodiments. is there.

  In the mobile terminal device of the present embodiment, an illuminance change program is added as the content of the program stored in the "main memory".

  Therefore, in the operation of the “CPU”, the illuminance of the light turned ON by using the light ON program changes with the passage of time.

<Process flow>
FIG. 6 is a flowchart showing an example of the flow of processing in the mobile terminal device of this embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded in a medium.

  First, there is a moving image ON step (S0601) and a light low ON step (S0602). Next, recognition is started as a pupil recognition step (S0603), it is confirmed whether or not the pupil recognition is successful (S0604), and the recognition step is repeated until the pupil recognition is successful. This pupil recognition continues until the whole processing is completed.

  Next, there is a pupil change calculation step (S0605). This pupil change calculation continues until the entire processing is completed. Next, as the content of the illuminance change step, it is confirmed whether the light is set to “high” or “low” (S0606). If it is “high”, the light is set to high (S0607), If it is "low", a process for setting the light to low is performed (S0608). To confirm whether to set the light to "high" or "low", for example, first set the light to high and then, after a predetermined time has elapsed or when the contraction stops or at a certain level or less. It is conceivable to set the light to low when the speed reaches. Considering that there are individual differences in biological reactions such as the miosis speed, it is desirable that the timing of turning the light low is not uniform, but at the time when the contraction is stopped or when the speed becomes a certain speed or less.

  Finally, there is a confirmation as to whether or not the entire processing is completed. If the entire processing is not completed, the process returns to the confirmation whether the light is set to “high” or “low” (S0606), and the entire processing ends. If so, the whole process is completed. Whether or not the whole process is completed is set to YES when a predetermined time has elapsed, when the motion is stopped or when the velocity is below a certain level, or when the size of the pupil at the time of recognition is returned. It is possible that

  The processing in this embodiment has an illuminance change step. The “illuminance change step” is a step in which the illuminance of the light turned on by the light ON unit is changed over time.

  FIG. 43 is a diagram showing an example in which the above steps are made into a time chart. When the user turns on the switch (SW), moving image shooting, lighting, and pupil recognition are started. Then, at the same time as the recognition is completed and the pupil recognition state becomes High, or slightly later, the pupil calculation is started and the illuminance of the light increases. After that, the light becomes low when a predetermined time elapses, when the contraction is stopped, or when the speed becomes a certain speed or less (the range indicated by α in the drawing). Then, when the predetermined time elapses, or when the scattering is stopped or when the speed becomes a certain speed or less (the range indicated by β in the drawing), the entire processing ends.

  Next, FIG. 54 is a diagram showing a time chart for enabling a configuration in which the processing is terminated when the size of the pupil at the time of recognition is returned. When the user turns on the switch (SW), moving image shooting, lighting, and pupil recognition are started. Then, at the same time when the recognition is completed and the pupil recognition state becomes High, or slightly later, the pupil calculation is started, and after the pupil in the original state is calculated, the illuminance of the light increases. After that, the light becomes low when a predetermined time elapses, when the contraction is stopped, or when the speed becomes a certain speed or less (the range indicated by α in the drawing). Then, when a predetermined time elapses, when the scatter stops, or when the velocity becomes lower than a certain level (range indicated by β in the drawing), or when the size of the pupil at the time of recognition returns. The whole process ends. As described above, by performing the calculation before the illuminance of the light becomes high, the pupil in the original state is calculated, and the entire process is terminated when the calculation result matches. it can.

  Examining the transition of the miosis using FIG. 43, the miosis gradually starts while the light is low and is ON, and when the light becomes high, the contraction speed is accelerated, and when the light becomes low again. It can be seen that mydriasis starts and stops after a certain amount of mydriasis.

  As described in the functional configuration, with such a processing flow, it is possible to obtain a highly accurate calculation result and to use the mobile terminal in a dark room.

<Outline>
As a more desirable embodiment, this example provides a mobile terminal device having a function of evaluating the degree of stress in addition to the configuration of any one of Examples 1 and 2.

  Hereinafter, the functions and hardware contents of the apparatus and the flow of processing in this embodiment will be described in detail.

<Functional configuration>
FIG. 7 is a diagram showing an example of functional blocks of the mobile terminal device of the present embodiment. As shown in this figure, the mobile terminal device (1001) of this embodiment has a camera unit (1002), a moving image ON unit (1003), a light unit (1004), a light ON unit (1005), and a pupil recognition unit (1006). ), A pupil change calculation unit (1007), and a stress evaluation unit (1008). The feature of this embodiment is the content of the stress evaluation unit. Therefore, the functional configuration of the stress evaluation unit will be mainly described. Since the other functions are the same as those described above, the description thereof will be omitted.

  The “stress evaluation unit” functions to evaluate the degree of stress based on the pupil change calculated by the pupil change calculation unit and obtain a stress evaluation result. For example, based on the data such as the diameter value, the area value, the time required for the pupil to become the minimum, the miosis rate, the miosis speed, and the miosis acceleration, which are calculated by the pupil change calculation unit, Calculate the degree of stress.

  The stress level may be a numerical value such as% or may be a large / medium / small level system. Further, it is also possible to use the previous day's ratio or the comparison value with the maximum value / minimum value. Further, when a correlation with sleep time or work intensity can be seen, the cause of stress may be derived based on the correlation.

  FIG. 49 is a diagram showing a stress evaluation method. When the statistical data of 2000 people becomes a curve as shown in the drawing, a method of showing stress evaluation by measuring the distance from the median t for the time to the post-miosis pupil is considered. For example, in FIG. 49, “normal” is in the center side of “+ 1σ”, “weak stress” is in the zone on the opposite side of the median t from “+ 1σ”, and the median value of “+ 2σ”. It is possible to judge that "moderate stress is present" in the case of t or "strong stress" in the zone of "+ 2σ" on the opposite side of the median t. Furthermore, in the case of the “over” part that is farther from the median t than “+ 2σ”, it can be considered that “very strong stress is present”.

  Further, when the degree of stress exceeds a certain level, a function of giving an alarm sound or the like may be provided. In addition, we will share information between parents and children and between employers and workers of work, for example, when children and employees have a certain amount of stress, a notice to that effect is sent to parents and users (company etc.) It is desirable to have a system that can reach

  Furthermore, it is desirable that there is a stress evaluation list output unit that outputs the stress evaluation results in a list. As a method of displaying the list, a graph, a table, a calendar, or the like can be considered.

  FIG. 37 is a diagram showing the contents of the statistical information output by the stress evaluation result output unit. The stress evaluation results for each date are listed in a calendar system. As a result, the stress level of the user can be listed, and the features of the stress level for each day of the week can be known. In addition to this, for example, information such as pupil change information and measurement conditions may be written in the date section. For example, if it is displayed together with information such as the brightness of the measured place, the measurement time, whether before or after meal, the sleeping time on the previous day, the presence or absence of work on the measurement day, and the intensity, the user can You can know the cause and help you think about life to reduce stress.

  With this function, the user can recognize the degree of stress at a glance by numerical values or marks. Also, if the cause can be guided, it can be utilized in everyday life. In addition, the provision of an alarm function and a notification function for others leads to the prevention of traffic accidents or the like, which is one of the causes of stress.

  Furthermore, in the case of the configuration added to the third embodiment, it is desirable that there is a list output unit that outputs the acquired body composition data and stress evaluation results in a list. By displaying the list in this way, it is possible to see at a glance what is the factor influencing the stress in the body composition data, as compared with the case where the pupil change.

<Hardware configuration>
FIG. 8 is a diagram showing an example of the configuration of the mobile terminal device when the functional components are implemented as hardware. The mobile terminal device includes a "CPU (central processing unit)" (1101), a "main memory" (1102), a "HDD" (1103), a "network I / F" (1104), and an "I / O" (1108). , And "system bus" (1105), and the external device (1106) has "I / F" (1107).

  With reference to these figures, the characteristic parts of the present embodiment out of the respective hardware components in each processing in this device will be described, and other parts will be the same as the description of the above-mentioned embodiments. is there.

  In the mobile terminal device according to the present embodiment, a stress evaluation program is added as the content of the program stored in the "main memory".

  Therefore, in the operation of the "CPU", the stress level is evaluated based on the pupil change obtained by the pupil change calculation program.

<Process flow>
FIG. 9 is a flowchart showing an example of the flow of processing in the mobile terminal device of this embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded in a medium.

  It has a moving image ON step (S1201), a light ON step (S1202), a pupil recognition step (S1203), a pupil change calculation step (S1204), and a stress evaluation step (S1205). With reference to these drawings, the characteristic part of the present embodiment in the flow of each processing in the present device will be described, and the other parts are the same as the description of the above-mentioned embodiments.

  The processing in this embodiment has a stress evaluation step. The “stress evaluation step” is a step of evaluating the degree of stress based on the pupil change calculated by the pupil change calculation step to obtain a stress evaluation result.

  As described in the functional configuration, the stress level may be a numerical value such as%, or may be a large / medium / small level system. Further, it is also possible to use the previous day's ratio or the comparison value with the maximum value / minimum value. Further, when a correlation with sleep time or work intensity can be seen, the cause of stress may be derived based on the correlation.

  Further, when the degree of stress exceeds a certain level, an alarm step for giving an alarm sound or the like may be provided. In addition, information is shared between parents and children and between users and workers of work. For example, when children and employees have a certain amount of stress, an alarm to that effect is sent to parents and users (company etc.) It would be even better if there was a system that could reach.

  Furthermore, it is desirable that there is a stress evaluation list output step for outputting the stress evaluation results in a list. As a method of displaying the list, a graph, a table, a calendar, or the like can be considered.

  With such a processing flow, the user can recognize the degree of stress at a glance by numerical values or marks. Also, if the cause can be guided, it can be utilized in everyday life. In addition, the provision of an alarm function and a notification function for others leads to the prevention of traffic accidents or the like, which is one of the causes of stress.

  Further, in the case of the configuration added to the third embodiment, it is desirable that there is a list output step for outputting the acquired body composition data and stress evaluation result in a list. By displaying the list in this way, it is possible to see at a glance what is the factor influencing the stress in the body composition data, as compared with the case where the pupil change.

<Outline>
As a more desirable embodiment, this embodiment provides a mobile terminal device having a function of transmitting a pupil change to a predetermined address in addition to the configuration of any one of the first to third embodiments.

  Hereinafter, the functions and hardware contents of the apparatus and the flow of processing in this embodiment will be described in detail.

<Functional configuration>
FIG. 10 is a diagram showing an example of functional blocks of the mobile terminal device of the present embodiment. As shown in this figure, the mobile terminal device (1301) of this embodiment has a camera unit (1302), a moving image ON unit (1303), a light unit (1304), a light ON unit (1305), and a pupil recognition unit (1306). ), A pupil change calculation unit (1307), and a pupil change transmission unit (1308). The feature of this embodiment resides in the contents of the pupil change transmitter. Therefore, the functional configuration of the pupil change transmitter will be mainly described. Since the other functions are the same as those described above, the description thereof will be omitted.

  The "pupil change transmitter" has a function of transmitting the pupil change calculated by the pupil change calculator to a predetermined address. For example, the pupil change information is transmitted to the HDD or main memory, which is a storage area of a statistical processing device for statistical processing, which is an example of an external device.

  In addition to these, it is desirable that the external device transmits the stress evaluation result when the stress evaluation is performed by the external device. Then, it is desirable that the mobile terminal device includes a stress evaluation result receiving unit.

  In addition, the external device holds the measure introduction information in association with the stress evaluation value, and when the stress evaluation value is high, for example, a proposal to eliminate the factor causing the stress or introduction of a hospital / store, etc. You may receive the referral information together with.

<Hardware configuration>
FIG. 11 is a diagram showing an example of the configuration of the mobile terminal device when the functional components are implemented as hardware. The mobile terminal device includes a "CPU (central processing unit)" (1401), a "main memory" (1402), a "HDD" (1403), a "network I / F" (1404), and an "I / O" (1408). , And "system bus" (1405), and the external device (1406) has "I / F" (1407).

  With reference to these figures, the characteristic parts of the present embodiment out of the respective hardware components in each processing in this device will be described, and other parts will be the same as the description of the above-mentioned embodiments. is there.

  In the mobile terminal device of this embodiment, a pupil change transmission program is added as the content of the program stored in the "main memory".

  Therefore, in the operation of the “CPU”, the pupil change information obtained by the pupil change calculation program is stored in, for example, the “HDD” or “main memory” of the mobile terminal device, or an external device or a statistical processing device (not shown). Will be sent to the main memory or HDD.

<Process flow>
FIG. 12 is a flow chart showing an example of the flow of processing in the mobile terminal device of the present embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded in a medium.

  It has a moving image ON step (S1501), a light ON step (S1502), a pupil recognition step (S1503), a pupil change calculation step (S1504), and a pupil change transmission step (S1505). With reference to these drawings, the characteristic part of the present embodiment in the flow of each processing in the present device will be described, and the other parts are the same as the description of the above-mentioned embodiments.

  The processing in this embodiment includes a pupil change transmission step. The "pupil change transmission step" is a step of transmitting the pupil change calculated by the pupil change calculation step to a predetermined address. As described in the functional configuration, the transmission destination may be the mobile terminal device or the statistical processing device.

  As described above, it is possible to save the storage area of the mobile terminal device necessary for accumulating the pupil change information. Further, by performing the statistical processing on a device other than the mobile terminal, the load on the mobile terminal device can be reduced.

<Outline>
As a more preferable embodiment, the present embodiment provides a mobile terminal device having a function of transmitting a stress evaluation result to a predetermined address in addition to the configuration of the third embodiment or the fourth embodiment added to the third embodiment. ..

  Hereinafter, the functions and hardware contents of the apparatus and the flow of processing in this embodiment will be described in detail.

<Functional configuration>
FIG. 13 is a diagram showing an example of functional blocks of the mobile terminal device of the present embodiment. As shown in this figure, the portable terminal device (1901) of this embodiment has a camera unit (1902), a moving image ON unit (1903), a light unit (1904), a light ON unit (1905), and a pupil recognition unit (1906). ), A pupil change calculation unit (1907), a stress evaluation unit (1908), and a stress evaluation result transmission unit (1909). The feature of this embodiment resides in the content of the stress evaluation result transmission unit. Therefore, the functional configuration of the stress evaluation result transmission unit will be mainly described. Since the other functions are the same as those described above, the description thereof will be omitted.

  The “stress evaluation result transmission unit” has a function of transmitting the stress evaluation result obtained by the stress evaluation unit to a predetermined address. For example, the stress evaluation is transmitted to a predetermined address of the HDD or main memory of the mobile terminal device. Alternatively, the stress evaluation information is transmitted to the HDD or main memory, which is the storage area of the statistical processing device for statistical processing. Thereby, it becomes possible to accumulate the stress evaluation information.

<Hardware configuration>
FIG. 14 is a diagram showing an example of the configuration of the mobile terminal device when the functional components are implemented as hardware. The mobile terminal device includes a "CPU (central processing unit)" (2001), a "main memory" (2002), a "HDD" (2003), a "network I / F" (2004), and an "I / O" (2008). , "System Bus" (2005), and the external device (2006) has "I / F" (2007).

  With reference to these figures, the characteristic parts of the present embodiment out of the respective hardware components in each processing in this device will be described, and other parts will be the same as the description of the above-mentioned embodiments. is there.

  In the mobile terminal device according to the present embodiment, a stress evaluation result transmission program is added as the content of the program stored in the "main memory".

  Therefore, in the operation of the "CPU", the stress evaluation information generated by the stress evaluation program is transmitted to, for example, the external device or the main memory or HDD of the statistical processing device (not shown).

<Process flow>
FIG. 15 is a flowchart showing an example of the flow of processing in the mobile terminal device of this embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded in a medium.

  It has a movie ON step (S2101), a light ON step (S2102), a pupil recognition step (S2103), a pupil change calculation step (S2104), a stress evaluation step (S2105), and a stress evaluation result transmission step (S2106). With reference to these drawings, the characteristic part of the present embodiment in the flow of each processing in the present device will be described, and the other parts are the same as the description of the above-mentioned embodiments.

  The processing in this embodiment includes a stress evaluation result transmission step. The "stress evaluation result transmission step" is a step of transmitting the stress evaluation result obtained in the stress evaluation step to a predetermined address. The destination may be, for example, a statistical processing device provided for statistical processing.

  As described above, it is possible to save the storage area of the mobile terminal device necessary for accumulating the stress evaluation results. Further, by performing the statistical processing on a device other than the mobile terminal, the load on the mobile terminal device can be reduced.

<Outline>
The invention of the present embodiment is a program in which a function for calculating a change in the pupil of a captured image by a unit of time is recorded in a portable terminal device so that the change in the pupil when light is applied is continuously recorded. Is. In addition, the program is a portable terminal device in which the program is readable and executable. Note that the present invention also includes the configuration of any one of the second to fifth embodiments added to the configuration of the present embodiment.

  Usually, the resolution of still images is higher than that of moving images. Therefore, by using a continuous still image instead of the moving image, the resolution is increased and the boundary portion of the pupil is accurately recognized.

  The functions and hardware contents of the mobile terminal device of this embodiment and the flow of processing will be described in detail below.

<Functional configuration>
FIG. 16 is an example of a functional block of the mobile terminal device of this embodiment. As shown in this figure, the portable terminal device (2501) of this embodiment has a camera unit (2502), a continuous ON unit (2503), a light unit (2504), a light ON unit (2505), and a pupil recognition unit (2506). ) And a pupil change calculation unit (2507). The characteristic function of this embodiment is in the continuous ON section. Therefore, in this embodiment, the function of the continuous ON unit will be mainly described.

  The "continuous ON section" functions to turn ON the continuous still image shooting function of the mobile terminal. Even in this case, in order to investigate the pupillary light reaction, it is necessary to keep the light in the pupil even when the continuous still images are not photographed. Therefore, the “light ON part” is the same as in the above-described embodiment. , The "light section" is continuously turned on.

  The "shooting" in the continuous ON section includes the case of saving still images taken at the same time, but the saving is not essential. For example, this embodiment also has a mode in which the continuous ON unit simply shoots a continuous still image, and the continuous still image during the shooting is calculated in real time by a “pupil recognition unit” and a “pupil change calculation unit” to be described later. Are included in.

  It is premised that the continuous still image being photographed here includes the pupil, but it is desirable that not only the pupil but also a part of the body around the pupil other than the pupil is included. The health condition of a person's eyes appears in various forms on the human eyeball. For example, in allergic conjunctivitis, infectious conjunctivitis, uveitis, dry eye, and the like, hyperemia of the eye may be caused. Therefore, there is an effect that the above-mentioned symptom can be known by the fact that, for example, red blood in the conjunctiva occurs.

  The shooting interval of the continuous still images is preferably short. This is because if it is short, it is possible to obtain information close to a moving image, and it is possible to obtain the time required for accurate pupil minimization, the miosis ratio, the miosis speed, and miosis acceleration. The short interval is, for example, one still image is taken every two seconds, and one still image is taken every one second.

  Further, since this photographing interval affects the calculation in the pupil change calculation unit, it is desirable that it exists as information associated with the pupil change information. For example, if one still image is captured in one second, the still image reveals a change in the pupil every second.

<Hardware configuration>
FIG. 17 is a diagram showing an example of the configuration of the mobile terminal device when the functional components are implemented as hardware. The operation of each hardware configuration unit in each processing in this device will be described with reference to this drawing.

  As shown in this figure, the mobile terminal device of this embodiment includes a "CPU" (2601) for performing various arithmetic processes and a "main memory" (2602). In addition, a “HDD” (2603), a “network I / F” (2604) for transmitting / receiving information to / from an “I / F” (2607) of an external device (2606), a camera, a light, a display, etc. It also has an "I / O" (2608) for transmitting and receiving information between them. Then, they are mutually connected by a data communication path such as a "system bus" (2605), and send / receive and process information.

  The programs stored in this embodiment are a continuous ON program that turns on the moving image shooting function of the mobile terminal, a light ON program that turns on the light provided on the shooting side of the mobile terminal, and an image being shot. A pupil recognition program for recognizing the pupils of eyes of animals including humans, and a pupil change calculation program for calculating a pupil change which is a temporal change in the degree of opening of the recognized pupils. The characteristic program in this embodiment is a continuous ON program. The other programs are the same as those described above.

  The pupil image of the user acquired by the continuous ON program of the mobile terminal device is stored in the addresses of the “main memory” and the “HDD”. Then, the "CPU" of the mobile terminal device calculates the temporal change of the pupil recognized by the logical operation processing using the continuous still image information stored in the "main memory". In addition, when the captured continuous still images are stored, the continuous still image information is stored in the “HDD” in this way. On the other hand, when the captured continuous still images are recognized and calculated in real time, the continuous continuous image information is stored in the “HDD”. The still image information will not be stored in the “HDD”.

<Process flow>
FIG. 18 is a flowchart showing an example of the flow of processing in the mobile terminal device of this embodiment. Note that the steps shown below may be processing steps forming a program for controlling a computer (mobile terminal) recorded in a medium.

  As shown in this figure, first, the continuous still image capturing function of the mobile terminal is turned on (S2701). Next, the light provided on the photographing side of the mobile terminal is turned on (S2702). Then, the pupils of the eyes of animals including humans are recognized from the image being photographed (S2703). Finally, a pupil change, which is a change with time of the recognized pupil opening degree, is calculated (S2704).

  The continuous ON step (S2701) for turning on the continuous still image shooting function of the mobile terminal and the light ON step (S2702) for turning on the light provided on the shooting side of the mobile terminal are in reverse order. Is also good. That is, first, the light provided on the photographing side of the mobile terminal is turned on. Next, the continuous still image shooting function of the mobile terminal is turned on. Then, the pupils of the eyes of animals including humans are recognized from the image being photographed. Finally, the flow of processing of calculating a change in the pupil, which is a change with time of the recognized degree of opening of the pupil, may be used.

  Through the above steps, the state of stress can be measured by using the mobile terminal device according to the objective symptom correlated with the stress.

<Outline>
As a more preferable embodiment, the present embodiment provides a statistical processing device connected to the mobile terminal device having the configuration of the fourth embodiment or the sixth embodiment having the pupil change transmitting unit. The outline of the configuration of the statistical processing device is that the pupil change information is received from the mobile terminal device, the pupil change information is accumulated, and the statistical processing is performed.

  By performing such statistical processing, for example, the user has an advantage in that an appropriate diagnosis can be obtained and an appropriate future prediction can be made. In addition, from the point of view of the provider, a huge amount of user information is acquired to improve the accuracy of diagnosis, a paid information providing service is provided, and it is used in the provision and development of products or services such as supplements / medicines and healing music, medical treatment. There is a merit that it can be used for research. This applies not only to the present embodiment but also to statistical processing in other embodiments.

  Hereinafter, the functions and hardware contents of the apparatus and the flow of processing in this embodiment will be described in detail.

<Functional configuration>
FIG. 19 is a diagram showing an example of functional blocks of the mobile terminal device and the statistical processing device of this embodiment. As shown in this figure, the portable terminal device (2801) of the present embodiment has a camera unit (2801), a moving image ON unit (2803), a light unit (2804), a light ON unit (2805), and a pupil recognition unit (2806). ), A pupil change calculation unit (2807), and a pupil change transmission unit (2808). The statistical processing device (2809) in the present embodiment has a pupil change receiving unit (2810), a pupil change accumulating unit (2811), and a pupil change statistical processing unit (2812). The pupil change statistical processing unit is connected to a user (2813), a manager (2814), a third party (2815) such as a medical institution. The user (2813) includes the mobile terminal device that has transmitted the pupil change from the pupil change transmitting unit to the statistical processing device. The feature of this embodiment resides in the contents of each part of the statistical processing device. Therefore, the functional configuration of each unit of the statistical processing device will be mainly described. Since the other functions are the same as those described above, the description thereof will be omitted.

  The “pupil change receiving unit” functions to receive the pupil change transmitted from the mobile terminal device. Therefore, for example, in addition to the diameter value and the area value, the statistical processing device receives information such as the time required until the pupil is minimized, the miosis ratio, the miosis speed, and the miosis acceleration. In addition, in accordance with this, name, gender, date of birth, birthplace, measurement time, measurement location (for example, residence, university, workplace, pub, train, park, etc.), before and after meals, after meals Includes food ingredients, occupation, presence / absence and intensity of work on the measurement day, and biometric data such as body temperature, heartbeat, blood pressure, skin electrical resistance, body movement, respiration, and fluctuations, as well as additional information such as body composition data and treatment history. It is desirable to also receive. It is desirable that the additional information is held in association with the pupil change information so that it can be easily used in the subsequent statistical processing.

  The “pupil change storage unit” functions to store received pupil changes. As for the mode of accumulation, first, a mode of accumulating for each user can be considered. Then, the change in the user's stress state can be grasped in the form of comparison of the accumulated pupil change information. Next, there is a method of dividing by date without the user dividing. According to this method, it is possible to store data for analyzing, for example, the influence of weather on stress.

  The “pupil change statistical processing unit” functions to statistically process the accumulated pupil changes. As the statistical processing, as described above, the method of performing the statistical processing for each user, the method of performing the statistical processing for each date, the method of performing the statistical processing for each additional information, and the like can be considered.

  As described above, the merits of performing the statistical processing are that the user can obtain an appropriate diagnosis and that an appropriate future prediction can be made, and from the provider, it is enormous. The purpose is to obtain accurate user information to improve the accuracy of diagnosis, to provide a paid information providing service, to be used for the provision and development of products or services such as supplements / medicines and healing music, and for medical research. Therefore, the statistical processing device may have a function of disclosing the result of the statistical processing to the user or the like. As a function for disclosing, for example, in a statistical processing device, a statistical information transmission unit for transmitting statistical information to a terminal such as a user is provided, and the statistical processing device transmits the statistical information to the terminal such as a user. A configuration including a statistical information receiving unit for receiving the statistical information that has been obtained is conceivable. In addition, if the terminal of the user or the like has a statistical information output unit for outputting the statistical information, for example, the user can recognize the statistical information through the display or the like of the mobile terminal device.

  For example, a method may be conceived in which a user inputs the degree of stress that he or she is aware of, and the degree of stress is compared with the degree of stress induced by a pupil change. FIG. 41 is a diagram in which the horizontal axis represents the miosis speed and the vertical axis represents the stress level. When the data as the statistic value becomes like the curve (5601), even if the stress level as the subjective symptom of the user is Y1 (5602), the stress level is Y2 in terms of the miosis speed. In the case of (5603), it is possible to diagnose that the degree of stress is equivalent to the portion of Y2 based on the statistical processing and the actual miosis speed, and output that effect. By doing so, the user can obtain an appropriate diagnosis.

  Next, FIG. 42 is a diagram showing a stress change with time during a specific work when the miosis speed is plotted on the horizontal axis and the stress level is plotted on the vertical axis. The curve (5701) derived as a result of the statistical processing represents stress changes with time, such as, for example, the Z1 part one hour after the Z0 part and the Z2 part one hour thereafter. .. In this figure, for example, if the stress state at the present time of the specific work is the portion Z2, the corresponding portion three hours later is the portion Z5, and the stress intensity is the portion Y3 (5702). Can be predicted. In this way, by statistically processing the stress change over time, an appropriate future prediction can be made.

  As described above, if the additional information is also received at the same time, the statistical processing can be performed for each weather, meal, work environment, hobby, etc., as well as the change for each time. Then, by comprehensively combining these, it is possible to perform highly accurate stress diagnosis and obtain information for identifying the cause of stress. Further, the cause of stress may be a cause of stress for each individual, and in the case of matters common to all users, it can be considered that the cause of stress is generally appropriate. In the case of a stress reason that is generally applicable in this way, the value of the information that tells the fact becomes socially high, which leads to providing information for a fee. Further, in addition to identifying the cause of stress, if a factor that reduces stress can be found, it may have a function of presenting that factor or a solution that eliminates the cause of stress. The significance and use of these statistical processes are not limited to the present embodiment, and are applicable to all statistical processes in the present invention.

<Hardware configuration>
FIG. 20 is a diagram showing an example of the configuration of the mobile terminal device and the statistical processing device when the functional components are implemented as hardware. The mobile terminal device includes a "CPU (central processing unit)" (2901), a "main memory" (2902), an "HDD" (2903), a "network I / F" (2904), and an "I / O" (2912). , "System bus" (2905), and the statistical processing unit (2906) has an "I / F" (2907), a "display" (2908), a "main memory" (2909), and a "CPU" ( 2910) and a "frame memory" (2911).

  With reference to these figures, the characteristic parts of the present embodiment out of the respective hardware components in each processing in this device will be described, and other parts will be the same as the description of the above-mentioned embodiments. is there.

  The feature of this embodiment lies in the statistical processing device. In the mobile terminal device, the moving image information and the pupil change information stored in the "main memory" of the mobile terminal device are transmitted to the statistical processing device through the "network I / F" by the "pupil change transmission program", and the statistical processing device In the above, the above information is received by the "pupil change receiving program" through the "I / F", stored in the "main memory" or the "HDD" (not shown) on the statistical processing device, and stored in the "pupil change storing program". The pupil change information is stored in the "main memory" or a "HDD" (not shown). The accumulated pupil change information is statistically processed by the "CPU" of the statistical processing device using a "pupil change statistical processing program".

  Further, as described in the functional configuration, the "main memory" of the statistical processing device may store a statistical information transmission program for transmitting the statistical processing result to the mobile terminal, or The "main memory" may store a statistical information reception program for receiving the transmitted statistical processing result from the statistical processing device and a statistical information output program for outputting the statistical information.

<Process flow>
FIG. 21 is a flow chart showing an example of the flow of processing in the mobile terminal device and the statistical processing device of this embodiment. The steps shown below may be processing steps constituting a program for controlling the computers (mobile terminal and statistical processing terminal) recorded in the medium.

  The mobile terminal device has a moving image ON step (S3001), a light ON step (S3002), a pupil recognition step (S3003), a pupil change calculation step (S3004), and a pupil change transmission step (S3005). Then, the statistical processing device has a pupil change receiving step (S3006), a pupil change accumulating step (S3007), and a pupil change statistical processing step (S3008). With reference to these drawings, the characteristic part of the present embodiment in the flow of each processing in the present device will be described, and the other parts are the same as the description of the above-mentioned embodiments.

  In the processing in this embodiment, processing steps in the statistical processing device are added. The "pupil change receiving step" in the statistical processing device is a stage in which the pupil change transmitted as a result of executing the pupil change transmitting step in the mobile terminal device is received. Further, the "pupil change accumulation step" in the statistical processing device is a step of accumulating the received pupil change. Then, the "pupil change statistical processing step" in the statistical processing device is a step of statistically processing the accumulated pupil changes.

  As described in the functional configuration, the aspect of the statistical processing is not limited to each user. Further, for example, when it is disclosed to the user by associating it with information such as the brightness of the measured place, the measurement time, the difference between before and after meal, the sleeping time on the previous day, the presence or absence of work on the measurement day, and the intensity, This contributes to analysis of the cause of user stress.

  As a device for disclosing the statistical information to the user, the statistical processing device further includes a statistical information transmitting step for transmitting the statistical information to the mobile terminal, and the portable terminal device is provided with the statistical processing device. It is also conceivable that the processing flow includes a statistical information receiving step for receiving the transmitted statistical information and a statistical information outputting step for outputting the received statistical information.

  With such a processing flow, it is possible to perform statistical processing of the pupillary light reaction, and thus it is possible to generate information for the user to know the characteristics of the cause of his / her stress and the like.

<Outline>
As a more preferable embodiment, the present embodiment provides a statistical processing device connected to the mobile terminal device in which the configuration of the fifth embodiment is added to the fourth embodiment. As an outline of the configuration of the statistical processing device, the stress evaluation result and the pupillary change are received from the mobile terminal device, accumulated, and the statistical processing is performed.

  Hereinafter, the functions and hardware contents of the apparatus and the flow of processing in this embodiment will be described in detail.

<Functional configuration>
FIG. 22 is a diagram showing an example of functional blocks of the mobile terminal device and the statistical processing device of the present embodiment. As shown in this figure, the portable terminal device (3401) of the present embodiment has a camera unit (3402), a moving image ON unit (3403), a light unit (3404), a light ON unit (3405), and a pupil recognition unit (3406). ), A pupil change calculation unit (3407), a stress evaluation unit (3408), a stress evaluation result transmission unit (3409), and a pupil change transmission unit (3414). The statistical processing device (3410) according to the present exemplary embodiment includes a stress evaluation result receiving unit (3411), a stress evaluation result accumulating unit (3412), a stress evaluation statistical processing unit (3413), a pupil change receiving unit (3415), and a pupil. It has a change accumulation unit (3416). The stress evaluation statistical processing unit is connected to a user (3417), a manager (3418), a third party (3419) such as a medical institution. The feature of this embodiment resides in the contents of each part of the statistical processing device. Therefore, the functional configuration of each unit of the statistical processing device will be mainly described. Since the other functions are the same as those described above, the description thereof will be omitted.

  The “stress evaluation result receiving unit” functions to receive the stress evaluation result transmitted from the mobile terminal device. Therefore, for example, the statistical processing device may receive a stress evaluation that indicates the degree of stress by a numerical value such as%, a level such as large / medium / small, and a comparison value with the previous day's ratio or maximum / minimum values. Become.

  The “stress evaluation result storage unit” functions to store the received stress evaluation result. As for the mode of accumulation, a mode of accumulating for each user can be considered. Then, the change in the stress evaluation result of the user can be grasped in the form of the accumulated change in the stress evaluation result. Next, there is a method of dividing by date without the user dividing. According to this method, it is possible to store for analyzing the influence of the weather on the stress evaluation result, for example.

  The “stress evaluation statistical processing unit” has a function of statistically processing the accumulated stress evaluation result and pupil change. As the statistical processing, as described above, the method of performing the statistical processing for each user, the method of performing the statistical processing for each date, and the like can be considered.

  Furthermore, as described in the above embodiment, the statistical processing device may have a function of disclosing the result of the statistical processing to the user or the like. As a function for disclosing, for example, in a statistical processing device, a statistical information transmission unit for transmitting statistical information to a terminal such as a user is provided, and the statistical processing device transmits the statistical information to the terminal such as a user. A configuration including a statistical information receiving unit for receiving the statistical information that has been obtained is conceivable.

  FIG. 47 is a diagram showing an aspect of output on the user side. 6 is a graph in which the vertical axis represents the stress evaluation value and the horizontal axis represents the pupil change rate. It can be seen that the higher the pupil change speed, the smaller the stress evaluation value.

  Further, FIG. 48 is a diagram showing an aspect of output on the user side. 6 is a graph in which the vertical axis represents stress evaluation values and the horizontal axis represents body composition. Regarding the body fat percentage, body weight, BMI, and pulse, it can be seen that the higher the body composition amount, the higher the stress evaluation value. On the other hand, regarding basal metabolism, it can be seen that the higher the body composition amount, the lower the stress evaluation value. If such statistical information is obtained, the user or the like can objectively analyze the cause of stress. Moreover, the stress evaluation value has a clearer correlation with the stress than the comparison with the change in the pupil, so that it is easy to understand at first glance.

  As described above, it is possible to perform the stress evaluation process in the statistical processing device, which is performed by the user's mobile terminal device that does not have the stress evaluation process, and then to receive the stress evaluation process.

<Hardware configuration>
FIG. 23 is a diagram showing an example of the configuration of the mobile terminal device and the statistical processing device when the functional components are implemented as hardware. The mobile terminal device includes a "CPU (central processing unit)" (3501), a "main memory" (3502), a "HDD" (3503), a "network I / F" (3504), and an "I / O" (3512). , "System bus" (3505), the statistical processing unit (3506), "I / F" (3507), "display" (3508), "main memory" (3509), "CPU" ( 3510) and a "frame memory" (3511).

  With reference to these figures, the characteristic parts of the present embodiment out of the respective hardware components in each processing in this device will be described, and other parts will be the same as the description of the above-mentioned embodiments. is there.

  The feature of this embodiment lies in the statistical processing device. In the mobile terminal device, the stress evaluation information stored in the “main memory” of the mobile terminal device is transmitted to the statistical processing device through the “network I / F” by the “stress evaluation result transmission program”. , The information is received by the "stress evaluation result receiving program" through the "I / F", and is stored in the "main memory" on the statistical processing device or "HDD" (not shown), and the "stress evaluation result storage program" is stored. Thus, the stress evaluation information is stored in the "main memory" or an "HDD" (not shown). The accumulated stress evaluation information is statistically processed by the “CPU” of the statistical processing device using a “stress evaluation statistical processing program”.

  Furthermore, a statistical information transmission program for transmitting the statistical processing result to the mobile terminal may be stored in the "main memory" of the statistical processing device, and the statistical information transmission program may be stored in the "main memory" of the portable terminal device. A statistical information reception program for receiving the statistical processing result from the statistical processing device and a statistical information output program for outputting the statistical information may be stored.

<Process flow>
FIG. 24 is a flowchart showing an example of the flow of processing in the mobile terminal device and the statistical processing device of this embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal, statistical processing terminal) recorded in a medium.

  The mobile terminal device has a video ON step (S3601), a light ON step (S3602), a pupil recognition step (S3603), a pupil change calculation step (S3603), a stress evaluation step (S3604), a stress evaluation result transmission step (S3605), It has a pupil change transmission step (S3606). Then, the statistical processing device performs a pupil change receiving step (S3607), a pupil change accumulating step (S3608), a stress evaluation result receiving step (S3609), a stress evaluation result accumulating step (S3610), and a stress evaluation statistical processing step (S3611). Have With reference to these drawings, the characteristic part of the present embodiment in the flow of each processing in the present device will be described, and the other parts are the same as the description of the above-mentioned embodiments.

  In the processing in this embodiment, processing steps in the statistical processing device are added. The “stress evaluation result receiving step” in the statistical processing device is a stage in which the stress evaluation result transmitted as a result of executing the stress evaluation result transmitting step in the mobile terminal device is received. This stress evaluation result will be sent in association with the pupil change. As a result, the burden of stress evaluation in the statistical processing device can be reduced. The “stress evaluation result accumulating step in the statistical processing device” is a step of accumulating the received stress evaluation results. The "stress evaluation statistical processing step" in the statistical processing device is a step of statistically processing the accumulated stress evaluation results.

  The “stress evaluation result receiving step” does not matter as long as it is after the stress evaluation result transmitting step in the mobile terminal device, the pre-post relationship in the statistical processing device. For example, it may be before the pupil change receiving step. Next, the “stress evaluation result accumulating unit” may have any predecessor relationship with other processes as long as it is after the body composition data receiving step in the statistical processing device. Similarly, regarding the “pupil change receiving step” and the “pupil change accumulating step”, the predecessor relationship with the stress evaluation result receiving step and the stress evaluation result accumulating step does not matter. However, since the "stress evaluation statistical processing step" uses the accumulated pupil change and the stress evaluation result, it is performed after both the stress evaluation result accumulating step and the pupil change accumulating step are completed.

  The mode of statistical processing is not limited to each user. Further, for example, when it is disclosed to the user by associating it with information such as the brightness of the measured place, the measurement time, the difference between before and after meal, the sleeping time on the previous day, the presence or absence of work on the measurement day, and the intensity, Contributes to analysis of user body composition data.

  As a device for disclosing the statistical information to the user, the statistical processing device further includes a statistical information transmitting step for transmitting the statistical information to the mobile terminal, and the portable terminal device is provided with the statistical processing device. It is also conceivable that the processing flow includes a statistical information receiving step for receiving the transmitted statistical information and a statistical information outputting step for outputting the received statistical information.

<Outline>
According to the invention of the present embodiment, a mobile terminal device is provided with a function for capturing a moving image of a change in the pupil when light is applied, retaining moving image information for calculating the change in the pupil in units of time, and transmitting the information to an external device. It is a program that is recorded so that it can be read and executed. In addition, the program is a portable terminal device in which the program is readable and executable.

  The functions and hardware contents of the mobile terminal device of this embodiment and the flow of processing will be described in detail below.

<Functional configuration>
FIG. 25 is an example of functional blocks of the mobile terminal device of the present embodiment. As shown in this figure, the mobile terminal device (4001) of this embodiment has a camera unit (4002), a moving image ON unit (4003), a light unit (4004), a light ON unit (4005), and a pupil recognition unit (4006). ), A moving image holding unit (4007), and a moving image transmitting unit (4008). The feature of this embodiment resides in the contents of the moving image holding unit and the moving image transmitting unit. Therefore, in the following, the contents of the moving image holding unit and the moving image transmitting unit will be mainly described.

  In all of the above-mentioned embodiments, the calculation of the pupil change is performed in the mobile terminal device. This is excellent in that the calculation can be completed in the mobile terminal device, but in order to achieve the object of the present invention, the calculation of the pupil change is not necessarily completed in the mobile terminal device. You don't have to. Therefore, the mobile terminal device in the present embodiment holds the moving image in which the pupil is recognized by the pupil recognition unit in the mobile terminal device, and transmits it to the external device, on the assumption that the calculation of the pupil change is performed in the external device. The object of the present invention is achieved by providing such a function. That is, it is premised that the external device has a function equivalent to the pupil change calculation unit in the first embodiment, and further, it is desirable that it has a function equivalent to the stress evaluation unit in the third embodiment.

  The “moving image holding unit” functions to hold the moving image shot by the moving image ON unit. The retained moving image may have a state in which the pupil portion is distinguished from the non-pupil portion.

  The “moving image transmitting section” functions to transmit the retained moving image to a predetermined address. Thereby, the moving image information of the user's pupil can be transmitted to the external device.

  As a result, the stress state can be measured in the external device by the objective symptom correlated with the stress without performing the pupil change calculation in the mobile terminal device. Then, this moving image information is accumulated in the frame memory of the external device and is output on the display or the like of the external device for each predetermined frame. Further, it is desirable that the stress-evaluated information and the like be displayed in a list on a user terminal including a mobile terminal device, as in the other examples.

  As described above, it is possible to reduce the processing in the mobile terminal device and reduce the load on the mobile terminal device.

<Hardware configuration>
FIG. 26 is a diagram showing an example of the configuration of the mobile terminal device when the functional components are implemented as hardware. The operation of each hardware configuration unit in each processing in this device will be described with reference to this drawing.

  As shown in this figure, the mobile terminal device of this embodiment includes a "CPU" (4101) for performing various arithmetic processes and a "main memory" (4102). In addition, a “HDD” (4103), a “network I / F” (4104) for transmitting and receiving information to and from an “I / F” (4107) of an external device (4106), a camera, a light, a display, and the like. An “I / O” (4108) for transmitting and receiving information between the two is also provided. Then, they are mutually connected by a data communication path such as a “system bus” (4105) and perform transmission / reception and processing of information. The external device includes a “frame memory” (4109), a “display” (4110) and a “CPU” (4111).

  Further, in the present embodiment, the programs stored in the "main memory" are a moving picture ON program for turning on the moving picture shooting function of the mobile terminal and a light ON for turning on the light provided on the shooting side of the mobile terminal. A program, a pupil recognition program for recognizing a pupil of an eye of an animal including a person from an image being photographed, a moving image transmission program for transmitting a retained moving image to a predetermined address, and the like.

  The pupil image of the user acquired by the moving image ON program of the mobile terminal device is stored in the addresses of the “main memory” and the “HDD”. Then, the "CPU" of the mobile terminal device transmits the moving image information stored in the "main memory" to the "I / F" of the external device through the "network I / F".

<Process flow>
FIG. 27 is a flowchart showing an example of the flow of processing in the mobile terminal device of this embodiment. Note that the steps shown below may be processing steps forming a program for controlling a computer (mobile terminal) recorded in a medium.

  As shown in this figure, first, the moving image capturing function of the mobile terminal is turned on (S4201). Next, the light provided on the photographing side surface of the mobile terminal is turned on (S4202). Next, the pupils of the eyes of animals including humans are recognized from the image being photographed (S4203). Then, the moving image captured in the moving image ON step is held. Finally, the held moving image is transmitted to a predetermined address (S4204).

  The moving picture ON step (S4201) for turning on the moving picture shooting function of the mobile terminal and the light ON step (S4202) for turning on the light provided on the shooting side surface of the mobile terminal may be reversed in order. .. That is, first, the light provided on the photographing side of the mobile terminal is turned on. Next, the video recording function of the mobile terminal is turned on. Next, the pupils of the eyes of animals including humans are recognized from the image being photographed. Then, the moving image captured in the moving image ON step is held. Finally, the flow of processing may be such that the held moving image is transmitted to a predetermined address.

  Through the above steps, the stress state can be measured by the objective symptom having the correlation with the stress without performing the pupil change calculation in the mobile terminal device.

<Outline>
The invention of the present embodiment is a function for capturing a continuous still image of the change of the pupil when light is applied, holding continuous still image information for calculating the pupil change in time units, and transmitting the information to an external device. Is a program that is readable and executable to be recorded in a mobile terminal device. In addition, the program is a portable terminal device in which the program is readable and executable.

  The functions and hardware contents of the mobile terminal device of this embodiment and the flow of processing will be described in detail below.

<Functional configuration>
FIG. 28 is an example of functional blocks of the mobile terminal device of the present embodiment. As shown in this figure, the mobile terminal device (4301) of the present embodiment has a camera unit (4302), a continuous ON unit (4303), a light unit (4304), a light ON unit (4305), and a pupil recognition unit (4306). ), A continuous still image holding unit (4307) and a continuous still image transmitting unit (4308). The feature of this embodiment resides in the contents of the continuous still image holding unit and the continuous still image transmitting unit. Therefore, in the following, the contents of the moving image holding unit and the moving image transmitting unit will be mainly described.

  In the present embodiment, as in the case of the ninth embodiment, on the premise that the pupil change is calculated in the external device, the portable terminal device holds the continuous still image in which the pupil is recognized by the pupil recognition unit, and the external device stores the continuous still image. It is intended to achieve the object of the present invention by providing a function of transmitting. The difference from the ninth embodiment is that the image to be captured is a continuous still image, and thus the image to be held and transmitted is a continuous still image. The point of capturing a continuous still image has already been described in the above-described embodiment, so the continuous still image holding unit and the continuous still image transmitting unit will be described below.

  The “continuous still image holding unit” has a function of holding continuous still images captured by the mobile terminal. The retained moving image may have a state in which the pupil portion is distinguished from the non-pupil portion.

  The “continuous still image transmitting unit” has a function of transmitting the retained continuous still images to a predetermined address. As a result, the continuous still image information obtained by photographing the user's pupil can be transmitted to the external device.

  As a result, the stress state can be measured by the objective symptom correlated with the stress without performing the pupil change calculation in the mobile terminal device.

<Hardware configuration>
FIG. 29 is a diagram showing an example of the configuration of the mobile terminal device when the functional components are implemented as hardware. The operation of each hardware configuration unit in each processing in this device will be described with reference to this drawing.

  As shown in this figure, the mobile terminal device of this embodiment includes a "CPU" (4401) for performing various arithmetic processes and a "main memory" (4402). Further, a “HDD” (4403), a “network I / F” (4404) for transmitting / receiving information to / from an “I / F” (4407) of an external device (4406), a camera, a light, a display, etc. It also has an "I / O" (4408) for transmitting and receiving information between them. Then, they are connected to each other by a data communication path such as a "system bus" (4405) and perform transmission / reception and processing of information.

  Further, the "main memory" provides a work area which is a work area of the program at the same time as the program for performing various processes is read for execution by the "CPU". Further, a plurality of addresses are assigned to each of the "main memory", "HDD", and "flash memory" (not shown), and the program executed by the "CPU" identifies each address and accesses each other. It is possible to exchange data with and process. The programs stored in this embodiment are a continuous ON program for turning on the continuous still image shooting function of the mobile terminal, a light ON program for turning on the light provided on the shooting side of the mobile terminal, and a shooting A pupil recognition program for recognizing the pupils of the eyes of animals including humans from the images, and a continuous still image transmission program for transmitting the retained continuous still images to a predetermined address.

  The pupil image of the user acquired by the continuous ON program of the mobile terminal device is stored in the addresses of the “main memory” and the “HDD”. Then, the "CPU" of the mobile terminal device transmits the continuous still image information stored in the "main memory" to the "I / F" of the external device through the "network I / F". is there.

<Process flow>
FIG. 30 is a flowchart showing an example of the flow of processing in the mobile terminal device of the present embodiment. Note that the steps shown below may be processing steps forming a program for controlling a computer (mobile terminal) recorded in a medium.

  As shown in this figure, first, the continuous still image capturing function of the mobile terminal is turned on (S4501). Next, the light provided on the photographing side of the mobile terminal is turned on (S4502). Next, the pupils of the eyes of animals including humans are recognized from the image being photographed (S4503). Then, the continuous still images captured in the continuous ON step are held. Finally, the held continuous still image is transmitted to a predetermined address (S4504).

  The continuous ON step (S4501) for turning on the continuous still image shooting function of the mobile terminal and the light ON step (S4502) for turning on the light provided on the shooting side of the mobile terminal are in reverse order. Is also good. That is, first, the light provided on the photographing side of the mobile terminal is turned on. Next, the continuous still image shooting function of the mobile terminal is turned on. Next, the pupils of the eyes of animals including humans are recognized from the image being photographed. Then, the continuous still images captured in the continuous ON step are held. Finally, the processing flow of transmitting the held continuous still images to a predetermined address may be used.

  Through the above steps, the stress state can be measured by the objective symptom having the correlation with the stress without performing the pupil change calculation in the mobile terminal device.

<Outline>
This embodiment provides a computing device having a function for computing the pupillary light reaction based on a moving image received from a mobile terminal device.

  Hereinafter, the functions and hardware contents of the apparatus and the flow of processing in this embodiment will be described in detail.

<Functional configuration>
FIG. 55 is a diagram showing an example of functional blocks of the arithmetic unit of this embodiment. As shown in this figure, the arithmetic unit (7301) of the present embodiment has a moving image receiving unit (7302), a pupil recognizing unit (7303) and a pupil change calculating unit (7304). In addition, in this embodiment, the mobile terminal normally has the camera unit, the light unit, the moving image ON unit, and the light ON unit in the first embodiment. The characteristics of the camera unit, the light unit, the moving image ON unit, and the light ON unit of the mobile terminal in that case are the same as those of the first embodiment. Further, the mobile terminal normally further includes a moving image transmitting unit for transmitting a moving image to the arithmetic device according to the present embodiment.

  The “video receiving unit” functions to receive a video including the eyes of animals including humans from the mobile terminal (7305). This allows the user to receive a moving image including the eyes of animals including humans.

Further, the pupil recognizing unit has a function of recognizing the pupil of the eyes of animals including humans from the received moving image. Further, the pupil change calculation unit has a function of calculating a pupil change, which is a change with time of the recognized opening degree of the pupil.
The details of the functions of the pupil recognition unit and the pupil change calculation unit are the same as the details of the pupil recognition unit and the pupil change calculation unit of the mobile terminal device according to the first embodiment.

  Furthermore, the arithmetic device may include a notification transmitting unit that transmits a notification to the user when the pupil cannot be recognized in the received moving image. By doing so, the user can capture and transmit the moving image again by receiving the notification, and thus it is possible to improve the situation in which the arithmetic device cannot calculate the pupil change. In order to further enhance such an effect, the notification may be provided with a reason adding means for adding the reason why the notification cannot be recognized.

  Furthermore, by transmitting the pupil change information, which is the calculation result, to the user, the user can know his / her stress state. Also, if it is provided to a third party other than the user, the third party can objectively understand the stress, so that it will be meaningful information at the time of medical examination in a medical institution. In that case, it also helps prevent illness and accidents due to stress. As described above, it is desirable to have a function of transmitting the pupil change information to, for example, the administrator (7306), the user (7307), and the third party (7308). Therefore, the arithmetic device may have a pupil change transmitting unit that transmits a pupil change to a predetermined address.

  As a result, the camera terminal, the light section, the moving picture ON section, the light ON section, and the moving picture transmitting section are all that is required for the mobile terminal device, so that the mobile terminal such as a smartphone has a special function. The operation can be performed without doing so. Therefore, the user does not necessarily have to prepare a dedicated terminal or a dedicated application, so that the present invention can be used easily.

<Hardware configuration>
FIG. 56 is a diagram showing an example of the configuration of the arithmetic device when the functional components are implemented as hardware. The arithmetic unit is a "CPU" (7401), "main memory" (7402), "HDD" (7403), "network I / F" (7404), "I / O" (7408), "system bus" ( 7405) and is connected to the mobile terminal device (7406) through the “network I / F”.

  With reference to these figures, each hardware configuration unit in each processing in this device will be described.

  In the arithmetic unit according to the present embodiment, the contents of the programs stored in the "main memory" are a moving image reception program, a pupil recognition program, and a pupil change calculation program.

  In the operation of the "CPU", the moving image receiving program is executed to receive the moving image including the eyes of animals including a person transmitted from the mobile terminal device through the "network I / F", and the received moving image is , “HDD” and “main memory”. Then, the pupil recognition program is executed to recognize the pupils of eyes of animals including humans in the received moving image, and the pupil change calculation program is executed to change the degree of opening of the recognized pupil with time. That is, the pupil change is calculated. The calculation result is stored in the “HDD” and the “main memory” as pupil change information.

<Process flow>
FIG. 57 is a flow chart showing an example of the flow of processing in the arithmetic unit of this embodiment. Note that the steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal, arithmetic terminal) recorded in a medium.

  As shown in this figure, first, a moving image including the eyes of animals including humans is received from the mobile terminal (S7501). Next, the pupils of the eyes of animals including humans are recognized from the received moving image (S7502). Next, a pupil change, which is a change with time of the recognized pupil opening degree, is calculated (S7503).

  The details of the functions of the pupil recognition step (S7502) and the pupil change calculation step (S7503) are the same as the details of the pupil recognition step and the pupil change calculation step of the mobile terminal device according to the first embodiment.

  Further, the arithmetic device may have a notification sending step of sending a notification to the user when the pupil cannot be recognized in the received moving image. Further, the notification may include a reason addition sub-step for adding the reason why the notification cannot be recognized.

  Further, a pupil change transmission step of transmitting a pupil change to a predetermined address may be included.

  As a result, it is possible to carry out the calculation without having a special function on a mobile terminal such as a smartphone. Therefore, the user does not necessarily have to prepare a dedicated terminal or a dedicated application, so that the present invention can be used easily.

<Outline>
This embodiment provides a computing device having a function for computing the pupillary light reaction based on the continuous still images received from the mobile terminal device.

  Hereinafter, the functions and hardware contents of the apparatus and the flow of processing in this embodiment will be described in detail.

<Functional configuration>
FIG. 58 is a diagram showing an example of functional blocks of the arithmetic unit of this embodiment. As shown in this figure, the arithmetic unit (7601) of this embodiment has a continuous still image receiving unit (7602), a pupil recognizing unit (7603), and a pupil change calculating unit (7604). Further, in this embodiment, the mobile terminal normally has the camera unit, the light unit, the continuous ON unit, and the light ON unit in the sixth embodiment. The characteristics of the camera unit, the light unit, the continuous ON unit, and the light ON unit of the mobile terminal in that case are the same as those of the sixth embodiment. Further, the mobile terminal normally further includes a continuous still image transmitting unit for transmitting continuous still images to the arithmetic device according to the present embodiment.

  The "continuous still image receiving unit" functions to receive continuous still images including the eyes of animals including humans from the mobile terminal (7605). As a result, a continuous still image including the eyes of animals including humans can be received from the user.

  Furthermore, the pupil recognition unit has a function of recognizing the pupils of the eyes of animals including humans from the received continuous still images. In addition, the pupil change calculation unit has a function of calculating a pupil change, which is a time change of the degree of opening of the recognized pupil. The details of the functions of the pupil recognition unit and the pupil change calculation unit are the same as the details of the pupil recognition unit and the pupil change calculation unit of the mobile terminal device according to the first embodiment.

  Further, as in the eleventh embodiment, the arithmetic device may include a notification sending unit that sends a notification to the user when the pupil cannot be recognized in the received continuous still images. Further, the notification may be provided with a reason adding means for adding the reason why the notification cannot be recognized.

  Furthermore, it is desirable to have a function for transmitting the pupil change information to, for example, an administrator (7606), a user (7607), or a third party (7608). Therefore, the arithmetic device may have a pupil change transmitting unit that transmits a pupil change to a predetermined address.

  As a result, the camera terminal, the light section, the continuous ON section, the light ON section, and the continuous still image transmission section are all that is required for the mobile terminal device. It is possible to carry out the calculation without having. Therefore, the user does not necessarily have to prepare a dedicated terminal or a dedicated application, so that the present invention can be used easily.

<Hardware configuration>
FIG. 59 is a diagram showing an example of the configuration of the arithmetic device when the functional components are implemented as hardware. The arithmetic unit is a "CPU" (7701), "main memory" (7702), "HDD" (7703), "network I / F" (7704), "I / O" (7708), "system bus" ( 7705) and is connected to the mobile terminal device (7706) through the “network I / F”.

  With reference to these figures, each hardware configuration unit in each processing in this device will be described.

  In the arithmetic unit according to the present embodiment, the contents of the programs stored in the "main memory" are a continuous still image receiving program, a pupil recognition program, and a pupil change arithmetic program.

  In the operation of the "CPU", the continuous still image receiving program is executed to receive the continuous still image including the eyes of animals including a person transmitted from the mobile terminal device through the "network I / F", and The received continuous still images are stored in the “HDD” and the “main memory”. Then, the pupil recognition program is executed to recognize the pupils of the eyes of animals including humans from the received continuous still images, and the pupil change calculation program is executed to change the degree of opening of the recognized pupil over time. That is, the pupil change is calculated. The calculation result is stored in the “HDD” and the “main memory” as pupil change information.

<Process flow>
FIG. 60 is a flow chart showing an example of the flow of processing in the arithmetic unit of this embodiment. Note that the steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal, arithmetic terminal) recorded in a medium.

  As shown in this figure, first, a continuous still image including the eyes of animals including humans is received from the mobile terminal (S7801). Next, the pupils of the eyes of animals including humans are recognized from the received continuous still images (S7802). Next, a pupil change, which is a change with time of the recognized pupil opening degree, is calculated (S7803).

  The details of the functions of the pupil recognition step (S7802) and the pupil change calculation step (S7803) are the same as the details of the pupil recognition step and the pupil change calculation step of the mobile terminal device according to the first embodiment.

  Furthermore, the arithmetic device may include a notification sending step of sending a notification to the user when the pupil cannot be recognized in the received continuous still images. Further, the notification may include a reason addition sub-step for adding the reason why the notification cannot be recognized.

  Further, a pupil change transmission step of transmitting a pupil change to a predetermined address may be included.

  As a result, it is possible to carry out the calculation without having a special function on a mobile terminal such as a smartphone. Therefore, the user does not necessarily have to prepare a dedicated terminal or a dedicated application, so that the present invention can be used easily.

4601, 4701, 4801, 4901: Mobile terminal devices 4602, 4702, 4802: Operation buttons 4603, 4703, 4803: Earpiece 4604, 4704, 4804: Cameras 4605, 4705, 4805: Lights 4606, 4706, 4806: Screen 4902: Pupil part

In order to solve the above problems, the present invention provides a moving image ON step of turning on a moving image shooting function of a mobile terminal, a light ON step of turning on a light provided on a shooting side of the portable terminal, and an image being shot. The pupil recognition step that recognizes the pupil of the eyes of animals including humans, lowers the illuminance of the light until the pupil is recognized , and when the pupil recognition is completed, raises the illuminance of the light higher than until recognition. Changed, when the pupil reduction speed is equal to or less than a certain speed, an illuminance change step of lowering the illuminance of the light, and a pupil change calculation step of calculating a pupil change that is a time change of the recognized opening degree of the pupil, We have developed a program that can be read and executed on a mobile terminal device.

Claims (1)

A video ON step to turn on the video shooting function of the mobile terminal,
A light ON step that turns on the light provided on the shooting side of the mobile terminal,
A pupil recognition step of recognizing a pupil of an eye of an animal including a person from an image being photographed,
A pupil change calculation step for calculating a pupil change which is a time change of the degree of opening of the recognized pupil,
A program that is recorded on a mobile terminal device so that it can be read and executed.

Images