WO2016189711A1 - 携帯端末用ストレス評価プログラム及び該プログラムを備えた携帯端末 - Google Patents
携帯端末用ストレス評価プログラム及び該プログラムを備えた携帯端末 Download PDFInfo
- Publication number
- WO2016189711A1 WO2016189711A1 PCT/JP2015/065307 JP2015065307W WO2016189711A1 WO 2016189711 A1 WO2016189711 A1 WO 2016189711A1 JP 2015065307 W JP2015065307 W JP 2015065307W WO 2016189711 A1 WO2016189711 A1 WO 2016189711A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pupil
- unit
- mobile terminal
- change
- video
- Prior art date
Links
- XDTMQSROBMDMFD-UHFFFAOYSA-N C1CCCCC1 Chemical compound C1CCCCC1 XDTMQSROBMDMFD-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
- G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
- G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
- G06T7/0014—Biomedical image inspection using an image reference approach
- G06T7/0016—Biomedical image inspection using an image reference approach involving temporal comparison
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/11—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for measuring interpupillary distance or diameter of pupils
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10141—Special mode during image acquisition
- G06T2207/10152—Varying illumination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
Definitions
- the present invention relates to a program for evaluating stress using a mobile terminal.
- Stress is a cause of various diseases. However, in general, stress is not visually recognizable, and it is not easy to measure using stress.
- Patent Document 1 there is a technique as disclosed in Patent Document 1.
- This patent document captures and analyzes the pupil response when light is applied to the eyeball, taking advantage of the fact that the degree of stress and relaxation has a correlation with the degree of pupillary light response.
- the idea is to measure the degree of stress. With this idea, the purpose of measuring the state of stress is achieved by the objective symptoms correlated with stress.
- an object of the present invention is to provide a technique for measuring the state of stress by objective symptoms having a correlation with stress in a mobile terminal device.
- the present invention provides a video ON step for turning on a video shooting function of a mobile terminal, a light ON step for turning on a light provided on the shooting side of the mobile terminal, and an image being shot
- a pupil recognition step for recognizing the pupils of the eyes of animals including humans, and a pupil change calculation step for calculating a pupil change, which is a temporal change in the degree of opening of the recognized pupil, can be read and executed by the mobile terminal device
- a recorded program was developed.
- the present invention has developed a program that further includes an illuminance changing step for changing the illuminance of the light turned ON in the light ON step with time.
- the present invention further develops a program further comprising a body composition data acquisition step for acquiring body composition data received from a body composition meter.
- the present invention further develops a program having a stress evaluation step for evaluating the degree of stress based on the pupil change obtained by the pupil change calculation step and obtaining a stress evaluation result. did.
- the present invention has developed a program further comprising a pupil change transmission step for transmitting the pupil change obtained by the pupil change calculation step to a predetermined address. Then, a pupil change receiving step for receiving a pupil change transmitted as a result of executing the program, a pupil change accumulating step for accumulating the received pupil change, and a pupil change statistical processing step for statistically processing the accumulated pupil change And developed a program that can be read and executed on a computer.
- the present invention has developed a program further comprising a stress evaluation result transmission step for transmitting the stress evaluation result obtained in the stress evaluation step to a predetermined address.
- a program comprising a pupil change receiving step for receiving a pupil change transmitted as a result of executing the program having the pupil change sending step, a pupil change accumulating step for accumulating the received pupil change, and a stress evaluation result sending step.
- a stress evaluation result receiving step for receiving the stress evaluation result transmitted as a result of executing the stress, a stress evaluation result accumulating step for accumulating the received stress evaluation result, and a stress for statistically processing the accumulated stress evaluation result and pupil change Developed a program that describes statistical processing steps and that can be read and executed on a computer.
- the present invention further provides a program in which a continuous ON step for turning on the continuous still image shooting function of the mobile terminal is used instead of the video ON step for turning on the video shooting function of the mobile terminal. developed.
- the present invention provides a video ON step for turning on the video shooting function of the mobile terminal, a light ON step for turning on the light provided on the shooting side of the mobile terminal, and a person from the image being shot.
- a pupil recognition step for recognizing the pupil of the animal's eye, a moving image holding step for holding the moving image shot in the moving image ON step, and a moving image transmitting step for transmitting the held moving image to a predetermined address;
- a continuous still image holding step for holding a continuous still image taken by a mobile terminal is used, and instead of a video transmission step for transmitting the held video to a predetermined address, continuous A program has been developed in which a continuous still image transmission step for transmitting a continuous still image held in the still image holding step is recorded in a portable terminal device so as to be readable and executable.
- the present invention also recognizes a moving image receiving step for receiving a moving image including an eye of an animal including a person from a mobile terminal, and a pupil recognition step for recognizing a pupil of an eye of the animal including a human from the received moving image.
- a moving image receiving step for receiving a moving image including an eye of an animal including a person from a mobile terminal
- a pupil recognition step for recognizing a pupil of an eye of the animal including a human from the received moving image.
- a continuous still image receiving step for receiving a continuous still image including the eyes of animals including humans from the mobile terminal, a pupil recognition step for recognizing the pupils of the eyes of animals including humans from the received continuous still images,
- a program that records the pupil change calculation step for calculating the pupil change, which is the time change of the recognized pupil opening, in a computer so that it can be read and executed.
- the present invention has developed a portable terminal device or a statistical processing device in which the above program is recorded so as to be readable and executable.
- the flowchart which shows an example of the flow of a process of Example 4. The figure which shows an example of the functional block of the portable terminal device of Example 5.
- the figure which shows an example of the functional block of the portable terminal device of Example 9. The figure which shows an example of the hardware constitutions of the portable terminal device of Example 9.
- FIG. 1 showing an example of a portable terminal device
- FIG. 2 showing an example of a portable terminal device
- FIG. 3 showing an example of a portable terminal device
- Figure 1 shows the contents of statistical processing
- Figure 2 shows the contents of statistical processing
- Figure 3 shows the contents of statistical processing Diagram for explaining the contents of pupil recognition Diagram for explaining the contents of pupil change calculation
- the figure for demonstrating the aspect of a pupil calculation Figure showing only the outline of the human eye Diagram of a mobile terminal device when a part of the screen is a white light image with strong illuminance Graph where the vertical axis is the stress evaluation value and the horizontal axis is the pupil change rate
- the figure for explaining the mechanism for tracing and continuing identification even if slight movement occurs in the pupil The figure which shows the replay image after completion
- the invention of the present embodiment is a program in which a function for calculating a pupil change of a photographed image in units of time is recorded in a portable terminal device so as to be readable and executable by capturing a moving image of the pupil when light is applied. . Further, the portable terminal device is recorded so that the program can be read and executed.
- a function for calculating a pupil change of a photographed image in units of time is recorded in a portable terminal device so as to be readable and executable by capturing a moving image of the pupil when light is applied.
- the portable terminal device is recorded so that the program can be read and executed.
- FIG. 1 is an example of functional blocks of the mobile terminal device of this embodiment.
- the mobile terminal device (0101) of the present embodiment includes a camera unit (0102), a moving image ON unit (0103), a light unit (0104), a light ON unit (0105), and a pupil recognition unit (0106). ) And a pupil change calculation unit (0107).
- a computer a CPU, a main memory, a bus, or a secondary storage device (a storage medium such as a hard disk, a nonvolatile memory, a CD-ROM or a DVD-ROM, and a read drive for the medium) Etc.), hardware components such as printing devices, display devices, other external peripheral devices, I / O ports for the external peripheral devices, driver programs for controlling the hardware, other application programs, and information input Examples include user interfaces that are used.
- these hardware and software process the program developed on the main memory with the CPU, process and store data stored on the memory and hard disk, data input via the interface, etc. This is used for output processing or for controlling each hardware component.
- the present invention can be realized not only as an apparatus but also as a method. A part of the invention can be configured as software. Furthermore, a software product used for causing a computer to execute such software and a recording medium in which the product is fixed to a recording medium are naturally included in the technical scope of the present invention (the same applies throughout the present specification). Is).
- the “camera unit” has a function for capturing moving images and / or still images.
- the “light unit” has a light function provided on the photographing side of the mobile terminal. Although the “camera part” and the “light” part may exist on the front surface and the back surface, the object of the present invention can be achieved in any case.
- FIG. 31 to FIG. 33 are schematic diagrams of so-called smartphones among mobile terminal devices.
- (A) is the front surface and
- (b) is the back surface, and actually there is a microphone (not shown).
- a camera (4604), a light (4605), and a screen (4606) are present on the front surface of the portable terminal device (4601) in FIG. There is no camera or light.
- buttons (4802) and the receiver (4803) a camera (4804), a light (4805), and a screen (4806) are present on the front surface of the portable terminal device (4801) in FIG.
- the camera (4804) also has a light (4805).
- the size of the pupil can be easily recognized by using infrared rays. Therefore, it is desirable that the camera has an infrared camera function.
- a part of the screen can be replaced with a light by using a white light image with high illuminance.
- a white light image with strong illuminance it may be the upper half of the screen, the lower half, or even the upper and lower quarters.
- the ratio of the portion where the captured image is displayed and the portion of the white light image with strong illuminance is not limited to 1: 1, and may be a ratio sufficient to measure the pupillary light reaction.
- FIG. 46 is a diagram of the mobile terminal device when a part of the screen is a white light image with strong illuminance. Since the mobile terminal device (6101) has a camera (6102) but no light, the lower half of the screen (6103) is a white light image (6104) with high illuminance. Thereby, it can substitute for a light.
- the “video ON part” works to turn on the video shooting function of the mobile terminal.
- the “light ON unit” functions to turn on the light unit including the light provided on the photographing side of the mobile terminal.
- the most basic configuration of the present invention it is necessary to shoot a moving image of the pupil when light is applied.
- the user can take a moving image of the pupil portion by the action of the moving image ON unit. Further, the user can shine light on the pupil by the function of the light ON unit.
- the “shooting” in the moving image ON part includes the case where recording is performed at the same time, but recording is not essential.
- the present embodiment also includes a mode in which the moving image ON unit simply captures a moving image and the moving image being captured is calculated in real time by a “pupil recognition unit” and a “pupil change calculation unit” described later. It is.
- the moving image being shot includes not only the moving image in the middle of lighting, but also the moving image before and after the lighting.
- the moving image being photographed here is premised on including a pupil, but it is preferable that not only the pupil but also a body part around the pupil other than the pupil is included.
- the health condition of the person appears in various forms. For example, in allergic conjunctivitis, infectious conjunctivitis, uveitis or dry eye, it may cause redness of the eyes. Therefore, there is an effect that the above-mentioned symptom can be known from the fact that the eye is congested, for example.
- mobile terminals include, for example, smartphones, mobile phones, laptop computers, tablet PCs, watch computers, wearable computers, digital cameras, PDAs, PocketPCs, Palms, dedicated terminals with lights and cameras, etc. Is included.
- the device has a camera function and a light function, but in the present invention, even if the unit sold is a device that does not have a camera function or a light function, an external camera is mounted. In such a case, when an external light is mounted, the device and the external camera or external light are regarded as a “portable terminal” as a whole.
- the intensity of the light irradiated by the light ON part may be an intensity sufficient to cause the pupil to react to light.
- the fundus examination is performed by applying light to the pupil, for example, it is possible to set the light intensity suitable for performing the fundus examination. By doing so, the fundus examination can be performed simultaneously with the measurement of the stress intensity.
- the mobile terminal device may include a control unit for controlling the moving image shooting function and the ON / OFF of the light.
- the control unit automatically controls the moving image shooting function and the light ON / OFF at an appropriate timing only by the user turning on the switch.
- control unit may automatically turn on the moving image shooting function and the light at a predetermined time set in advance. By doing so, it is possible to measure the pupillary light response at a specific time, and for example, it is possible to automatically measure it during work.
- a moving image can be taken of the pupil when light is applied.
- the “pupil recognition unit” functions to recognize the pupils of the eyes of animals including humans from the image being photographed.
- the point of the problem of the present invention is that, for example, in the case of shooting with a mobile terminal device such as a smartphone, it is impossible to grasp the change in the size of the pupil.
- the face recognition function is permeated, but the function for recognizing the pupil portion of the eyeball is not general.
- the pupil recognition unit makes it possible to recognize the size of the pupil and is an important function in solving the problems of the present invention.
- the color or lightness of the pupil is registered based on the moving image or still image information of the pupil, and the same color or lightness as the registered color or lightness. It is conceivable that the pupil is recognized by specifying the portion having.
- the portable terminal device may include means for automatically recognizing the pupil portion even if the color or brightness of the pupil is not registered.
- a color brightness recognition means for recognizing the dark color part (A) light color part (B) white part (C) and skin part (D) of the eyeball by color or brightness.
- iris recognition means for finding the boundary E between the light color part (B) and the white part (C) of the eyeball, and finally the boundary that is the boundary between the dark color part (A) and the light color part (B) of the eyeball
- a method of having a pupil finding means for finding F may also be used.
- the mobile terminal device (6801) is turned sideways and the screen (6804) is close to the camera (6802) or the light (6803). It is conceivable to display an image (6805) such as a moving image on the screen so that the pupil portion is fixed by viewing one point. In this case, if white light is displayed on the screen (6804), it can be replaced with light of light. It is desirable to position the image such as a moving image so that the moving image center line is on the center line of the camera (6802). Further, a point of interest (6806) to be noted in the image may be clearly indicated.
- the circumference may be a darker color.
- the inner side of the darker portion may be the boundary F
- the outer side may be the boundary F
- the middle point may be the boundary F.
- FIG. 45 is a diagram showing only the outline of the human eye.
- the human eye (6001) there are an iris boundary part (6002) and a pupil boundary part (6003), and there is a part (6004) that is darker to the part indicated by the dotted line inside the pupil.
- the darker portion (6004) may be the boundary F
- the pupil boundary portion (6003) may be the boundary F
- the middle point may be the boundary F.
- FIG. 34 is a diagram showing a screen when the pupil is recognized.
- the portable terminal device (4901) As shown in the left figure and the pupil part is successfully recognized, it indicates that the pupil has been recognized as shown in the right figure. In this way, the pupil part (4902) may be raised. Thereby, it is possible to inform the user that the pupil recognition has been successful.
- the means for notifying that pupil recognition has been successful may be, for example, a method of notifying through a notification sound of “pea” or a change in illuminance, but is not limited thereto.
- the camera In order to automatically recognize the pupil, it is desirable that the camera has an autofocus function for automatically focusing. In addition, since there is a possibility that camera shake may occur when the user operates, it is desirable to have a camera shake prevention function. Thereby, accurate pupil recognition can be performed.
- a notification sound or a function of notifying by light ON / OFF, color or illuminance may be provided.
- a blink when recognized, it may have an extension function for extending the shooting time (measurement time) instead of starting moving image shooting from the beginning.
- the “pupil change calculator” functions to calculate a pupil change, which is a time change of the recognized degree of pupil opening. As described above, the degree of stress and relaxation has a correlation with the degree of pupillary light reaction. Therefore, the pupil change calculation unit calculates the pupillary light reaction.
- the pupillary light reaction is, specifically, a time change of the size of the pupil due to light being applied to the pupil.
- the pupil data to be calculated may be the diameter of the pupil or the area of the pupil.
- the pupil diameter can be measured more accurately even when the entire image of the pupil is blocked by eyelashes or when the eyelid is not fully open. desirable.
- the area of the pupil is desirable in that the size of the pupil can be accurately captured without the above circumstances.
- the user can obtain information on his / her pupillary photoreaction based on the value obtained by this calculation. Then, the user can objectively know the degree of stress and relaxation of the user by performing an analysis based on the correlation between the degree of stress and the degree of relaxation.
- a “light-off unit” that turns off the light provided on the photographing side of the mobile terminal may be provided. According to this, it is possible to know a more accurate degree of stress.
- the diameter measuring means for measuring the diameter of the iris part and the pupil part on the moving image can be realized by having a division means for performing division on the diameter.
- a registration information receiving unit that receives registration information such as name, age, date of birth, and eating tendency may be included. And it is good to accept user's iris information in association with this registration information. Thereby, by performing iris recognition with the camera of the mobile terminal device, reliable user management can be performed without depending on the user ID or the like.
- an external fan to the mobile terminal device, it is possible to activate the fan and simultaneously perform an intraocular pressure test.
- the intraocular pressure test when the eye blinks, the intraocular pressure tends to be measured high, and the notification function when the eye blinks is useful for the fundus examination.
- Fundus examination reveals illnesses such as glaucoma, retinal detachment, choroid detachment, and iridocyclitis.
- the mobile terminal device in order to increase the value of pupil change information, name, gender, date of birth, birthplace, measurement time, measurement location (eg, residence, university, workplace, tavern, train, park, etc.)
- measurement location eg, residence, university, workplace, tavern, train, park, etc.
- the date of birth is necessary to realize a stress evaluation that takes into account the biorhythm that can be a noise in the stress evaluation.
- the GPS function may be used to recognize the longitude / latitude of the measurement location and to calculate the altitude from the information. Since stress can be affected even by the concentration of oxygen, it has meaning as additional information.
- a pupil change list output unit that outputs pupil changes in a list.
- a list display method a graph, a table, a calendar, etc. can be considered.
- FIG. 35 is a diagram illustrating the contents of statistical information processed in the pupil change list output unit.
- pupil changes are listed for each date.
- information such as measurement conditions may be written in the date column. For example, if it is shown together with information such as brightness of the measured location, measurement time, before / after meals, sleep time of the previous day, presence / absence of work on the measurement day and intensity, the user can Know the cause and help you think about life to reduce stress.
- the mobile terminal device includes a replay unit for replaying the captured moving image after completion of the calculation.
- a replay unit for replaying the captured moving image after completion of the calculation.
- the portion recognized as the pupil is whitened or the portion used for the measurement is shown as the diameter as shown in FIG. It is desirable to have a configuration that does this.
- FIG. 52 is an example in which data relating to pupil contraction is displayed on the screen. The timing when the light brightens, the illuminance of the light, the transition of pupil changes, etc. are listed, and the user can see the calculation result at a glance.
- FIG. 2 is a diagram illustrating an example of a configuration of the mobile terminal device when the functional components described above are realized as hardware. The operation of each hardware component in each process in this apparatus will be described using this figure.
- the portable terminal device of this embodiment includes a “CPU (central processing unit)” (0201) and “main memory” (0202) for performing various arithmetic processes.
- “HDD” (0203)
- “Network I / F (Network Interface)” (0204) for transmitting / receiving information to / from “I / F (Interface)” (0207) of the external device (0206)
- An “I / O (input / output)” (0208) for transmitting / receiving information to / from a camera, a light, a display, or the like is also provided.
- These are connected to each other via a data communication path such as a “system bus” (0205), and perform transmission / reception and processing of information.
- the “external device” in this case usually refers to an administrator terminal or a statistical processing device, but includes a user terminal or a terminal of a medical institution, for example.
- the “main memory” provides a work area that is also a work area of the program at the same time as reading a program for performing various processes for the CPU to execute.
- a plurality of addresses are assigned to each of the “main memory”, “HDD”, and “flash memory” (not shown), and programs executed by the “CPU” specify each address to access each other. Data can be exchanged and processed.
- the programs stored in this embodiment are a video ON program for turning on the video shooting function of the mobile terminal, a light ON program for turning on the light provided on the shooting side of the mobile terminal, and an image being shot.
- a pupil recognition program that recognizes the pupils of the eyes of animals including humans, and a pupil change calculation program that calculates a pupil change that is a temporal change in the degree of opening of the recognized pupil.
- the user's pupil image acquired by the moving image ON program of the mobile terminal device is stored in the addresses of “main memory” and “HDD”. Then, in the “CPU” of the mobile terminal device, the time change of the pupil recognized by the logical operation process is calculated using the moving image information stored in the “main memory”.
- the video information is stored in the “HDD” as described above.
- the recorded video is recognized and calculated in real time, the video information is stored in the “HDD”. It will not be stored.
- FIG. 3 is a flowchart illustrating an example of a process flow in the mobile terminal device of the present embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded on a medium.
- the moving image shooting function of the portable terminal is turned on (S0301).
- the light provided on the photographing side of the portable terminal is turned on (S0302).
- the pupil of the eye of the animal including the person is recognized from the image being photographed (S0303).
- the pupil change which is the time change of the recognized pupil opening degree, is calculated (S0304).
- the moving image ON step (S0301) for turning on the moving image shooting function of the mobile terminal and the light ON step (S0302) for turning on the light provided on the shooting side of the mobile terminal are:
- the order may be reversed. That is, first, the light provided on the photographing side of the portable terminal is turned on. Next, the moving image shooting function of the mobile terminal is turned on. Then, the pupil of the eye of the animal including the person is recognized from the image being photographed. Finally, a process flow of calculating a pupil change, which is a time change of the recognized pupil opening degree, may be used.
- the light ON step (S0302) for turning on the light provided on the photographing side of the mobile terminal and the pupils of the eyes of animals including humans are recognized from the image being photographed.
- the pupil recognition step (S0303) may be performed in the reverse order. That is, first, the moving image shooting function of the mobile terminal is turned on. Next, the pupil of the eye of an animal including a person is recognized from the image being photographed. Then, the light provided on the photographing side of the mobile terminal is turned on. Finally, a process flow of calculating a pupil change, which is a time change of the recognized pupil opening degree, may be used.
- the light provided on the shooting side of the mobile terminal is turned off in order to obtain the time required until the pupil with reduced pupils is normal (original state), mydriatic ratio, mydriatic speed, and mydriatic acceleration. There may be a “light OFF step”. According to this, it is possible to know a more accurate degree of stress.
- FIG. 53 is a diagram showing the flow of processing in the case of having a light OFF step.
- a moving image ON step S7101
- a light ON step S7102
- recognition is started as a pupil recognition step (S7103), whether or not pupil recognition is successful is confirmed (S7104), and the recognition step is repeated until pupil recognition is successful.
- This pupil recognition continues until the entire process is completed.
- a pupil change calculation step S7105
- This pupil change calculation continues until the entire process is completed.
- the confirmation of whether or not to turn off the light may be that the light is turned off when a predetermined time elapses, when contraction is stopped, or when the speed becomes a certain speed or less. Considering that there is an individual difference in the biological reaction such as the miosis speed, it is desirable that the timing of turning off the light is not uniform, but is when the contraction stops or when the speed becomes a certain speed or less.
- the entire process is completed. If NO, the process is continued, and if YES, the entire process is terminated. Confirm whether or not the entire process is completed when the predetermined time has elapsed, when the scatter is stopped, when the speed is below a certain level, or when the pupil size is returned to the recognition level. It can be considered.
- FIG. 40 is a diagram showing an example when the above steps are set as a time chart.
- SW switch
- moving image shooting and pupil recognition start.
- the pupil calculation starts at the same time or slightly after the recognition is completed and the pupil recognition function is turned ON (high).
- the light is turned off when a predetermined time elapses, when contraction is stopped, or when the speed is below a certain level (range indicated by ⁇ in the drawing).
- a predetermined time elapses when scattering is stopped, when the speed is below a certain level (range indicated by ⁇ in the drawing), or when the size of the pupil at the time of recognition is restored.
- the entire process ends.
- FIG. 44 is a diagram for explaining a mode of pupil calculation.
- model data there are data of A and B, and the miosis speed (a 1 ) when the light of the line A is applied, the miosis ratio (h a ) when the most miosis is achieved, and the light is turned off.
- mydriasis rate after (a 2) will measure and time (t a) from OFF the light until mydriasis stops.
- the miosis speed (b 1 ) when the light is applied the miosis ratio (h b ) when the most miosis occurs, the mydriatic speed (b 2 ) after turning off the light, The time (t b ) until the mydriatic stops after turning off the light is measured.
- miosis begins after the light is turned on
- mydriasis begins after the light is turned off. If these two model data are analyzed on the premise that the higher the speed of both miosis and mydriasis, the lower the stress, the model data B is less stressed than the model data A. I understand.
- the state of stress can be measured by objective symptoms having a correlation with stress using the mobile terminal device.
- this example provides a mobile terminal device having a function of changing the illuminance of a light with the passage of time in addition to the configuration of Example 1.
- FIG. 4 is a diagram illustrating an example of functional blocks of the mobile terminal device according to the present embodiment.
- the mobile terminal device (0401) of the present embodiment includes a camera unit (0402), a moving image ON unit (0403), a light unit (0404), a light ON unit (0405), and a pupil recognition unit (0406). ), A pupil change calculation unit (0407), and an illuminance change unit (0408).
- the feature of the present embodiment is the content of the illuminance changing section. Therefore, it demonstrates centering on the functional structure of an illumination intensity change part. Other functions are the same as those described above, and thus description thereof is omitted.
- the “illuminance changing unit” (0406) functions to change the illuminance of the light turned on in the light ON unit with time. For example, in order to recognize the pupil, it is necessary to apply light to the pupil. On the other hand, if the illuminance of light is too strong, a pupil-light reaction occurs. If it does so, a change will be started before a pupil recognition part recognizes a pupil, and the precision of a calculation will be impaired. Therefore, with the function of the illuminance change unit, for example, light with weak illuminance is applied until pupil recognition, and from that point when the pupil is recognized, light with an illuminance that causes a pupil-to-light reaction is applied. High calculation results can be obtained.
- the pupil change be performed in a dark room.
- the operation of the illuminance changing unit enables an operation using a minimum amount of light, and the use in the dark room using a portable terminal becomes possible.
- FIG. 5 is a diagram illustrating an example of a configuration of the mobile terminal device when the functional components described above are realized as hardware.
- the mobile terminal device includes “CPU (central processing unit)” (0501), “main memory” (0502), “HDD” (0503), “network I / F” (0504), “I / O” (0508). , “System bus” (0505), and the external device (0506) has “I / F” (0507).
- an illuminance change program is added as the contents of the program stored in the “main memory”.
- FIG. 6 is a flowchart illustrating an example of a process flow in the mobile terminal device of the present embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded on a medium.
- a pupil change calculation step (S0605). This pupil change calculation continues until the entire process is completed.
- the contents of the illuminance change step there is confirmation whether the light is set to “high” or “low” (S0606), and if it is “high”, a process of setting the light to high (S0607) is performed. If “low”, the process of setting the write to low is performed (S0608).
- the confirmation of whether the light is set to “high” or “low” here is, for example, that the light is set to high for the first time, and then when a predetermined time elapses or when contraction is stopped or below a certain level. It can be considered that the light is set to low when the speed becomes. Considering that there is an individual difference in the biological reaction such as the miosis speed, it is desirable that the timing of turning the light low is not uniform, but when the contraction stops or when the speed becomes a certain speed or less.
- the processing in this embodiment has an illuminance change step.
- the “illuminance change step” is a stage in which the illuminance of the light turned on in the light ON section is changed over time.
- FIG. 43 is a diagram showing an example when the above steps are used as a time chart.
- SW switch
- moving image shooting, light, and pupil recognition start.
- the pupil calculation starts, and the illuminance of the light increases.
- the light goes low when a predetermined time elapses, when contraction is stopped, or when the speed is below a certain level (range indicated by ⁇ in the drawing).
- a predetermined time elapses when the scattering is stopped, or when the speed is below a certain level (range indicated by ⁇ in the drawing)
- the entire process ends.
- FIG. 54 is a diagram showing a time chart for enabling a configuration in which processing is ended when the size of the pupil at the time of recognition is returned.
- SW switch
- moving image shooting, light, and pupil recognition start.
- the pupil calculation starts at the same time or slightly after the recognition is completed and the pupil recognition state becomes High, and after calculating the pupil in the original state, the illuminance of the light increases.
- the light goes low when a predetermined time elapses, when contraction is stopped, or when the speed is below a certain level (range indicated by ⁇ in the drawing).
- such a processing flow can obtain a calculation result with high accuracy and can be used in a dark room using a portable terminal.
- This example provides a portable terminal device having a function of evaluating the degree of stress in addition to the configuration of any one of Examples 1 and 2, as a more desirable embodiment.
- FIG. 7 is a diagram illustrating an example of functional blocks of the mobile terminal device according to the present embodiment.
- the portable terminal device (1001) of the present embodiment includes a camera unit (1002), a moving image ON unit (1003), a light unit (1004), a light ON unit (1005), and a pupil recognition unit (1006). ), A pupil change calculation unit (1007), and a stress evaluation unit (1008).
- the feature of this embodiment is the content of the stress evaluation unit. Therefore, it demonstrates centering on the functional structure of a stress evaluation part. Other functions are the same as those described above, and thus description thereof is omitted.
- the “stress evaluation unit” functions to obtain a stress evaluation result by evaluating the degree of stress based on the pupil change calculated by the pupil change calculation unit. For example, based on data such as the diameter value, area value, time required until the pupil is minimized, miosis rate, miosis speed, miosis acceleration, etc., calculated by the pupil change calculation unit, Calculate the degree of stress.
- the degree of stress may be a numerical value such as%, or may be a step method such as large / medium / small. In addition, a comparison with the previous day or a maximum / minimum value may be used. Furthermore, when a correlation with sleep time or work intensity can be seen, the cause of stress may be derived based on the correlation.
- FIG. 49 is a diagram showing a stress evaluation method.
- the statistical data of 2000 people have a curve as shown in the drawing, it is conceivable to show a stress evaluation by measuring the distance from the median t with respect to the time until the dilated pupil after miosis.
- “normal” is set for the center side of “+ 1 ⁇ ”
- “weak stress” is set for the zone on the opposite side of the median value t among “+ 1 ⁇ ”
- the median value of “+ 2 ⁇ ” is set.
- t it may be determined that “moderate stress is present”, and in the zone on the opposite side of the median value t among “+ 2 ⁇ ”, “strong stress is present”.
- an “over” portion that is more distant from the median t than “+ 2 ⁇ ” it may be determined that “there is a very strong stress”.
- a function of notifying with an alarm sound or the like may be provided.
- information is shared among parents and children and between employers and workers. For example, when a child or employee has a certain level of stress, a notice to that effect is sent to the parent or user (company, etc.). If there is a system that can reach
- a stress evaluation list output unit that outputs a list of stress evaluation results.
- a list display method a graph, a table, a calendar, or the like can be considered.
- FIG. 37 is a diagram illustrating the contents of statistical information output in the stress evaluation result output unit.
- the stress assessment results for each date are listed in a calendar format.
- the user's stress level can be listed, and the characteristics of the stress level for each day of the week can be known.
- information such as pupil change information and measurement conditions may be written in the date column. For example, if it is shown together with information such as brightness of the measured location, measurement time, before / after meals, sleep time of the previous day, presence / absence of work on the measurement day and intensity, the user can Know the cause and help you think about life to reduce stress.
- Example 3 Furthermore, if it is the structure added to Example 3, it is desirable that there exists a list output part which outputs the acquired body composition data and stress evaluation result in a list. By displaying in a list in this way, it is possible to know at a glance what is an element influencing the stress in the body composition data as compared to the case of pupil change.
- FIG. 8 is a diagram illustrating an example of a configuration of the mobile terminal device when the functional components are realized as hardware.
- the portable terminal device includes “CPU (Central Processing Unit)” (1101), “Main Memory” (1102), “HDD” (1103), “Network I / F” (1104), “I / O” (1108). , “System bus” (1105), and external device (1106) has “I / F” (1107).
- a stress evaluation program is added as the contents of the program stored in the “main memory”.
- the degree of stress is evaluated based on the pupil change obtained by the pupil change calculation program.
- FIG. 9 is a flowchart illustrating an example of a process flow in the mobile terminal device of the present embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded on a medium.
- a moving image ON step (S1201), a light ON step (S1202), a pupil recognition step (S1203), a pupil change calculation step (S1204), and a stress evaluation step (S1205) are included.
- S1201 a moving image ON step
- S1202 a light ON step
- S1203 a pupil recognition step
- S1204 a pupil change calculation step
- S1205 a stress evaluation step
- the processing in this embodiment has a stress evaluation step.
- the “stress evaluation step” is a stage in which a stress evaluation result is obtained by evaluating the degree of stress based on the pupil change obtained by the pupil change calculation step.
- the degree of stress may be a numerical value such as%, or may be a step system such as large, medium, and small.
- a comparison with the previous day or a maximum / minimum value may be used.
- the cause of stress may be derived based on the correlation.
- an alarm step for notifying with an alarm sound or the like may be provided.
- information is shared between parents and children and between employers and workers. For example, when a child or employee has a certain level of stress, an alarm to that effect is sent to the parent or user (company, etc.). If there is a system that can reach
- a stress evaluation list output step for outputting the stress evaluation results as a list.
- a list display method a graph, a table, a calendar, or the like can be considered.
- the user can understand the degree of stress at a glance with a numerical value or a mark.
- the cause can be derived, it can be utilized in daily life.
- the provision of an alarm function or a notification function to others leads to prevention of traffic accidents and the like that may be caused by stress.
- This embodiment provides a portable terminal device having a function of transmitting a pupil change to a predetermined address in addition to the configuration of any one of the first to third embodiments as a more preferable embodiment.
- FIG. 10 is a diagram illustrating an example of functional blocks of the mobile terminal device according to the present embodiment.
- the mobile terminal device (1301) of the present embodiment includes a camera unit (1302), a moving image ON unit (1303), a light unit (1304), a light ON unit (1305), and a pupil recognition unit (1306). ), A pupil change calculation unit (1307), and a pupil change transmission unit (1308).
- the feature of the present embodiment is the content of the pupil change transmitting unit. Therefore, the functional configuration of the pupil change transmission unit will be mainly described. Other functions are the same as those described above, and thus description thereof is omitted.
- the “pupil change transmission unit” functions to transmit the pupil change obtained by the pupil change calculation unit to a predetermined address.
- pupil change information is transmitted to an HDD or main memory that is a storage area of a statistical processing device for statistical processing, which is an example of an external device.
- the mobile terminal device when stress evaluation is performed in an external device, it is desirable that a stress evaluation result is transmitted from the external device. Then, it is desirable that the mobile terminal device includes a stress evaluation result receiving unit.
- information on countermeasures is held in association with stress evaluation values on external devices. If the stress evaluation value is high, for example, proposals to eliminate factors causing stress, introductions to hospitals, stores, etc. You may receive the introduction information.
- FIG. 11 is a diagram illustrating an example of a configuration of the mobile terminal device when the functional components are realized as hardware.
- the mobile terminal device includes a “CPU (central processing unit)” (1401), a “main memory” (1402), an “HDD” (1403), a “network I / F” (1404), and an “I / O” (1408).
- “System bus” (1405), and external device (1406) has “I / F” (1407).
- a pupil change transmission program is added as the contents of the program stored in the “main memory”.
- the pupil change information obtained by the pupil change calculation program is stored in, for example, the “HDD” or “main memory” of the mobile terminal device, or the external device or the statistical processing device (not shown) To the main memory or HDD.
- FIG. 12 is a flowchart illustrating an example of a process flow in the mobile terminal device of the present embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded on a medium.
- a moving image ON step (S1501), a light ON step (S1502), a pupil recognition step (S1503), a pupil change calculation step (S1504), and a pupil change transmission step (S1505) are included.
- S1501 a moving image ON step
- S1502 a light ON step
- S1503 a pupil recognition step
- S1504 a pupil change calculation step
- S1505 a pupil change transmission step
- the processing in this embodiment includes a pupil change transmission step.
- the “pupil change transmission step” is a step of transmitting the pupil change obtained by the pupil change calculation step to a predetermined address.
- the transmission destination may be the mobile terminal device or the statistical processing device.
- the load of a portable terminal device can be reduced by performing a statistical process other than a portable terminal.
- the present embodiment provides a mobile terminal device having a function of transmitting a stress evaluation result to a predetermined address in addition to the configuration of the fourth embodiment added to the third or third embodiment. .
- FIG. 13 is a diagram illustrating an example of functional blocks of the mobile terminal device according to the present embodiment.
- the mobile terminal device (1901) of the present embodiment includes a camera unit (1902), a moving image ON unit (1903), a light unit (1904), a light ON unit (1905), and a pupil recognition unit (1906). ), A pupil change calculation unit (1907), a stress evaluation unit (1908), and a stress evaluation result transmission unit (1909).
- the feature of this embodiment is the content of the stress evaluation result transmission unit. Therefore, the functional configuration of the stress evaluation result transmission unit will be mainly described. Other functions are the same as those described above, and thus description thereof is omitted.
- the “stress evaluation result transmission unit” functions to transmit the stress evaluation result obtained by the stress evaluation unit to a predetermined address.
- the stress evaluation is transmitted to a predetermined address of the HDD or the main memory of the mobile terminal device.
- the stress evaluation information is transmitted to the HDD or main memory which is a storage area of the statistical processing device for statistical processing. Thereby, accumulation of stress evaluation information becomes possible.
- FIG. 14 is a diagram illustrating an example of a configuration of the mobile terminal device when the functional components described above are realized as hardware.
- the portable terminal device includes “CPU (Central Processing Unit)” (2001), “Main Memory” (1402), “HDD” (1403), “Network I / F” (1404), “I / O” (1408). , “System bus” (1405), and external device (1406) has “I / F” (1407).
- a stress evaluation result transmission program is added as the contents of the program stored in the “main memory”.
- the stress evaluation information generated by the stress evaluation program is transmitted to, for example, an external device or a main memory or HDD of a statistical processing device (not shown).
- FIG. 15 is a flowchart illustrating an example of a process flow in the mobile terminal device of the present embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded on a medium.
- the moving image ON step (S2101), the light ON step (S2102), the pupil recognition step (S2103), the pupil change calculation step (S2104), the stress evaluation step (S2105), and the stress evaluation result transmission step (S2106) are used.
- the processing in this embodiment has a stress evaluation result transmission step.
- the “stress evaluation result transmission step” is a step of transmitting the stress evaluation result obtained in the stress evaluation step to a predetermined address.
- a statistical processing device provided for statistical processing may be used.
- the load of a portable terminal device can be reduced by performing a statistical process other than a portable terminal.
- the invention of the present embodiment is a program in which a function for calculating a pupil change of a captured image in units of time is recorded in a portable terminal device so as to be readable and executable. It is. Further, the portable terminal device is recorded so that the program can be read and executed. In addition, what added any one structure of Example 2 to 5 to the structure of a present Example is also included in this invention.
- still images have higher resolution than moving images. Therefore, by using a continuous still image instead of a moving image, the resolution increases and the recognition of the boundary portion of the pupil becomes accurate.
- FIG. 16 is an example of functional blocks of the mobile terminal device of the present embodiment.
- the portable terminal device (2501) of the present embodiment includes a camera unit (2502), a continuous ON unit (2503), a light unit (2504), a light ON unit (2505), and a pupil recognition unit (2506). ) And a pupil change calculation unit (2507).
- the characteristic function of this embodiment is in the continuous ON section. For this reason, in this embodiment, the function of the continuous ON unit will be mainly described.
- the “continuous ON section” works to turn on the continuous still image shooting function of the mobile terminal. Even in this case, in order to examine the pupillary light reaction, it is necessary to keep the light on the pupil even when continuous still images are not being shot. Therefore, the “light ON part” is the same as in the previous embodiment. The “light part” is continuously turned on.
- “Shooting” in the continuous ON section includes the case of saving still images taken at the same time, but saving is not essential.
- the continuous ON unit only captures a continuous still image, and the continuous still image being captured is calculated in real time by a “pupil recognition unit” and a “pupil change calculation unit” described later. Is included.
- the continuous still image being photographed here is premised on including a pupil, but it is desirable that not only the pupil but also a body part around the pupil other than the pupil is included.
- the health condition of the person appears in various forms. For example, in allergic conjunctivitis, infectious conjunctivitis, uveitis or dry eye, it may cause redness of the eyes. Therefore, there is an effect that the above-mentioned symptom can be known from the fact that the eye is congested, for example.
- the continuous still image shooting interval is preferably a short interval. If it is short, information close to a moving image can be obtained, and the time, the miosis ratio, the miosis speed and the miosis acceleration required until the accurate pupil is minimized can be obtained.
- the short interval means, for example, that one still image is shot every two seconds, or one still image per second.
- this imaging interval affects the calculation in the pupil change calculation unit, it is desirable that the imaging interval exists as information associated with the pupil change information. For example, if one still image is taken per second, the pupil change per second is revealed by the still image.
- FIG. 17 is a diagram illustrating an example of a configuration of the mobile terminal device when the functional components are realized as hardware. The operation of each hardware component in each process in this apparatus will be described using this figure.
- the mobile terminal device of this embodiment includes a “CPU” (2601) for performing various arithmetic processes and a “main memory” (2602). Also, “HDD” (2603), “Network I / F” (2604) for transmitting / receiving information to / from “I / F” (2607) of external device (2606), camera, light, display, etc. "I / O” (2608) for transmitting and receiving information between the devices. Then, they are connected to each other via a data communication path such as a “system bus” (2605) to perform transmission / reception and processing of information.
- a “system bus” (2605) to perform transmission / reception and processing of information.
- the programs stored in this embodiment are a continuous ON program that turns on the video shooting function of the mobile terminal, a light ON program that turns on the light provided on the shooting side of the mobile terminal, and an image being shot
- a pupil recognition program that recognizes the pupils of the eyes of animals including humans, and a pupil change calculation program that calculates a pupil change that is a temporal change in the degree of opening of the recognized pupil.
- a characteristic program in the present embodiment is a continuous ON program.
- the other programs are the same as described above.
- the user's pupil image acquired by the continuous ON program of the portable terminal device is stored in the addresses of “main memory” and “HDD”. Then, in the “CPU” of the mobile terminal device, the temporal change of the pupil recognized by the logical operation process is calculated using the continuous still image information stored in the “main memory”. In addition, when storing captured still images, continuous still image information is stored in the “HDD” as described above. On the other hand, when the captured still images are recognized and calculated in real time, they are continuously stored. Still image information is not stored in the “HDD”.
- FIG. 18 is a flowchart illustrating an example of a process flow in the mobile terminal device of the present embodiment. Note that the steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded on a medium.
- the continuous still image shooting function of the portable terminal is turned on (S2701).
- the light provided on the photographing side of the portable terminal is turned on (S2702).
- the pupil of the eye of the animal including the person is recognized from the image being photographed (S2703).
- the pupil change which is the time change of the recognized pupil opening degree, is calculated (S2704).
- the continuous ON step (S2701) for turning on the continuous still image shooting function of the portable terminal and the light ON step (S2702) for turning on the light provided on the shooting side of the portable terminal are in reverse order. Also good. That is, first, the light provided on the photographing side of the portable terminal is turned on. Next, the continuous still image shooting function of the mobile terminal is turned on. Then, the pupil of the eye of the animal including the person is recognized from the image being photographed. Finally, a process flow of calculating a pupil change, which is a time change of the recognized pupil opening degree, may be used.
- the state of stress can be measured by objective symptoms having a correlation with stress using the mobile terminal device.
- This example provides a statistical processing device connected to a portable terminal device having the configuration of Example 4 or Example 6 having a pupil change transmission unit as a more desirable embodiment.
- pupil change information is received from a portable terminal device, pupil change information is accumulated, and statistical processing is performed.
- FIG. 19 is a diagram illustrating an example of functional blocks of the mobile terminal device and the statistical processing device according to the present embodiment.
- the mobile terminal device (2801) of the present embodiment includes a camera unit (2801), a moving image ON unit (2803), a light unit (2804), a light ON unit (2805), and a pupil recognition unit (2806). ), A pupil change calculation unit (2807), and a pupil change transmission unit (2808).
- the statistical processing device (2809) in the present embodiment includes a pupil change receiving unit (2810), a pupil change accumulating unit (2811), and a pupil change statistical processing unit (2812).
- the pupil change statistical processing unit is connected to a user (2813), an administrator (2814), or a third party (2815) such as a medical institution.
- This user (2813) includes the portable terminal device that has transmitted pupil changes from the pupil change transmitter to the statistical processing device.
- the feature of this embodiment is the contents of each part of the statistical processing apparatus. Therefore, it demonstrates centering on the functional structure of each part of a statistical processing apparatus. Other functions are the same as those described above, and thus description thereof is omitted.
- the “pupil change receiving unit” functions to receive pupil changes transmitted from the mobile terminal device. Therefore, for example, in addition to the diameter value and the area value, the statistical processing apparatus receives information such as the time required until the pupil is minimized, the miosis ratio, the miosis speed, and the miosis acceleration.
- the name, gender, date of birth, birthplace, measurement time, measurement location for example, residence, university, workplace, tavern, in the train, park, etc.
- Additional information such as food ingredients, occupation, presence / absence and intensity of work on the date of measurement, body temperature, heart rate, blood pressure, skin electrical resistance, body movement, respiration, and fluctuations, and body composition data and treatment history It is desirable to receive also about.
- the additional information is preferably stored in association with pupil change information so that it can be easily used for later statistical processing.
- the “pupil change accumulation unit” functions to accumulate the received pupil change.
- a mode of accumulation for each user can be considered. Then, a change in the user's stress state can be grasped in the form of comparison of accumulated pupil change information.
- a method of dividing by date without dividing by the user for example, accumulation for analyzing the influence of weather on stress can be performed.
- the “pupil change statistical processing unit” functions to statistically process the accumulated pupil changes.
- a method of performing statistical processing for each user a method of performing statistical processing for each date, a method of performing statistical processing for each additional information, and the like can be considered.
- the merits of statistical processing are that a user can obtain an appropriate diagnosis and that an appropriate future prediction can be made.
- the statistical processing device may have a function for disclosing the result of statistical processing to a user or the like.
- a statistical processing device has a statistical information transmission unit for transmitting statistical information to a terminal such as a user, and is transmitted from the statistical processing device in a terminal such as a user.
- a configuration including a statistical information receiving unit for receiving the statistical information obtained can be considered.
- a terminal such as a user has a statistical information output unit for outputting statistical information, for example, the user can recognize the statistical information through a display or the like of the mobile terminal device.
- FIG. 41 is a diagram when the miosis speed is taken on the horizontal axis and the stress level is taken on the vertical axis.
- the statistical data becomes a curve (5601)
- the stress level is Y 2 based on the miosis speed.
- the statistical processing and based on the actual miosis rate was diagnosed with stress degree corresponding to the portion of Y 2, it is a condition that can be output to that effect . By doing so, the user can obtain an appropriate diagnosis.
- FIG. 42 is a diagram showing a stress change with time during a specific work when the miosis speed is taken on the horizontal axis and the stress level is taken on the vertical axis.
- Results derived curve of statistical processing (5701), for example, the portion from one hour after part of Z 1 to the Z 0, the portion of Z 2 to 1 hour later, the stress change for each time so on Represents.
- the current stress state of the specific work is the Z 2 part
- the corresponding part after 3 hours is the Z 5 part
- the stress intensity is the Y 3 part.
- (5702) can be predicted. As described above, it is possible to make an appropriate future prediction by statistically processing the stress change over time.
- the statistical processes can be performed for each weather, meal, work environment, hobby, etc. without being limited to changes over time. Then, by combining these comprehensively, it is possible to make a highly accurate stress diagnosis and to obtain information for specifying the cause of the stress.
- the cause of stress may be a cause of stress for each individual, and in the case of matters common to all users, it can be considered that the cause of stress is generally appropriate. Thus, in the case of a stress cause that is generally valid, the value of the information that tells the fact becomes socially high, which leads to providing information for a fee.
- an element that reduces stress may have a function of presenting the element or presenting a solution that eliminates the cause of stress.
- the significance and usage of these statistical processes are not limited to the present embodiment, but are applicable to the overall statistical processes in the present invention.
- FIG. 20 is a diagram illustrating an example of a configuration of the mobile terminal device and the statistical processing device when the functional components are realized as hardware.
- the portable terminal device includes “CPU (central processing unit)” (2901), “main memory” (2902), “HDD” (2903), “network I / F” (2904), “I / O” (2912). , “Statistical processing device” (2906), “I / F” (2907), “display” (2908), “main memory” (2909), “CPU” ( 2910) and “frame memory” (2911).
- the feature of this embodiment is the statistical processing device.
- the moving image information and pupil change information stored in the “main memory” of the mobile terminal device are transmitted to the statistical processing device via the “network I / F” by the “pupil change transmission program”, and the statistical processing device ,
- the information is received through the “I / F” by the “pupil change receiving program”, stored in “main memory” or “HDD” (not shown) on the statistical processing apparatus, and the “pupil change accumulation program”.
- pupil change information is stored in the “main memory” or “HDD” (not shown).
- the accumulated pupil change information is subjected to statistical processing using the “pupil change statistical processing program” by the operation of the “CPU” of the statistical processing device.
- the “main memory” of the statistical processing device may store a statistical information transmission program for transmitting the statistical processing result to the mobile terminal
- the “main memory” may store a statistical information receiving program for receiving the transmitted statistical processing result from the statistical processing device and a statistical information output program for outputting the statistical information.
- FIG. 21 is a flowchart illustrating an example of a processing flow in the mobile terminal device and the statistical processing device according to the present embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal and statistical processing terminal) recorded on a medium.
- the mobile terminal device has a moving image ON step (S3001), a light ON step (S3002), a pupil recognition step (S3003), a pupil change calculation step (S3004), and a pupil change transmission step (S3005).
- the statistical processing apparatus includes a pupil change receiving step (S3006), a pupil change accumulating step (S3007), and a pupil change statistical processing step (S3008).
- the “pupil change receiving step” in the statistical processing device is a step of receiving a pupil change transmitted as a result of executing the pupil change transmitting step in the mobile terminal device. Further, the “pupil change accumulation step” in the statistical processing device is a stage in which the received pupil change is accumulated. The “pupil change statistical processing step” in the statistical processing device is a stage for statistically processing the accumulated pupil change.
- the mode of statistical processing is not limited for each user.
- the mode of statistical processing is not limited for each user.
- the statistical processing device further includes a statistical information transmission step for transmitting statistical information to the mobile terminal, and in the mobile terminal device, the statistical processing device It is also conceivable that the processing flow includes a statistical information receiving step for receiving the transmitted statistical information and a statistical information output step for outputting the received statistical information.
- the present embodiment provides a statistical processing device connected to a portable terminal device in which the configuration of the fifth embodiment is added to the fourth embodiment as a more preferable embodiment.
- the outline of the configuration of the statistical processing device is to receive the stress evaluation result and pupil change from the portable terminal device, accumulate them, and perform statistical processing.
- FIG. 22 is a diagram illustrating an example of functional blocks of the mobile terminal device and the statistical processing device according to the present embodiment.
- the mobile terminal device (3401) of the present embodiment includes a camera unit (3402), a moving image ON unit (3403), a light unit (3404), a light ON unit (3405), and a pupil recognition unit (3406). ), Pupil change calculation unit (3407), stress evaluation unit (3408), stress evaluation result transmission unit (3409), and pupil change transmission unit (3414).
- the statistical processing device (3410) in this embodiment includes a stress evaluation result receiving unit (3411), a stress evaluation result accumulating unit (3412), a stress evaluation statistical processing unit (3413), a pupil change receiving unit (3415), and a pupil.
- a change accumulation unit (3416) is included.
- the stress evaluation statistical processing unit is connected to a user (3417), an administrator (3418), or a third party (3419) such as a medical institution.
- the feature of this embodiment is the contents of each part of the statistical processing apparatus. Therefore, it demonstrates centering on the functional structure of each part of a statistical processing apparatus. Other functions are the same as those described above, and thus description thereof is omitted.
- the “stress evaluation result receiving unit” functions to receive the stress evaluation result transmitted from the mobile terminal device. Therefore, for example, the statistical processing device receives a stress evaluation that indicates the degree of stress as a numerical value such as%, a stage such as large / medium / small, a comparison with the previous day, or the maximum / minimum values. Become.
- the “stress evaluation result accumulation unit” serves to accumulate the received stress evaluation results.
- an accumulation mode an accumulation mode for each user can be considered. Then, a change in the user stress evaluation result can be grasped in the form of a change in the accumulated stress evaluation result.
- the “stress evaluation statistical processing unit” functions to statistically process the accumulated stress evaluation results and pupil changes.
- the statistical processing as described above, a method of performing statistical processing for each user, a method of performing statistical processing for each date, and the like can be considered.
- the statistical processing apparatus may have a function for disclosing the result of statistical processing to a user or the like.
- a statistical processing device has a statistical information transmission unit for transmitting statistical information to a terminal such as a user, and is transmitted from the statistical processing device in a terminal such as a user.
- a configuration including a statistical information receiving unit for receiving the statistical information obtained can be considered.
- FIG. 47 is a diagram showing a mode of output on the user side. It is a graph with the vertical axis as the stress evaluation value and the horizontal axis as the pupil change rate. It can be seen that the higher the pupil change speed, the smaller the stress evaluation value.
- FIG. 48 is a diagram showing a mode of output on the user side. It is a graph with the vertical axis as the stress evaluation value and the horizontal axis as the body composition amount. As for body fat percentage, body weight, BMI, and pulse, it can be seen that the higher the body composition amount, the higher the stress evaluation value. On the other hand, regarding basal metabolism, it can be seen that the higher the body composition amount, the lower the stress evaluation value. If such statistical information is obtained, the user or the like can objectively analyze the cause of stress. Moreover, since the correlation between the stress evaluation value and the stress is clearer than the comparison with the pupil change, it is easy to understand at a glance.
- the data can be received after the user's portable terminal device lacking the stress evaluation process to be performed in the statistical processing device, and the load on the server is reduced.
- FIG. 23 is a diagram illustrating an example of a configuration of the mobile terminal device and the statistical processing device when the functional components are realized as hardware.
- the portable terminal device includes “CPU (central processing unit)” (3501), “main memory” (3502), “HDD” (3503), “network I / F” (3504), “I / O” (3512).
- a statistical processing device (3506) includes an “I / F” (3507), a “display” (3508), a “main memory” (3509), a “CPU” ( 3510) and “frame memory” (3511).
- the feature of this embodiment is the statistical processing device.
- the stress evaluation information stored in the “main memory” of the mobile terminal device is transmitted to the statistical processing device through the “network I / F” by the “stress evaluation result transmission program”.
- the information is received through the “I / F” by the “stress evaluation result receiving program” and stored in “main memory” or “HDD” (not shown) on the statistical processing apparatus.
- stress evaluation information is stored in the “main memory” or “HDD” (not shown).
- the accumulated stress evaluation information is subjected to statistical processing using a “stress evaluation statistical processing program” by the function of the “CPU” of the statistical processing device.
- the “main memory” of the statistical processing device may store a statistical information transmission program for transmitting the statistical processing result to the mobile terminal, or may be transmitted to the “main memory” of the mobile terminal device.
- a statistical information receiving program for receiving the statistical processing result from the statistical processing device and a statistical information output program for outputting the statistical information may be stored.
- FIG. 24 is a flowchart illustrating an example of a processing flow in the mobile terminal device and the statistical processing device of the present embodiment. The steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal, statistical processing terminal) recorded on a medium.
- the mobile terminal device includes a moving image ON step (S3601), a light ON step (S3602), a pupil recognition step (S3603), a pupil change calculation step (S3603), a stress evaluation step (S3604), a stress evaluation result transmission step (S3605), A pupil change transmission step (S3606) is included.
- the statistical processing device includes a pupil change receiving step (S3607), a pupil change accumulating step (S3608), a stress evaluation result receiving step (S3609), a stress evaluation result accumulating step (S3610), and a stress evaluation statistical processing step (S3611).
- the “stress evaluation result receiving step” in the statistical processing device is a step of receiving the stress evaluation result transmitted as a result of executing the stress evaluation result transmitting step in the mobile terminal device. This stress evaluation result is sent in association with the pupil change. Thereby, the burden of performing stress evaluation in the statistical processing apparatus can be reduced.
- the “stress evaluation result accumulation step in the statistical processing device” is a stage in which the received stress evaluation result is accumulated.
- the “stress evaluation statistical processing step” in the statistical processing device is a stage for statistically processing the accumulated stress evaluation results.
- the stress evaluation result receiving step is after the stress evaluation result transmitting step in the portable terminal device, the prior relationship in the statistical processing device does not matter. For example, it may be ahead of the pupil change receiving step.
- the “stress evaluation result accumulation unit” is after the body composition data reception step in the statistical processing apparatus, the relationship with other processes is not questioned.
- the “pupil change receiving step” and the “pupil change accumulating step” may be related to the stress evaluation result receiving step and the stress evaluation result accumulating step.
- the stress evaluation statistical processing step uses the accumulated pupil change and the stress evaluation result, it is performed after both the stress evaluation result accumulation step and the pupil change accumulation step are completed.
- the mode of statistical processing is not limited for each user.
- the statistical processing device further includes a statistical information transmission step for transmitting statistical information to the mobile terminal, and in the mobile terminal device, the statistical processing device It is also conceivable that the processing flow includes a statistical information receiving step for receiving the transmitted statistical information and a statistical information output step for outputting the received statistical information.
- the invention of the present embodiment is a portable terminal device having a function for photographing a change in the pupil when illuminated with light, holding movie information for calculating the pupil change in time units, and transmitting the information to an external device.
- FIG. 25 is an example of functional blocks of the mobile terminal device of this embodiment.
- the mobile terminal device (4001) of the present embodiment includes a camera unit (4002), a moving image ON unit (4003), a light unit (4004), a light ON unit (4005), and a pupil recognition unit (4006). ), A moving image holding unit (4007), and a moving image transmitting unit (4008).
- the feature of this embodiment is the contents of the moving image holding unit and the moving image transmission unit. Therefore, below, it demonstrates centering on the content of a moving image holding
- each of the above-described embodiments is the content that the pupil change is calculated in the portable terminal device. This is excellent in that the calculation can be completed in the portable terminal device, but in order to achieve the object of the present invention, the calculation of the pupil change is not necessarily completed in the portable terminal device. You don't have to. Therefore, on the assumption that the pupil change is calculated in the external device, the mobile terminal device in the present embodiment holds the moving image in which the pupil is recognized in the pupil recognition unit and transmits it to the external device. By providing this function, the object of the present invention is achieved. That is, it is premised that the external device has a function corresponding to the pupil change calculation unit in the first embodiment, and further, it is desirable if the external device has a function corresponding to the stress evaluation unit in the third embodiment.
- the “moving image holding unit” functions to hold a moving image shot by the moving image ON unit.
- the held moving image may be in a state where the pupil portion is distinguished from the non-pupil portion.
- the “moving image transmission unit” functions to transmit the stored moving image to a predetermined address. Thereby, the moving image information which image
- the moving image information is stored in the frame memory of the external device, and is output on a display of the external device for every predetermined frame. Moreover, it is desirable that the stress-evaluated information or the like is displayed as a list on a user terminal including a mobile terminal device, as in other embodiments.
- the processing in the mobile terminal device can be reduced, and the load on the mobile terminal device can be reduced.
- FIG. 26 is a diagram illustrating an example of a configuration of the mobile terminal device when the functional components are realized as hardware. The operation of each hardware component in each process in this apparatus will be described using this figure.
- the portable terminal device of this embodiment includes a “CPU” (4101) for performing various arithmetic processes and a “main memory” (4102). Also, “HDD” (4103), “Network I / F” (4104) for transmitting / receiving information to / from “I / F” (4107) of external device (4106), camera, light, display, etc. "I / O” (4108) for sending and receiving information between the two. Then, they are connected to each other via a data communication path such as a “system bus” (4105) to perform transmission / reception and processing of information.
- the external device includes a “frame memory” (4109), a “display” (4110), and a “CPU” (4111).
- the programs stored in the “main memory” are a video ON program that turns on the video shooting function of the mobile terminal and a light ON that turns on the light provided on the shooting side of the mobile terminal.
- the user's pupil image acquired by the moving image ON program of the mobile terminal device is stored in the addresses of “main memory” and “HDD”. Then, in the “CPU” of the mobile terminal device, the moving image information stored in the “main memory” is transmitted to the “I / F” of the external device through the “network I / F”.
- FIG. 27 is a flowchart illustrating an example of a process flow in the mobile terminal device according to the present embodiment. Note that the steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded on a medium.
- the moving image shooting function of the portable terminal is turned on (S4201).
- the light provided on the photographing side of the portable terminal is turned on (S4202).
- the pupil of the eye of the animal including a person is recognized from the image being photographed (S4203).
- photographed by the moving image ON step is hold
- the stored moving image is transmitted to a predetermined address (S4204).
- the order of the moving image ON step (S4201) for turning on the moving image shooting function of the portable terminal and the light ON step (S4202) for turning on the light provided on the shooting side of the portable terminal may be reversed. . That is, first, the light provided on the photographing side of the portable terminal is turned on. Next, the moving image shooting function of the mobile terminal is turned on. Next, the pupil of the eye of an animal including a person is recognized from the image being photographed. And the moving image image
- the state of stress can be measured by objective symptoms having a correlation with stress without performing pupil change calculation in the mobile terminal device.
- the invention of the present embodiment is a function for capturing continuous still image shooting of changes in the pupil when light is applied, holding continuous still image information for calculating pupil changes in units of time, and transmitting the information to an external device. Is recorded in a portable terminal device so that it can be read and executed. Further, the portable terminal device is recorded so that the program can be read and executed.
- FIG. 28 is an example of functional blocks of the mobile terminal device according to the present embodiment.
- the mobile terminal device (4301) of the present embodiment includes a camera unit (4302), a continuous ON unit (4303), a light unit (4304), a light ON unit (4305), and a pupil recognition unit (4306). ), A continuous still image holding unit (4307), and a continuous still image transmission unit (4308).
- the feature of this embodiment is the contents of the continuous still image holding unit and the continuous still image transmission unit. Therefore, below, it demonstrates centering on the content of a moving image holding
- the portable terminal device holds the continuous still image in which the pupil is recognized in the pupil recognition unit, and the external device
- the object of the present invention is achieved by providing a function of transmitting.
- the difference from the ninth embodiment is that since the image to be captured is a continuous still image, the image to be held and transmitted is a continuous still image. Since the point at which continuous still images are captured has already been described in the above-described embodiments, the continuous still image holding unit and the continuous still image transmission unit will be described below.
- the “continuous still image holding unit” functions to hold a continuous still image taken by the mobile terminal.
- the held moving image may be in a state where the pupil portion is distinguished from the non-pupil portion.
- the “continuous still image transmitting unit” functions to transmit the held continuous still image to a predetermined address. Thereby, the continuous still image information which image
- the stress state can be measured based on the objective symptom correlated with the stress without performing the pupil change calculation in the portable terminal device.
- FIG. 29 is a diagram illustrating an example of a configuration of the mobile terminal device when the functional components described above are realized as hardware. The operation of each hardware component in each process in this apparatus will be described using this figure.
- the mobile terminal device of this embodiment includes a “CPU” (4401) for performing various arithmetic processes and a “main memory” (4402).
- “HDD” (4403) “Network I / F” (4404) for transmitting / receiving information to / from “I / F” (4407) of external device (4406), camera, light, display, etc.
- “I / O” (4408) for sending and receiving information between the two.
- a data communication path such as a “system bus” (4405), and perform transmission / reception and processing of information.
- the “main memory” provides a work area that is also a work area of the program at the same time as reading a program for performing various processes for the CPU to execute.
- a plurality of addresses are assigned to each of the “main memory”, “HDD”, and “flash memory” (not shown), and programs executed by the “CPU” specify each address to access each other. Data can be exchanged and processed.
- the programs stored in this embodiment are a continuous ON program that turns on the continuous still image shooting function of the mobile terminal, a light ON program that turns on the light provided on the shooting side of the mobile terminal, and during shooting A pupil recognition program for recognizing the pupils of the eyes of animals including humans, and a continuous still image transmission program for transmitting the held continuous still image to a predetermined address.
- the user's pupil image acquired by the continuous ON program of the portable terminal device is stored in the addresses of “main memory” and “HDD”. Then, in the “CPU” of the mobile terminal device, the continuous still image information stored in the “main memory” is transmitted to the “I / F” of the external device through the “network I / F”. is there.
- FIG. 30 is a flowchart illustrating an example of a flow of processing in the mobile terminal device according to the present embodiment. Note that the steps shown below may be processing steps constituting a program for controlling a computer (mobile terminal) recorded on a medium.
- the continuous still image shooting function of the portable terminal is turned on (S4501).
- the light provided on the photographing side of the portable terminal is turned on (S4502).
- the pupil of the eye of the animal including the person is recognized from the image being photographed (S4503).
- photographed by the continuous ON step is hold
- the held continuous still image is transmitted to a predetermined address (S4504).
- the continuous ON step (S4501) for turning on the continuous still image shooting function of the portable terminal and the light ON step (S4502) for turning on the light provided on the shooting side of the portable terminal are in reverse order. Also good. That is, first, the light provided on the photographing side of the portable terminal is turned on. Next, the continuous still image shooting function of the mobile terminal is turned on. Next, the pupil of the eye of an animal including a person is recognized from the image being photographed. And the continuous still image image
- the state of stress can be measured by objective symptoms having a correlation with stress without performing pupil change calculation in the mobile terminal device.
- the present embodiment provides an arithmetic device having a function for calculating a pupillary light reaction based on a moving image received from a mobile terminal device.
- FIG. 55 is a diagram illustrating an example of functional blocks of the arithmetic device according to the present embodiment.
- the computing device (7301) of the present embodiment has a moving image receiving unit (7302), a pupil recognizing unit (7303), and a pupil change computing unit (7304).
- the mobile terminal normally has the camera unit, the light unit, the moving image ON unit, and the light ON unit in the first embodiment.
- the features of the camera unit, the light unit, the moving image ON unit, and the light ON unit of the portable terminal are the same as those in the first embodiment.
- the mobile terminal usually has a moving image transmission unit for transmitting a moving image to the arithmetic device in the present embodiment.
- the “moving image receiving unit” functions to receive moving images including the eyes of animals including humans from the mobile terminal (7305). Thereby, the moving image including the eyes of animals including people can be received from the user.
- the pupil recognizing unit has a function of recognizing the pupil of the eye of an animal including a person from the received moving image.
- the pupil change calculation unit has a function of calculating a pupil change, which is a time change of the recognized degree of opening of the pupil.
- the details of the functions of the pupil recognition unit and the pupil change calculation unit are the same as the details of the pupil recognition unit and the pupil change calculation unit of the mobile terminal device in the first embodiment.
- the arithmetic unit may include a notification transmission unit that transmits a notification to the user when the pupil cannot be recognized for the received moving image. If it does so, since a user can image
- the notification may be provided with reason adding means for adding the reason why the notification could not be recognized.
- the arithmetic device may include a pupil change transmission unit that transmits pupil changes to a predetermined address.
- the camera terminal, the light unit, the video ON unit, the light ON unit, and the video transmission unit need only be provided for the mobile terminal device.
- the calculation can be performed without it. Therefore, it is not always necessary for the user to prepare a dedicated terminal or a dedicated application, so that the present invention can be used easily.
- FIG. 56 is a diagram illustrating an example of a configuration of the arithmetic device when the functional component requirements are realized as hardware.
- the arithmetic units are “CPU” (7401), “main memory” (7402), “HDD” (7403), “network I / F” (7404), “I / O” (7408), “system bus” ( 7405) and is connected to the mobile terminal device (7406) through the “network I / F”.
- the contents of the programs stored in the “main memory” are a moving image reception program, a pupil recognition program, and a pupil change calculation program.
- a moving image receiving program is executed to receive a moving image including the eyes of an animal including a person transmitted from the mobile terminal device through the “network I / F”. , “HDD” and “main memory”. Then, the pupil recognition program is executed to recognize the pupils of the eyes of animals including humans from the received video, the pupil change calculation program is executed, and the recognized pupil opening time changes. For example, the pupil change is calculated. The calculation result is stored in the “HDD” and the “main memory” as pupil change information.
- FIG. 57 is a flowchart illustrating an example of a processing flow in the arithmetic device according to the present embodiment. Note that the steps shown below may be processing steps that constitute a program for controlling a computer (mobile terminal, arithmetic terminal) recorded on a medium.
- a moving image including the eyes of an animal including a person is received from a portable terminal (S7501).
- the pupil of the eye of the animal including a person is recognized from the received moving image (S7502).
- the pupil change which is the time change of the recognized pupil opening degree is calculated (S7503).
- Details of the functions of the pupil recognition step (S7502) and the pupil change calculation step (S7503) are the same as the details of the pupil recognition step and the pupil change calculation step of the mobile terminal device in the first embodiment.
- the arithmetic device may have a notification transmission step of transmitting a notification to the user when the pupil cannot be recognized for the received moving image.
- the notification may include a reason addition sub-step for adding a reason why the notification could not be recognized.
- the present embodiment provides an arithmetic device having a function for calculating a pupillary light reaction based on a continuous still image received from a mobile terminal device.
- FIG. 58 is a diagram illustrating an example of functional blocks of the arithmetic device according to the present embodiment.
- the calculation device (7601) of the present embodiment includes a continuous still image reception unit (7602), a pupil recognition unit (7603), and a pupil change calculation unit (7604).
- the mobile terminal normally has the camera unit, the light unit, the continuous ON unit, and the light ON unit in the sixth embodiment.
- the features of the camera unit, the light unit, the continuous ON unit, and the light ON unit of the portable terminal are the same as those in the sixth embodiment.
- the mobile terminal usually has a continuous still image transmission unit for transmitting continuous still images to the arithmetic device in the present embodiment.
- the “continuous still image receiving unit” functions to receive a continuous still image including the eyes of animals including people from the portable terminal (7605). Thereby, the continuous still image containing the eyes of animals including humans can be received from the user.
- the pupil recognition unit has a function of recognizing the pupils of the eyes of animals including humans from the received continuous still images.
- the pupil change calculation unit has a function of calculating a pupil change, which is a time change of the recognized degree of opening of the pupil.
- the details of the functions of the pupil recognition unit and the pupil change calculation unit are the same as the details of the pupil recognition unit and the pupil change calculation unit of the mobile terminal device in the first embodiment.
- the arithmetic device may include a notification transmission unit that transmits a notification to the user when the pupil cannot be recognized for the received continuous still images.
- the notification may be provided with reason adding means for adding the reason why the notification could not be recognized.
- the arithmetic device may include a pupil change transmission unit that transmits pupil changes to a predetermined address.
- FIG. 59 is a diagram illustrating an example of a configuration of the arithmetic device when the above functional components are realized as hardware.
- the arithmetic units are “CPU” (7701), “main memory” (7702), “HDD” (7703), “network I / F” (7704), “I / O” (7708), “system bus” ( 7705) and is connected to the portable terminal device (7706) through the “network I / F”.
- the contents of the programs stored in the “main memory” are a continuous still image receiving program, a pupil recognition program, and a pupil change calculation program.
- a continuous still image receiving program is executed to receive a continuous still image including the eyes of an animal including a person transmitted from the mobile terminal device through the “network I / F”.
- the received continuous still images are stored in “HDD” and “main memory”.
- the pupil recognition program is executed to recognize the pupils of the animal's eyes including humans from the received continuous still images, and the pupil change calculation program is executed to change the recognized pupil opening over time. That is, the pupil change is calculated.
- the calculation result is stored in the “HDD” and the “main memory” as pupil change information.
- FIG. 60 is a flowchart illustrating an example of a processing flow in the arithmetic device according to the present embodiment. Note that the steps shown below may be processing steps that constitute a program for controlling a computer (mobile terminal, arithmetic terminal) recorded on a medium.
- a continuous still image including the eyes of animals including people is received from the portable terminal (S7501).
- the pupil of the eye of an animal including a person is recognized from the received continuous still images (S7502).
- the pupil change which is the time change of the recognized pupil opening degree is calculated (S7503).
- Details of the functions of the pupil recognition step (S7502) and the pupil change calculation step (S7503) are the same as the details of the pupil recognition step and the pupil change calculation step of the mobile terminal device in the first embodiment.
- the arithmetic unit may include a notification transmission step of transmitting a notification to the user when the pupil cannot be recognized for the received continuous still images.
- the notification may include a reason addition sub-step for adding a reason why the notification could not be recognized.
- 4601, 4701, 4801, 4901 mobile terminal devices 4602, 4702, 4802: operation buttons 4603, 4703, 4803: receivers 4604, 4704, 4804: cameras 4605, 4705, 4805: lights 4606, 4706, 4806: screen 4902: Pupil part
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Theoretical Computer Science (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Primary Health Care (AREA)
- Epidemiology (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Pathology (AREA)
- Hospice & Palliative Care (AREA)
- Developmental Disabilities (AREA)
- Veterinary Medicine (AREA)
- Biophysics (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Child & Adolescent Psychology (AREA)
- Heart & Thoracic Surgery (AREA)
- Educational Technology (AREA)
- Molecular Biology (AREA)
- Psychiatry (AREA)
- Psychology (AREA)
- Social Psychology (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Business, Economics & Management (AREA)
- Quality & Reliability (AREA)
- Multimedia (AREA)
- General Business, Economics & Management (AREA)
- Human Computer Interaction (AREA)
- Ophthalmology & Optometry (AREA)
- Eye Examination Apparatus (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
Abstract
Description
本実施例の発明は、光を当てた際の瞳孔の変化を動画撮影して、撮影画像の瞳孔変化を時間単位で演算するための機能を携帯端末装置に読取実行可能に記録したプログラムである。また、そのプログラムが読取実行可能に記録された携帯端末装置である。これらは、携帯端末に限らず、デスクトップパソコンなどの端末によっても実現され得るが、以下では、一例として携帯端末を用いた場合について説明する。
図1は、本実施例の携帯端末装置の機能ブロックの一例である。この図にあるように、本実施例の携帯端末装置(0101)は、カメラ部(0102)と動画ON部(0103)とライト部(0104)とライトON部(0105)と瞳孔認識部(0106)と瞳孔変化演算部(0107)とを有する。
「カメラ部」は、動画又は/及び静止画の撮影のための機能を有する。また、「ライト部」は、携帯端末の撮影側面に備えられているライトの機能を有する。「カメラ部」及び「ライト」部は、表面と裏面に存在する場合があるが、いずれであっても本発明の目的は達成し得る。
図31から図33は、携帯端末装置のうちいわゆるスマートフォンの概略図である。(a)が表面、(b)が裏面であり、実際には図示しないマイクが存在する。
図31の携帯端末装置(4601)の表面には、操作ボタン(4602)と受話部(4603)の他に、カメラ(4604)及びライト(4605)並びに画面(4606)が存在するが裏面にはカメラもライトも存在しない。
図32の携帯端末装置(4701)の表面には、操作ボタン(4702)と受話部(4703)と画面(4706)が存在するもののカメラもライトも存在しないが、裏面にはカメラ(4704)及びライト(4705)が存在する。
図33の携帯端末装置(4801)の表面には、操作ボタン(4802)と受話部(4803)の他に、カメラ(4804)及びライト(4805)並びに画面(4806)が存在し、裏面にもカメラ(4804)もライト(4805)が存在する。
例えば、図31又は図33のような携帯端末を用いた場合、ユーザは、画面を見ながら動画撮影をすることができる。そのため、携帯端末は、図31のように、画面のある表側にカメラとライトが存在しているものを使用することが望ましい。
これに対して、図32のような携帯端末を用いた場合、ライトを瞳孔に当てるためには、ユーザの側には画面の裏側がくることになる。そうすると、ユーザは、自分の目や瞳孔が、画面のいずれの部分に存在するかを把握することができずに、操作の容易性に問題を抱えることとなる。そのため、例えば、瞳孔が画面の中央付近にない場合には「右」「左」などと指示音声で知らせる、他方、瞳孔が画面の中心付近にきた場合には「ピー」という音でユーザに知らせる等の瞳孔位置把握手段を備えていても良い。瞳孔位置把握手段は、音以外でも、例えば、ライトの照度変化によってユーザに瞳孔が画面の中心付近にきていることを知らせることでも良い。
図46は、画面の一部を照度の強い白色光映像とする場合の携帯端末装置の図である。携帯端末装置(6101)は、カメラ(6102)があるもののライトがないので、画面(6103)の下半分が照度の強い白色光映像(6104)となっている。これにより、ライトに代替することができる。
「動画ON部」は、携帯端末の動画撮影機能をONにする働きをする。また、「ライトON部」は、携帯端末の撮影側面に備えられているライトを含むライト部をONにする働きをする。本発明の最も基本的な構成として、光を当てた際の瞳孔を動画撮影する必要がある。ユーザは、動画ON部の働きによって、瞳孔部分を動画撮影することができるようになる。また、ユーザは、ライトON部の働きによって、瞳孔に対して光を当てることができる。
「瞳孔認識部」は、撮影中の像の中から人を含む動物の目の瞳孔を認識する働きをする。本発明の課題のポイントは、例えばスマートフォンなどの携帯端末装置による撮影の場合、瞳孔の大きさの変化を把握することができないという点にある。具体的には、携帯端末の動画撮影機能において、例えば顔認識機能については浸透しているものの、眼球のうち瞳孔の部分を認識するための機能は一般的ではない。本発明の課題を解決するためには、瞳孔の大きさを動画情報をもとに演算する必要があるところ、瞳孔の大きさを認識できなければ、そのような演算も不可能である。瞳孔認識部は、瞳孔の大きさを認識することを可能にするものであり、本発明の課題を解決する上で重要な機能である。
また、携帯端末装置は、瞳孔の色彩又は明度を登録していなくとも自動的に瞳孔部分を認識するための手段を備えていても良い。具体的に図38を用いて説明すると、まず眼球の濃色部分(A)薄色部分(B)白色部分(C)及び肌部分(D)を色彩又は明度によって認識する色彩明度認識手段を有し、次に眼球の薄色部分(B)と白色部分(C)の境界Eを発見する虹彩認識手段、最後に眼球の濃色部分(A)と薄色部分(B)の境界である境界Fを発見する瞳孔発見手段を有するという方法であっても良い。
そして、瞳孔部分に微動が生じたとしても、トレースして識別を継続することが望ましい。また、瞳孔部分の微動を防止するため、例えば、図50のように、携帯端末装置(6801)を横向きにして、かつ、カメラ(6802)やライト(6803)に近い場所の画面(6804)上に動画などの画像(6805)を表示させることによって、1点を見させることで瞳孔部分を固定するという工夫が考えられる。この場合、画面(6804)に白色光を表示させれば、ライトの光に代えることができる。動画などの画像を表示させる場所は、カメラ(6802)の中心線上に動画の中心線がくるように位置付けることが望ましい。また、画像中に注目しておくべき注目点(6806)を明示しても良い。
この点につき、図45を用いて説明する。図45は、人の目の部分を輪郭線のみで示した図である。人の目(6001)には、虹彩境界部分(6002)と瞳孔境界部分(6003)が存在し、瞳孔の内側に点線で示した箇所まで一層濃い色となっている部分(6004)が存在する場合がある。そのような場合、一層濃い色となっている部分(6004)を境界Fとしても良いし、瞳孔境界部分(6003)を境界Fとしても良いし、中間点を境界Fとしても良い。
図34は、瞳孔を認識した場合の画面を示す図である。携帯端末装置(4901)を用いて左図のように瞳孔を含んだ目の部分を撮影して、瞳孔部分の認識に成功した場合、右図のように、瞳孔を認識済みであることを示すように瞳孔部分(4902)を浮き上がらせる構成としても良い。これにより、瞳孔認識に成功したことをユーザに伝えることができる。その他にも、瞳孔認識に成功したことを伝える手段としては、例えば「ピー」という通知音や照度の変化などを通して伝えるという方法であっても良いし、それらに限らない。
「瞳孔変化演算部」は、認識した瞳孔の開き具合の時間変化である瞳孔変化を演算する働きをする。前述のとおり、ストレスやリラックスの程度は、瞳孔対光反応の度合いとの関係において相関関係を有する。したがって、瞳孔変化演算部は、瞳孔対光反応を演算するものである。そして、その瞳孔対光反応とは、具体的には、瞳孔に光を当てたことによる瞳孔の大きさの時間変化である。
瞳孔演算の具体的内容については、撮影部分との距離が一定ではないことを考えると、動画中の瞳孔部分の大きさを漫然と測定しても正確な瞳孔変化を測定することができない。正確な測定を実現するためには、撮影部分との距離や角度などによる影響が小さい構成があることが望ましい。具体的には、たとえば図39を用いて説明すると、虹彩部分(5401)の直径αと瞳孔部分(5402)の直径βとを除算により比較することで値を求めれば、撮影部分との距離や角度などによって左右されない。例えば、図39の例でいえば、演算結果は、「α/β」又は「β/α」である。そのため、虹彩部分の直径αと瞳孔部分の直径βとを比較する構成であることが望ましい。
このような比較を実現するためには、前記図38において説明した虹彩発見手段と瞳孔発見手段を踏まえた上で、虹彩部分及び瞳孔部分の直径を動画上で測定する直径測定手段、及び測定した直径について除算を実施する除算手段を有することで実現可能である。
その他にも、ユーザを特定するための識別情報として、氏名、年齢及び生年月日、食事傾向などの登録情報を受け付ける登録情報受付部を有していても良い。そして、この登録情報と関連付けて、ユーザの虹彩情報を受け付けると良い。それにより、携帯端末装置のカメラにより虹彩認識をすることにより、ユーザIDなどによらずに、確実なユーザ管理を行うことができる。
図35は、瞳孔変化一覧出力部において処理される統計情報の内容を示す図である。ユーザAについて、日付ごとに瞳孔変化が一覧で示されている。これに加えて、例えば、日付欄の部分に測定条件などの情報が記されていても良い。例えば、測定した場所の明るさ、測定時刻、食前・食後の別、前日の睡眠時間、計測日の仕事の有無及び強度などの情報と一緒に示されていれば、ユーザは、自己のストレスの原因を知ることができ、ストレスを減らすための生活を考えることに役立つ。
また、瞳孔の境界が分かりにくい場合には、図36のように、瞳孔認識部で認識した瞳孔部分を目立たせる工夫をしても良い。図36では、瞳孔の輪郭が明確になる工夫がされている。
加えて、携帯端末装置は、演算終了後、撮影した動画をリプレイするためのリプレイ部を備えていることが望ましい。このリプレイにおいては、瞳孔認識部における瞳孔認識が正確か否かを確認できるよう、瞳孔と認識した部分について白抜きなどするか、又は、図51にあるように径として測定に用いた部分を明示する構成にすることが望ましい。
さらには、撮影直後に瞳孔変化を確認できるように、画面中に瞳孔収縮に関するデータを表示しても良い。図52は、画面中に瞳孔収縮に関するデータを表示した一例である。ライトが明るくなったタイミング、ライトの照度、瞳孔の変化の推移などが一覧になっており、ユーザが一目して演算結果がわかるようになっている。
以上のような機能を有することによって、携帯端末装置において、ストレスとの相関関係のある他覚症状によって、ストレスの状態を測定するための技術を提供することができる。
図2は、上記機能的な各構成要件をハードウェアとして実現した際の携帯端末装置における構成の一例を示す図である。この図を利用して本装置での各処理におけるそれぞれのハードウェア構成部の働きについて説明する。
図3は、本実施例の携帯端末装置における処理の流れの一例を示すフローチャートである。以下に示すステップは、媒体に記録された計算機(携帯端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
図53は、ライトOFFステップを有する場合の処理の流れを示す図である。まず、動画ONステップ(S7101)、ライトONステップ(S7102)が存在する。次に、瞳孔認識ステップ(S7103)として認識がスタートし、瞳孔認識に成功したか否かを確認し(S7104)、瞳孔認識に成功するまで認識ステップが繰り返される。この瞳孔認識は、全体の処理が終了するまで継続する。次に、瞳孔変化演算ステップ(S7105)が存在する。この瞳孔変化演算は、全体の処理が終了するまで継続する。次に、ライトOFFステップの内容として、ライトをOFFにするかの確認があり(S7106)、YESであればライトをOFFにし(S7108)、NOであればYESになるまで処理を継続する。ここでのライトをOFFにするかの確認は、予め定められた時間の経過時、あるいは縮動停止時又は一定以下の速度となった時にライトをOFFにするということが考えられる。縮瞳速度などの生体反応に個人差があることを考えると、ライトをOFFにするタイミングは、一律ではなく、縮動停止時又は一定以下の速度となった時とすることが望ましい。最後に、全体の処理が終了か否かの確認があり、NOであれば処理を継続し、YESであれば全体の処理を終える。全体の処理が終了か否かの確認は、予め定められた時間の経過時、散動停止時又は一定以下の速度となった時、あるいは認識時の瞳孔の大きさに戻った時にYESにするということが考えられる。
図40は、以上のステップをタイムチャートにした場合の一例を示す図である。ユーザがスウィッチ(SW)をONにすることにより、動画撮影及び瞳孔認識が開始する。その上で、認識が完了して瞳孔認識機能がON(high)になると同時に又は若干遅れて、瞳孔演算が開始する。その後、予め定められた時間の経過時、あるいは縮動停止時又は一定以下の速度(図面のαで示されている範囲)となった時にライトがOFFになる。そして、更に予め定められた時間の経過時、あるいは散動停止時又は一定以下の速度(図面のβで示されている範囲)となった時、あるいは認識時の瞳孔の大きさに戻った時に全体の処理が終了する。
図40を用いて縮瞳の推移を検討すれば、ライトがONになっている間に縮瞳し、OFFになってから散瞳を始め、一定程度散瞳したところで散瞳が止まるということが分かる。
図44は、瞳孔演算の態様を説明するための図である。モデルデータとして、A及びBのデータがあったとして、Aの線のライトを当てた際の縮瞳速度(a1)、最も縮瞳した時の縮瞳率(ha)、ライトを切った後の散瞳速度(a2)、ライトをOFFにしてから散瞳が止まるまでの時間(ta)などを測定することになる。Bの線についても、同じく、ライトを当てた際の縮瞳速度(b1)、最も縮瞳した時の縮瞳率(hb)、ライトを切った後の散瞳速度(b2)、ライトをOFFにしてから散瞳が止まるまでの時間(tb)などを測定することになる。このように、ライトをONにしてから縮瞳が始まり、ライトをOFFにしてから散瞳が始まる。そして、この2つのモデルデータについて、縮瞳及び散瞳のいずれも速度が速い方がストレスが弱いという前提で分析をすれば、モデルデータAよりもモデルデータBの方がストレスが弱いということが分かる。
本実施例は、より望ましい実施形態として、実施例1の構成に加えて、ライトの照度を時間経過とともに変化させる機能を有する携帯端末装置を提供する。
図4は、本実施例の携帯端末装置の機能ブロックの一例を示す図である。この図にあるように、本実施例の携帯端末装置(0401)は、カメラ部(0402)、動画ON部(0403)とライト部(0404)とライトON部(0405)と瞳孔認識部(0406)と瞳孔変化演算部(0407)と照度変化部(0408)とを有する。本実施例の特徴は、照度変化部の内容にある。そのため、照度変化部の機能的構成を中心に説明する。なお、その他の機能については、既述の説明と同様であるので、説明を省略する。
図5は、上記機能的な各構成要件をハードウェアとして実現した際の携帯端末装置における構成の一例を示す図である。携帯端末装置は、「CPU(中央演算装置)」(0501)、「主メモリ」(0502)、「HDD」(0503)、「ネットワークI/F」(0504)、「I/O」(0508)、「システムバス」(0505)を有しており、外部機器(0506)は、「I/F」(0507)を有している。
図6は、本実施例の携帯端末装置における処理の流れの一例を示すフローチャートである。以下に示すステップは、媒体に記録された計算機(携帯端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
図43は、以上のステップをタイムチャートにした場合の一例を示す図である。ユーザがスウィッチ(SW)をONにすることにより、動画撮影、ライト及び瞳孔認識が開始する。その上で、認識が完了して瞳孔認識状態がHighになると同時に又は若干遅れて、瞳孔演算が開始し、ライトの照度が上がる。その後、予め定められた時間の経過時、あるいは縮動停止時又は一定以下の速度(図面のαで示されている範囲)となった時にライトがlowになる。そして、更に予め定められた時間の経過時、あるいは散動停止時又は一定以下の速度(図面のβで示されている範囲)となった時に全体の処理が終了する。
次に、図54は、認識時の瞳孔の大きさに戻ったときに処理を終了するという構成を可能とするためのタイムチャートを示す図である。ユーザがスウィッチ(SW)をONにすることにより、動画撮影、ライト及び瞳孔認識が開始する。その上で、認識が完了して瞳孔認識状態がHighになると同時に又は若干遅れて、瞳孔演算が開始し、元の状態の瞳孔の演算をした後、ライトの照度が上がる。その後、予め定められた時間の経過時、あるいは縮動停止時又は一定以下の速度(図面のαで示されている範囲)となった時にライトがlowになる。そして、更に予め定められた時間の経過時、あるいは散動停止時又は一定以下の速度(図面のβで示されている範囲)となった時、あるいは認識時の瞳孔の大きさに戻った時に全体の処理が終了する。このように、ライトの照度がhighになる前に演算を実施することにより、元の状態の瞳孔を演算して、その演算結果との一致をもって、全体の処理を終了するという構成をとることができる。
図43を用いて縮瞳の推移を検討すれば、ライトがlowでONになっている間に縮瞳が緩やかに始まり、ライトがhighになると縮動速度が加速し、ライトが再びlowになると散瞳が始まり、一定程度散瞳したところで散瞳が止まるということが分かる。
本実施例は、より望ましい実施形態として、実施例1から2のいずれか一の構成に加えて、ストレスの度合いを評価する機能を有する携帯端末装置を提供する。
<機能的構成>
図7は、本実施例の携帯端末装置の機能ブロックの一例を示す図である。この図にあるように、本実施例の携帯端末装置(1001)は、カメラ部(1002)と動画ON部(1003)とライト部(1004)とライトON部(1005)と瞳孔認識部(1006)と瞳孔変化演算部(1007)とストレス評価部(1008)とを有する。本実施例の特徴は、ストレス評価部の内容にある。そのため、ストレス評価部の機能的構成を中心に説明する。なお、その他の機能については、既述の説明と同様であるので、説明を省略する。
図49は、ストレス評価の方法を示した図である。2000人の統計データが図面のような曲線になる場合、縮瞳後散瞳までの時間について、中央値tからの距離を測定することにより、ストレス評価を示すという方法が考えられる。例えば、図49では、「+1σ」よりも中央側については「正常」、「+1σ」のうち中央値tよりも反対側のゾーンの場合には「弱いストレスあり」、「+2σ」のうち中央値tよりの場合には「中度のストレスあり」、「+2σ」のうち中央値tよりも反対側のゾーンの場合には「強いストレスあり」などと判断することが考えられる。さらには「+2σ」よりも中央値tと離れた「over」部分の場合には、「とても強いストレスあり」と判断することが考えられる。
図37は、ストレス評価結果出力部において出力される統計情報の内容を示す図である。日付ごとのストレス評価結果がカレンダー方式で一覧になって示されている。これにより、ユーザのストレス度合いを一覧することができ、曜日毎のストレス度合いの特徴等を知ることができる。これに加えて、例えば、日付欄の部分に瞳孔変化情報や測定条件などの情報が記されていても良い。例えば、測定した場所の明るさ、測定時刻、食前・食後の別、前日の睡眠時間、計測日の仕事の有無及び強度などの情報と一緒に示されていれば、ユーザは、自己のストレスの原因を知ることができ、ストレスを減らすための生活を考えることに役立つ。
<ハードウェア的構成>
図8は、上記機能的な各構成要件をハードウェアとして実現した際の携帯端末装置における構成の一例を示す図である。携帯端末装置は、「CPU(中央演算装置)」(1101)、「主メモリ」(1102)、「HDD」(1103)、「ネットワークI/F」(1104)、「I/O」(1108)、「システムバス」(1105)を有しており、外部機器(1106)は、「I/F」(1107)を有している。
<処理の流れ>
図9は、本実施例の携帯端末装置における処理の流れの一例を示すフローチャートである。以下に示すステップは、媒体に記録された計算機(携帯端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
本実施例は、より望ましい実施形態として、実施例1から3のいずれか一の構成に加えて、瞳孔変化を所定のアドレスに対して送信する機能を有する携帯端末装置を提供する。
<機能的構成>
図10は、本実施例の携帯端末装置の機能ブロックの一例を示す図である。この図にあるように、本実施例の携帯端末装置(1301)は、カメラ部(1302)と動画ON部(1303)とライト部(1304)とライトON部(1305)と瞳孔認識部(1306)と瞳孔変化演算部(1307)と瞳孔変化送信部(1308)とを有する。本実施例の特徴は、瞳孔変化送信部の内容にある。そのため、瞳孔変化送信部の機能的構成を中心に説明する。なお、その他の機能については、既述の説明と同様であるので、説明を省略する。
<ハードウェア的構成>
図11は、上記機能的な各構成要件をハードウェアとして実現した際の携帯端末装置における構成の一例を示す図である。携帯端末装置は、「CPU(中央演算装置)」(1401)、「主メモリ」(1402)、「HDD」(1403)、「ネットワークI/F」(1404)、「I/O」(1408)、「システムバス」(1405)を有しており、外部機器(1406)は、「I/F」(1407)を有している。
<処理の流れ>
図12は、本実施例の携帯端末装置における処理の流れの一例を示すフローチャートである。以下に示すステップは、媒体に記録された計算機(携帯端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
本実施例は、より望ましい実施形態として、実施例3又は3に追加された実施例4の構成に加えて、ストレス評価結果を所定のアドレスに対して送信する機能を有する携帯端末装置を提供する。
<機能的構成>
図13は、本実施例の携帯端末装置の機能ブロックの一例を示す図である。この図にあるように、本実施例の携帯端末装置(1901)は、カメラ部(1902)と動画ON部(1903)とライト部(1904)とライトON部(1905)と瞳孔認識部(1906)と瞳孔変化演算部(1907)とストレス評価部(1908)とストレス評価結果送信部(1909)を有する。本実施例の特徴は、ストレス評価結果送信部の内容にある。そのため、ストレス評価結果送信部の機能的構成を中心に説明する。なお、その他の機能については、既述の説明と同様であるので、説明を省略する。
<ハードウェア的構成>
図14は、上記機能的な各構成要件をハードウェアとして実現した際の携帯端末装置における構成の一例を示す図である。携帯端末装置は、「CPU(中央演算装置)」(2001)、「主メモリ」(1402)、「HDD」(1403)、「ネットワークI/F」(1404)、「I/O」(1408)、「システムバス」(1405)を有しており、外部機器(1406)は、「I/F」(1407)を有している。
<処理の流れ>
図15は、本実施例の携帯端末装置における処理の流れの一例を示すフローチャートである。以下に示すステップは、媒体に記録された計算機(携帯端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
本実施例の発明は、光を当てた際の瞳孔の変化を連続静止画撮影して、撮影画像の瞳孔変化を時間単位で演算するための機能を携帯端末装置に読取実行可能に記録したプログラムである。また、そのプログラムが読取実行可能に記録された携帯端末装置である。なお、本実施例の構成に実施例2から5のいずれか一の構成を追加したものも、本発明に含まれる。
<機能的構成>
図16は、本実施例の携帯端末装置の機能ブロックの一例である。この図にあるように、本実施例の携帯端末装置(2501)は、カメラ部(2502)と連続ON部(2503)とライト部(2504)とライトON部(2505)と瞳孔認識部(2506)と瞳孔変化演算部(2507)とを有する。本実施例の特徴的な機能は、連続ON部にある。そのため、本実施例では、連続ON部の機能を中心に説明する。
<ハードウェア的構成>
図17は、上記機能的な各構成要件をハードウェアとして実現した際の携帯端末装置における構成の一例を示す図である。この図を利用して本装置での各処理におけるそれぞれのハードウェア構成部の働きについて説明する。
<処理の流れ>
図18は、本実施例の携帯端末装置における処理の流れの一例を示すフローチャートである。なお、以下に示すステップは、媒体に記録された計算機(携帯端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
本実施例は、より望ましい実施形態として、実施例4又は瞳孔変化送信部を有する実施例6の構成を備えた携帯端末装置と繋がれている統計処理装置を提供する。統計処理装置の構成の概要としては、携帯端末装置から瞳孔変化情報を受信して、瞳孔変化情報を蓄積して、統計処理を行うというものである。
<機能的構成>
図19は、本実施例の携帯端末装置及び統計処理装置の機能ブロックの一例を示す図である。この図にあるように、本実施例の携帯端末装置(2801)は、カメラ部(2801)と動画ON部(2803)とライト部(2804)とライトON部(2805)と瞳孔認識部(2806)と瞳孔変化演算部(2807)と瞳孔変化送信部(2808)とを有する。そして、本実施例における統計処理装置(2809)は、瞳孔変化受信部(2810)、瞳孔変化蓄積部(2811)、瞳孔変化統計処理部(2812)とを有する。そして、瞳孔変化統計処理部は、ユーザ(2813)や管理者(2814)や医療機関などの第三者(2815)と繋がっている。このユーザ(2813)は、瞳孔変化送信部から瞳孔変化を統計処理装置に送信した当該携帯端末装置を含むものである。本実施例の特徴は、統計処理装置の各部の内容にある。そのため、統計処理装置の各部の機能的構成を中心に説明する。なお、その他の機能については、既述の説明と同様であるので、説明を省略する。
例えば、ユーザから自覚しているストレス度合いを入力してもらい、そのストレス度合いと瞳孔変化によって導かれるストレス度合いを比較するという方法が考えられる。図41は、縮瞳速度を横軸にとり、ストレス度合いを縦軸にとった場合の図である。統計値としてのデータが曲線(5601)のようになったとき、ユーザの自覚症状としてのストレス度合いがY1の部分(5602)であったとしても、縮瞳速度からすればストレス度合いがY2の部分(5603)である場合には、統計処理及び実際の縮瞳速度に基づいて、Y2の部分に相当するストレス度合いであると診断し、その旨を出力することができるという具合である。このようにすることで、ユーザは、適切な診断を得ることができる。
次に、図42は、縮瞳速度を横軸にとり、ストレス度合いを縦軸にとった場合の特定の作業中の時間毎のストレス変化を示す図である。統計処理の結果導き出された曲線(5701)は、例えば、Z0の部分から1時間後にはZ1の部分に、その1時間後にはZ2の部分に、という具合に時間毎のストレス変化を表している。この図において、例えば、当該特定の作業の現時点でのストレス状態がZ2の部分であるとすれば、その3時間後に対応する部分はZ5の部分であり、ストレス強度としてはY3の部分(5702)であることが予測できる。このように、時間毎のストレス変化を統計処理することによって、適切な将来予測をすることができる。
<ハードウェア的構成>
図20は、上記機能的な各構成要件をハードウェアとして実現した際の携帯端末装置及び統計処理装置における構成の一例を示す図である。携帯端末装置は、「CPU(中央演算装置)」(2901)、「主メモリ」(2902)、「HDD」(2903)、「ネットワークI/F」(2904)、「I/O」(2912)、「システムバス」(2905)を有しており、統計処理装置(2906)は、「I/F」(2907)、「ディスプレイ」(2908)、「主メモリ」(2909)、「CPU」(2910)、「フレームメモリ」(2911)を有している。
<処理の流れ>
図21は、本実施例の携帯端末装置及び統計処理装置における処理の流れの一例を示すフローチャートである。以下に示すステップは、媒体に記録された計算機(携帯端末及び統計処理端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
本実施例は、より望ましい実施形態として、実施例4に実施例5の構成が追加された携帯端末装置と繋がれている統計処理装置を提供する。統計処理装置の構成の概要としては、携帯端末装置からストレス評価結果と瞳孔変化を受信して、それらを蓄積して、統計処理を行うというものである。
<機能的構成>
図22は、本実施例の携帯端末装置及び統計処理装置の機能ブロックの一例を示す図である。この図にあるように、本実施例の携帯端末装置(3401)は、カメラ部(3402)と動画ON部(3403)とライト部(3404)とライトON部(3405)と瞳孔認識部(3406)と瞳孔変化演算部(3407)とストレス評価部(3408)と、ストレス評価結果送信部(3409)と瞳孔変化送信部(3414)を有する。そして、本実施例における統計処理装置(3410)は、ストレス評価結果受信部(3411)、ストレス評価結果蓄積部(3412)、ストレス評価統計処理部(3413)と瞳孔変化受信部(3415)と瞳孔変化蓄積部(3416)を有する。そして、ストレス評価統計処理部は、ユーザ(3417)や管理者(3418)や医療機関などの第三者(3419)と繋がっている。本実施例の特徴は、統計処理装置の各部の内容にある。そのため、統計処理装置の各部の機能的構成を中心に説明する。なお、その他の機能については、既述の説明と同様であるので、説明を省略する。
図47は、ユーザ側での出力の態様を示す図である。縦軸をストレス評価値として、横軸を瞳孔変化速度としたグラフである。瞳孔変化速度が高ければ高いほどストレス評価値が小さくなるということが分かる。
[規則91に基づく訂正 27.07.2015]
さらに、図48は、ユーザ側での出力の態様を示す図である。縦軸をストレス評価値として、横軸を体組成量としたグラフである。体脂肪率、体重、BMI、脈拍については、体組成量が高ければ高いほどストレス評価値が高くなるということが分かる。他方、基礎代謝については、体組成量が高ければ高いほどストレス評価値が低くなるということが分かる。このような統計情報が得られれば、ユーザ等は、ストレスの原因を客観的に分析することができるようになる。しかも、瞳孔変化との比較より、ストレス評価値の方がストレスとの相関が明確なので、一見して分かり易い。
<ハードウェア的構成>
図23は、上記機能的な各構成要件をハードウェアとして実現した際の携帯端末装置及び統計処理装置における構成の一例を示す図である。携帯端末装置は、「CPU(中央演算装置)」(3501)、「主メモリ」(3502)、「HDD」(3503)、「ネットワークI/F」(3504)、「I/O」(3512)、「システムバス」(3505)を有しており、統計処理装置(3506)は、「I/F」(3507)、「ディスプレイ」(3508)、「主メモリ」(3509)、「CPU」(3510)、「フレームメモリ」(3511)を有している。
<処理の流れ>
図24は、本実施例の携帯端末装置及び統計処理装置における処理の流れの一例を示すフローチャートである。以下に示すステップは、媒体に記録された計算機(携帯端末、統計処理端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
本実施例の発明は、光を当てた際の瞳孔の変化を動画撮影して、瞳孔変化を時間単位で演算するための動画情報を保持し、外部機器に送信するための機能を携帯端末装置に読取実行可能に記録したプログラムである。また、そのプログラムが読取実行可能に記録された携帯端末装置である。
<機能的構成>
図25は、本実施例の携帯端末装置の機能ブロックの一例である。この図にあるように、本実施例の携帯端末装置(4001)は、カメラ部(4002)と動画ON部(4003)とライト部(4004)とライトON部(4005)と瞳孔認識部(4006)と動画保持部(4007)と動画送信部(4008)とを有する。本実施例の特徴は、動画保持部及び動画送信部の内容にある。そのため、以下では、動画保持部及び動画送信部の内容を中心に説明する。
<ハードウェア的構成>
図26は、上記機能的な各構成要件をハードウェアとして実現した際の携帯端末装置における構成の一例を示す図である。この図を利用して本装置での各処理におけるそれぞれのハードウェア構成部の働きについて説明する。
<処理の流れ>
図27は、本実施例の携帯端末装置における処理の流れの一例を示すフローチャートである。なお、以下に示すステップは、媒体に記録された計算機(携帯端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
本実施例の発明は、光を当てた際の瞳孔の変化を連続静止画撮影して、瞳孔変化を時間単位で演算するための連続静止画情報を保持し、外部機器に送信するための機能を携帯端末装置に読取実行可能に記録したプログラムである。また、そのプログラムが読取実行可能に記録された携帯端末装置である。
<機能的構成>
図28は、本実施例の携帯端末装置の機能ブロックの一例である。この図にあるように、本実施例の携帯端末装置(4301)は、カメラ部(4302)と連続ON部(4303)とライト部(4304)とライトON部(4305)と瞳孔認識部(4306)と連続静止画保持部(4307)と連続静止画送信部(4308)とを有する。本実施例の特徴は、連続静止画保持部及び連続静止画送信部の内容にある。そのため、以下では、動画保持部及び動画送信部の内容を中心に説明する。
<ハードウェア的構成>
図29は、上記機能的な各構成要件をハードウェアとして実現した際の携帯端末装置における構成の一例を示す図である。この図を利用して本装置での各処理におけるそれぞれのハードウェア構成部の働きについて説明する。
<処理の流れ>
図30は、本実施例の携帯端末装置における処理の流れの一例を示すフローチャートである。なお、以下に示すステップは、媒体に記録された計算機(携帯端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
本実施例は、携帯端末装置から受信した動画をもとに瞳孔対光反応を演算するための機能を有する演算装置を提供する。
<機能的構成>
図55は、本実施例の演算装置の機能ブロックの一例を示す図である。この図にあるように、本実施例の演算装置(7301)は、動画受信部(7302)、瞳孔認識部(7303)と瞳孔変化演算部(7304)を有する。また、本実施例において、携帯端末は、通常、実施例1におけるカメラ部とライト部と動画ON部とライトON部を有している。その場合の携帯端末のカメラ部とライト部と動画ON部とライトON部の特徴は、実施例1と同様である。また、さらに携帯端末は、通常、動画を本実施例における演算装置に送信するための動画送信部を有している。
瞳孔認識部と瞳孔変化演算部の機能の詳細は、実施例1における携帯端末装置の瞳孔認識部及び瞳孔変化演算部の詳細と同様である。
<ハードウェア的構成>
図56は、上記機能的な各構成要件をハードウェアとして実現した際の演算装置における構成の一例を示す図である。演算装置は、「CPU」(7401)、「主メモリ」(7402)、「HDD」(7403)、「ネットワークI/F」(7404)、「I/O」(7408)、「システムバス」(7405)を有しており、携帯端末装置(7406)と「ネットワークI/F」を通じて繋がっている。
<処理の流れ>
図57は、本実施例の演算装置における処理の流れの一例を示すフローチャートである。なお、以下に示すステップは、媒体に記録された計算機(携帯端末、演算端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
本実施例は、携帯端末装置から受信した連続静止画をもとに瞳孔対光反応を演算するための機能を有する演算装置を提供する。
<機能的構成>
図58は、本実施例の演算装置の機能ブロックの一例を示す図である。この図にあるように、本実施例の演算装置(7601)は、連続静止画受信部(7602)、瞳孔認識部(7603)と瞳孔変化演算部(7604)を有する。また、本実施例において、携帯端末は、通常、実施例6におけるカメラ部とライト部と連続ON部とライトON部を有している。その場合の携帯端末のカメラ部とライト部と連続ON部とライトON部の特徴は、実施例6と同様である。また、さらに携帯端末は、通常、連続静止画を本実施例における演算装置に送信するための連続静止画送信部を有している。
<ハードウェア的構成>
図59は、上記機能的な各構成要件をハードウェアとして実現した際の演算装置における構成の一例を示す図である。演算装置は、「CPU」(7701)、「主メモリ」(7702)、「HDD」(7703)、「ネットワークI/F」(7704)、「I/O」(7708)、「システムバス」(7705)を有しており、携帯端末装置(7706)と「ネットワークI/F」を通じて繋がっている。
<処理の流れ>
図60は、本実施例の演算装置における処理の流れの一例を示すフローチャートである。なお、以下に示すステップは、媒体に記録された計算機(携帯端末、演算端末)を制御するためのプログラムを構成する処理ステップであっても構わない。
4602、4702、4802:操作ボタン
4603、4703、4803:受話部
4604、4704、4804:カメラ
4605、4705、4805:ライト
4606、4706、4806:画面
4902:瞳孔部分
Claims (25)
- 携帯端末の動画撮影機能をONにする動画ONステップと、
携帯端末の撮影側面に備えられているライトをONにするライトONステップと、
撮影中の像の中から人を含む動物の目の瞳孔を認識する瞳孔認識ステップと、
認識した瞳孔の開き具合の時間変化である瞳孔変化を演算する瞳孔変化演算ステップと、
を携帯端末装置に読取実行可能に記録したプログラム。 - ライトONステップにてONにされたライトの照度を時間経過とともに変化させる照度変化ステップをさらに有する請求項1に記載のプログラム。
- さらに、
瞳孔変化演算ステップにて演算されて得られた瞳孔変化に基づいてストレス度合いを評価してストレス評価結果を得るストレス評価ステップを有する請求項1又は2に記載のプログラム。 - 瞳孔変化演算ステップにて演算されて得られた瞳孔変化を所定のアドレスに対して送信する瞳孔変化送信ステップをさらに有する請求項1から3のいずれか一に記載のプログラム。
- ストレス評価ステップにて得られたストレス評価結果を所定のアドレスに対して送信するストレス評価結果送信ステップをさらに有する請求項3又は請求項3に従属する請求項4に記載のプログラム。
- 請求項4に記載のプログラムを実行した結果として送信された瞳孔変化を受信する瞳孔変化受信ステップと、
受信した瞳孔変化を蓄積する瞳孔変化蓄積ステップと、
蓄積された瞳孔変化を統計処理する瞳孔変化統計処理ステップと、
を計算機に読取実行可能に記載したプログラム。 - さらに請求項4に従属する請求項5に記載のプログラムを実行した結果として送信されたストレス評価結果を受信するストレス評価結果受信ステップと、
受信したストレス評価結果を蓄積するストレス評価結果蓄積ステップと、
さらに請求項4に従属する請求項5に記載のプログラムを実行した結果として送信された瞳孔変化を受信する瞳孔変化受信ステップと、
受信した瞳孔変化を蓄積する瞳孔変化蓄積ステップと、
蓄積されたストレス評価結果及び瞳孔変化を統計処理するストレス統計処理ステップと、
を計算機に読取実行可能に記載したプログラム。 - 携帯端末の動画撮影機能をONにする動画ONステップに代えて、
携帯端末の連続静止画撮影機能をONにする連続ONステップとした請求項1から7のいずれか一に記載のプログラム。 - 携帯端末の動画撮影機能をONにする動画ONステップと、
携帯端末の撮影側面に備えられているライトをONにするライトONステップと、
撮影中の像の中から人を含む動物の目の瞳孔を認識する瞳孔認識ステップと、
動画ONステップにて撮影された動画を保持する動画保持ステップと、
保持された動画を所定のアドレスに送信する動画送信ステップと、
を携帯端末装置に読取実行可能に記録したプログラム。 - 携帯端末の動画撮影機能をONにする動画ONステップに代えて、携帯端末の連続静止画撮影機能をONにする連続ONステップとし、
動画ONステップにて撮影された動画を保持する動画保持ステップに代えて、携帯端末にて撮影された連続静止画を保持する連続静止画保持ステップとし、
保持された動画を所定のアドレスに送信する動画送信ステップに代えて、連続静止画保持ステップにて保持されている連続静止画を送信する連続静止画送信ステップを、
携帯端末装置に読取実行可能に記録した請求項14に記載のプログラム。 - 携帯端末から人を含む動物の目を含む動画を受信する動画受信ステップと、
受信した動画の中から人を含む動物の目の瞳孔を認識する瞳孔認識ステップと、
認識した瞳孔の開き具合の時間変化である瞳孔変化を演算する瞳孔変化演算ステップと、
を計算機に読取実行可能に記録したプログラム。 - 携帯端末から人を含む動物の目を含む連続静止画を受信する連続静止画受信ステップと、
受信した連続静止画の中から人を含む動物の目の瞳孔を認識する瞳孔認識ステップと、
認識した瞳孔の開き具合の時間変化である瞳孔変化を演算する瞳孔変化演算ステップと、
を計算機に読取実行可能に記録したプログラム。 - 動画又は/及び静止画の撮影のためのカメラ部と、
携帯端末の撮影側面に備えられているライトを含むライト部と、
携帯端末の動画撮影機能をONにする動画ON部と、
前記ライトをONにするライトON部と、
撮影中の像の中から人を含む動物の目の瞳孔を認識する瞳孔認識部と、
認識した瞳孔の開き具合の時間変化である瞳孔変化を演算する瞳孔変化演算部と、
を有する携帯端末装置。 - ライトON部にてONにされたライトの照度を時間経過とともに変化させる照度変化部をさらに有する請求項13に記載の携帯端末装置。
- さらに、
瞳孔変化演算部にて演算されて得られた瞳孔変化に基づいてストレス度合いを評価してストレス評価結果を得るストレス評価部を有する請求項13又は14に記載の携帯端末装置。 - 瞳孔変化演算部にて演算されて得られた瞳孔変化を所定のアドレスに対して送信する瞳孔変化送信部をさらに有する請求項13から15のいずれか一に記載の携帯端末装置。
- ストレス評価部にて得られたストレス評価結果を所定のアドレスに対して送信するストレス評価結果送信部をさらに有する請求項15又は15に従属する請求項16に記載の携帯端末装置。
- 携帯端末の動画撮影機能をONにする動画ON部に代えて、
携帯端末の連続静止画撮影機能をONにする連続ON部を有する請求項13から請求項17のいずれか一に記載の携帯端末装置。 - 請求項16又は請求項16の一部を代替した請求項18に記載の携帯端末装置を実行した結果として送信された瞳孔変化を受信する瞳孔変化受信部と、
受信した瞳孔変化を蓄積する瞳孔変化蓄積部と、
蓄積された瞳孔変化を統計処理する瞳孔変化統計処理部と、
を有する統計処理装置。 - さらに請求項16に従属する請求項17に記載の携帯端末装置を実行した結果として送信されたストレス評価結果を受信するストレス評価結果受信部と、
受信したストレス評価結果を蓄積するストレス評価結果蓄積部と、
さらに請求項16に従属する請求項17に記載の携帯端末装置を実行した結果として送信された瞳孔変化を受信する瞳孔変化受信部と、
受信した瞳孔変化を蓄積する瞳孔変化蓄積部と、
蓄積されたストレス評価結果及び瞳孔変化を統計処理するストレス統計処理部と、
を有する携帯端末装置。 - 動画又は/及び静止画の撮影のためのカメラ部と、
携帯端末の撮影側面に備えられているライトを含むライト部と、
携帯端末の動画撮影機能をONにする動画ON部と、
前記ライトをONにするライトON部と、
撮影中の像の中から人を含む動物の目の瞳孔を認識する瞳孔認識部と、
動画ON部にて撮影された動画を保持する動画保持部と、
保持された動画を所定のアドレスに送信する動画送信部と、
を有する携帯端末装置。 - 携帯端末の動画撮影機能をONにする動画ON部に代えて、携帯端末の連続静止画撮影機能をONにする連続ON部とし、
動画ON部にて撮影された動画を保持する動画保持部に代えて、携帯端末にて撮影された連続静止画を保持する連続静止画保持部とし、
保持された動画を所定のアドレスに送信する動画送信部に代えて、連続静止画保持部にて保持されている連続静止画を送信する連続静止画送信部を、
有する請求項21に記載の携帯端末装置。 - 携帯端末から人を含む動物の目を含む動画を受信する動画受信部と、
受信した動画の中から人を含む動物の目の瞳孔を認識する瞳孔認識部と、
認識した瞳孔の開き具合の時間変化である瞳孔変化を演算する瞳孔変化演算部を、
有する演算装置。 - 携帯端末から人を含む動物の目を含む連続静止画を受信する連続静止画受信部と、
受信した連続静止画の中から人を含む動物の目の瞳孔を認識する瞳孔認識部と、
認識した瞳孔の開き具合の時間変化である瞳孔変化を演算する瞳孔変化演算部を、
有する演算装置。 - 携帯端末に代えて端末とした請求項1から12のいずれか一に記載のプログラム。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2017520169A JP6612339B2 (ja) | 2015-05-27 | 2015-05-27 | 携帯端末用ストレス評価プログラム及び該プログラムを備えた携帯端末 |
PCT/JP2015/065307 WO2016189711A1 (ja) | 2015-05-27 | 2015-05-27 | 携帯端末用ストレス評価プログラム及び該プログラムを備えた携帯端末 |
CN201580081951.9A CN108024768B (zh) | 2015-05-27 | 2015-05-27 | 可移动终端压力评估程序及包含该程序的可移动终端 |
US15/576,965 US10741286B2 (en) | 2015-05-27 | 2015-05-27 | Stress evaluation program for mobile terminal and mobile terminal provided with program |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2015/065307 WO2016189711A1 (ja) | 2015-05-27 | 2015-05-27 | 携帯端末用ストレス評価プログラム及び該プログラムを備えた携帯端末 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016189711A1 true WO2016189711A1 (ja) | 2016-12-01 |
Family
ID=57393882
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/065307 WO2016189711A1 (ja) | 2015-05-27 | 2015-05-27 | 携帯端末用ストレス評価プログラム及び該プログラムを備えた携帯端末 |
Country Status (4)
Country | Link |
---|---|
US (1) | US10741286B2 (ja) |
JP (1) | JP6612339B2 (ja) |
CN (1) | CN108024768B (ja) |
WO (1) | WO2016189711A1 (ja) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2019155011A (ja) * | 2018-03-16 | 2019-09-19 | 株式会社トプコン | 携帯端末及び携帯端末の制御方法 |
GB2576139A (en) * | 2018-07-20 | 2020-02-12 | Ophthalmic Surgical Services Ltd | Ocular assessment |
JP2020049154A (ja) * | 2018-09-28 | 2020-04-02 | スカラ株式会社 | 眼球撮像装置 |
JP2020528817A (ja) * | 2017-07-28 | 2020-10-01 | ピューピルスキャン コーポレイション | 毒性及び神経生理性を判定するために瞳孔画像をキャプチャ及び分析するシステム及び方法 |
WO2021059080A1 (ja) * | 2019-09-27 | 2021-04-01 | 株式会社半導体エネルギー研究所 | 統計モデルの作成方法、状態推定方法、および状態推定システム |
WO2022123746A1 (ja) * | 2020-12-10 | 2022-06-16 | 日本電信電話株式会社 | 情報処理方法、情報処理装置、及びプログラム |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11403881B2 (en) * | 2017-06-19 | 2022-08-02 | Paypal, Inc. | Content modification based on eye characteristics |
EP3430973A1 (en) * | 2017-07-19 | 2019-01-23 | Sony Corporation | Mobile system and method |
EP3841957B1 (en) * | 2018-08-20 | 2024-05-01 | FUJIFILM Corporation | Medical image processing system |
US11354805B2 (en) * | 2019-07-30 | 2022-06-07 | Apple Inc. | Utilization of luminance changes to determine user characteristics |
CN113572993B (zh) * | 2020-04-27 | 2022-10-11 | 华为技术有限公司 | 一种视频处理方法及移动终端 |
CN114120435A (zh) * | 2020-08-25 | 2022-03-01 | 深圳爱酷智能科技有限公司 | 吸毒人员检测方法、装置、计算机可读存储介质及设备 |
JP2023045097A (ja) * | 2021-09-21 | 2023-04-03 | 株式会社東芝 | 分析システム、分析方法、プログラム、及び記憶媒体 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008212179A (ja) * | 2007-02-28 | 2008-09-18 | Morinaga & Co Ltd | ストレス強度演算方法、及びストレス強度演算装置 |
JP2009142469A (ja) * | 2007-12-14 | 2009-07-02 | Panasonic Electric Works Co Ltd | ストレス判定システム |
JP2011239891A (ja) * | 2010-05-17 | 2011-12-01 | Nippon Telegr & Teleph Corp <Ntt> | 感情強度測定装置、感情強度測定方法、及びプログラム |
JP2012065016A (ja) * | 2010-09-14 | 2012-03-29 | Kddi Corp | 携帯通信端末装置、ストレス状態演算方法、ストレス状態演算プログラム |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB0119859D0 (en) * | 2001-08-15 | 2001-10-10 | Qinetiq Ltd | Eye tracking system |
US6896655B2 (en) * | 2002-08-05 | 2005-05-24 | Eastman Kodak Company | System and method for conditioning the psychological state of a subject using an adaptive autostereoscopic display |
US20040176991A1 (en) * | 2003-03-05 | 2004-09-09 | Mckennan Carol | System, method and apparatus using biometrics to communicate dissatisfaction via stress level |
JP4179968B2 (ja) | 2003-11-13 | 2008-11-12 | 花王株式会社 | リラックス感評価用瞳孔対光反応計測具 |
JP4247100B2 (ja) | 2003-12-03 | 2009-04-02 | 三菱重工業株式会社 | 排気ガス再循環装置付き内燃機関 |
CN100427033C (zh) * | 2006-12-19 | 2008-10-22 | 潘晓刚 | 心理状态参数测试系统 |
US8911087B2 (en) * | 2011-05-20 | 2014-12-16 | Eyefluence, Inc. | Systems and methods for measuring reactions of head, eyes, eyelids and pupils |
CN102727223B (zh) * | 2012-06-18 | 2015-04-29 | 天津大学 | 基于多生理参数pca融合的脑力负荷测量方法 |
US9596508B2 (en) * | 2013-03-15 | 2017-03-14 | Sony Corporation | Device for acquisition of viewer interest when viewing content |
JPWO2014184868A1 (ja) * | 2013-05-14 | 2017-02-23 | 株式会社東芝 | 電子機器および生体信号測定方法 |
US9360672B2 (en) * | 2013-07-11 | 2016-06-07 | Seiko Epson Corporation | Head mounted display device and control method for head mounted display device |
US20150025917A1 (en) * | 2013-07-15 | 2015-01-22 | Advanced Insurance Products & Services, Inc. | System and method for determining an underwriting risk, risk score, or price of insurance using cognitive information |
US9237847B2 (en) * | 2014-02-11 | 2016-01-19 | Welch Allyn, Inc. | Ophthalmoscope device |
US20150297140A1 (en) * | 2014-04-21 | 2015-10-22 | Microsoft Corporation | User stress detection and mitigation |
-
2015
- 2015-05-27 JP JP2017520169A patent/JP6612339B2/ja active Active
- 2015-05-27 US US15/576,965 patent/US10741286B2/en active Active
- 2015-05-27 WO PCT/JP2015/065307 patent/WO2016189711A1/ja active Application Filing
- 2015-05-27 CN CN201580081951.9A patent/CN108024768B/zh active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008212179A (ja) * | 2007-02-28 | 2008-09-18 | Morinaga & Co Ltd | ストレス強度演算方法、及びストレス強度演算装置 |
JP2009142469A (ja) * | 2007-12-14 | 2009-07-02 | Panasonic Electric Works Co Ltd | ストレス判定システム |
JP2011239891A (ja) * | 2010-05-17 | 2011-12-01 | Nippon Telegr & Teleph Corp <Ntt> | 感情強度測定装置、感情強度測定方法、及びプログラム |
JP2012065016A (ja) * | 2010-09-14 | 2012-03-29 | Kddi Corp | 携帯通信端末装置、ストレス状態演算方法、ストレス状態演算プログラム |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020528817A (ja) * | 2017-07-28 | 2020-10-01 | ピューピルスキャン コーポレイション | 毒性及び神経生理性を判定するために瞳孔画像をキャプチャ及び分析するシステム及び方法 |
JP2019155011A (ja) * | 2018-03-16 | 2019-09-19 | 株式会社トプコン | 携帯端末及び携帯端末の制御方法 |
JP7096020B2 (ja) | 2018-03-16 | 2022-07-05 | 株式会社トプコン | 携帯端末及び携帯端末の制御方法 |
GB2576139A (en) * | 2018-07-20 | 2020-02-12 | Ophthalmic Surgical Services Ltd | Ocular assessment |
JP2020049154A (ja) * | 2018-09-28 | 2020-04-02 | スカラ株式会社 | 眼球撮像装置 |
WO2021059080A1 (ja) * | 2019-09-27 | 2021-04-01 | 株式会社半導体エネルギー研究所 | 統計モデルの作成方法、状態推定方法、および状態推定システム |
WO2022123746A1 (ja) * | 2020-12-10 | 2022-06-16 | 日本電信電話株式会社 | 情報処理方法、情報処理装置、及びプログラム |
Also Published As
Publication number | Publication date |
---|---|
US10741286B2 (en) | 2020-08-11 |
CN108024768B (zh) | 2021-02-05 |
JP6612339B2 (ja) | 2019-11-27 |
JPWO2016189711A1 (ja) | 2018-04-26 |
CN108024768A (zh) | 2018-05-11 |
US20180255167A1 (en) | 2018-09-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6612339B2 (ja) | 携帯端末用ストレス評価プログラム及び該プログラムを備えた携帯端末 | |
US20210165490A1 (en) | Wearable computing apparatus and method | |
US9521335B2 (en) | Detecting febrile seizure with a thermal video camera | |
US9723992B2 (en) | Mental state analysis using blink rate | |
US10614564B2 (en) | Method to determine impaired ability to operate a motor vehicle | |
DE102016118777A1 (de) | Systeme zur Detektion von physiologischen Reaktionen auf der Grundlage von thermischen Messwerten des Gesichts | |
US20170258319A1 (en) | System and method for assessing eyesight acuity and hearing ability | |
Ishii et al. | Measuring attentional bias to peripheral facial deformities | |
US20150215412A1 (en) | Social network service queuing using salience | |
JP2020512120A (ja) | ユーザーの健康状態を評価するシステム | |
US8911090B2 (en) | System for at-home eye movement monitoring | |
EP3420713B1 (en) | Devices, system and methods for determining a priority level and/or conversation duration of a call | |
JP2007068620A (ja) | 心理状態計測装置 | |
EP3897388B1 (en) | System and method for reading and analysing behaviour including verbal, body language and facial expressions in order to determine a person's congruence | |
US10070787B2 (en) | System and method for detection and monitoring of a physical condition of a user | |
JP2020120908A (ja) | 精神状態推定システム、精神状態推定方法、及び、プログラム | |
JP6811971B2 (ja) | 携帯端末用ストレス評価プログラム及び該プログラムを備えた携帯端末 | |
JP6952293B2 (ja) | 携帯端末用ストレス評価プログラム及び該プログラムを備えた携帯端末 | |
US20220409120A1 (en) | Information Processing Method, Computer Program, Information Processing Device, and Information Processing System | |
US20210369161A1 (en) | System and method for detection and continuous monitoring of neurological condition of a user | |
US11589833B2 (en) | Imaging system and control method for imaging system | |
US20230148944A1 (en) | Systems and methods for screening subjects for neuropathology associated with a condition utilizing a mobile device | |
US20230229372A1 (en) | Display device, display method, and computer-readable storage medium | |
WO2022212052A1 (en) | Stress detection | |
CN114219772A (zh) | 预测健康参数的方法、装置、终端设备、存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15893345 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2017520169 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15576965 Country of ref document: US |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 15893345 Country of ref document: EP Kind code of ref document: A1 |