JP2006087832A - Pupil detection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a pupil detection apparatus capable of automatically extracting a zone corresponding to a pupil from a captured image of an eyeball and accurately and easily detecting the pupil without extracting dark zones in the captured image such as a mole, eyelashes, or a spot, together with the pupil. <P>SOLUTION: The pupil detection apparatus has an imaging part 1 for imaging the eyeball E and a display part 2 for displaying the image captured by the imaging part 1. The pupil detection apparatus also has a selection means for selecting an arbitrary position in the captured image displayed in the display part 2, and an arithmetic operation control part 3 for extracting only a zone corresponding to the pupil from the information on the captured image when the selected position corresponds to the pupil in the captured image based on the information on the captured image and the positional information on the position selected by the selection means. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a pupil detection device that detects a pupil by extracting a region corresponding to the pupil from a captured image obtained by imaging an eyeball.

  2. Description of the Related Art Conventionally, as an apparatus for performing various kinds of measurements by performing image processing on an image obtained by imaging a subject's eyeball, the pupil diameter and the size of the pupil are based on an image obtained by imaging the eyeball using a CCD camera or the like. Proposals have been made of measuring nystagmus or performing nystagmus examination by measuring pupil movement (see Patent Documents 1 and 2).

  In such a pupil detection device, a region corresponding to the pupil is extracted from the captured image by performing binarization processing on the image obtained by capturing the eyeball, and the position and size of the pupil are measured based on this, or Further, the center position of the pupil is detected.

Various methods for extracting a region corresponding to a pupil from a captured image have been studied, but extraction by binarization processing is usually performed. In this case, an operation unit such as an operation switch for changing the threshold value according to the setting operation of the operator is provided, and the operator manually changes the threshold value while checking the binarized image on the monitor or the like. I was doing. It is conceivable to set this threshold value by automatic control, but it is appropriate that the pupil and iris are distinguished, or that dark areas in the captured image such as moles, eyelashes, and stains are not extracted together with the pupil. It was difficult to set a simple threshold.
JP 2001-178679 A JP 2003-102124 A

  Especially in the case of nystagmus examination, it is important to accurately detect the position of the pupil, and in order to improve the operability and speed up the measurement when performing detection, the pupil is automatically detected based on the captured image. It is desirable to extract the corresponding region.

  The present invention has been made in view of the above points, and can automatically extract a region corresponding to a pupil from a captured image of an eyeball, and can also be used in captured images such as moles, eyelashes, and spots. It is an object of the present invention to provide a pupil detection device capable of accurately and simply detecting a pupil without extracting a dark region together with the pupil.

  The pupil detection device according to the present invention includes an imaging unit 1 that images the eyeball E, a display unit 2 that displays the captured image, a selection unit that selects an arbitrary position in the captured image displayed on the display unit 2, and an imaging An arithmetic control unit that extracts only the region corresponding to the pupil P from the information of the captured image when the selected position corresponds to the pupil P in the captured image based on the information of the image and the position information of the selection position by the selection unit 3.

  The arithmetic control unit 3 determines the position where the density value of the pixel connected to the pixel corresponding to the selection position by the selection unit in the captured image changes beyond a predetermined range as the boundary of the region corresponding to the pupil, It is desirable to extract the inside of the boundary as a region corresponding to the pupil.

  It is also desirable that the arithmetic control unit 3 derives the center position of the region corresponding to the extracted pupil P.

  Further, the arithmetic control unit 3 derives a circle inscribed in the region corresponding to the extracted pupil P, and derives the center position of the circle as the center position of the region corresponding to the pupil P. Is desirable.

  It is also desirable that the arithmetic control unit 3 derives the center of gravity of the region corresponding to the extracted pupil P and derives the center of gravity of the circle as the center position of the region corresponding to the pupil P.

  According to the present invention, when extracting a region corresponding to the pupil from the captured image obtained by the imaging unit, the region corresponding to the pupil is extracted from the captured image without requiring an external threshold setting operation. It is possible to prevent the region not corresponding to the pupil from being erroneously extracted. Thereby, the detection of the pupil can be performed accurately and simply.

  Hereinafter, the present invention will be described based on the best mode for carrying out the invention.

  The pupil detection device according to the present invention accurately and simply measures the position and shape of the pupil P of the subject or the center position of the pupil P, and can be used for various examinations. Can be used. Described below is an example when the pupil detection device according to the present invention is formed as an nystagmus examination device.

  The pupil detection device according to the present invention includes an imaging unit 1 that images the eyeball E of a subject, a display unit 2 that displays the captured image, and a selection unit that selects an arbitrary part of the image displayed on the display unit 2. And when the selected part is a part corresponding to the pupil P in the image based on the image information picked up by the image pickup unit 1 and the position information of the part selected by the selection unit, the pupil from the image information An arithmetic control unit 3 that detects the center position of the pupil P by extracting a region corresponding to P or further deriving the center of this region is provided.

  The imaging unit 1 can be housed in a goggle type measuring device 10 that can be worn on the face of a subject. As this imaging part 1, what comprises a CCD camera, a CMOS sensor, etc. can be provided, for example.

  FIG. 1 shows an example of the configuration of an optical system including the imaging unit 1 in the measurement apparatus 10. As shown in the figure, the optical system of the eyeball E detection device according to the present embodiment includes a concave mirror 5 disposed so as to face the eyeball E when the subject wears the measuring device 10, and a space between the eyeball E and the concave mirror 5. And a half mirror 6 disposed on the surface. The half mirror 6 is inclined about an axis in a direction orthogonal to a line connecting the eyeball E and the concave mirror 5. Further, the image display means 4 such as a small liquid crystal panel and the image pickup unit 1 such as a CCD camera arranged with the half mirror 6 interposed therebetween are provided. At this time, the line connecting the image display means 4 and the image pickup unit 1 is the eyeball E. And the line connecting the concave mirror 5 and the tilt axis of the half mirror 6 are orthogonal to each other. Further, one surface of the half mirror 6 faces in an inclined state toward the eyeball E and the imaging unit 1, and the other surface of the half mirror 6 faces in an inclined state toward the video presentation means 4 and the concave mirror 5. Yes. In this embodiment, two such optical systems are provided so as to correspond to both eyes of the subject.

  Further, in the present embodiment, infrared light irradiating means 9 such as an infrared light emitting diode that is located near the eyeball E of the subject and irradiates the subject's eyeball E with infrared light is provided, and the imaging unit 1 is an infrared transmission type. The filter 8 is provided and has sensitivity to infrared rays. In addition, the half mirror 6 is provided with a wavelength selection layer 7 that is formed by coating on the surface on the imaging unit 1 side and reflects infrared light.

  The goggle-type measuring apparatus 10 is configured by providing a holding member 12 such as a belt on a main body 11 that holds two sets of optical system units as described above. When the subject wears the measuring device 10, the body 11 is held and fixed by, for example, wrapping a holding member 12 such as a belt around the head of the subject while the body 11 is positioned in front of the subject's eyes. (See FIG. 2)).

  Moreover, the display part 2 can be provided in the control apparatus 13 separate from the said measuring apparatus 10, and the calculation control part 3 can also be provided in this control apparatus 13 (refer FIG. 3). Information exchange between the measuring device 10 and the control device 13 can be performed by a cable 15 or the like connecting the devices.

  As the display unit 2, a liquid crystal display panel, a CRT display, or other appropriate video display device can be used. In addition, in order to select an arbitrary position in the region corresponding to the portion of the pupil P of the captured image on the screen of the display unit 2 as described above, any type of selection means can be employed. A touch panel display is adopted as the screen of the display unit 2, and the operator selects an arbitrary position in the region corresponding to the pupil P portion of the captured image on the screen of the display unit 2 with a finger or a stylus. Can do. As the touch panel, an appropriate type such as a resistance film type, a capacitance type, an electromagnetic induction type, an ultrasonic type, an infrared type, a pressure sensor type, or the like can be adopted.

  The arithmetic control unit 3 can be composed of necessary software, captured image information, selection information by the selection means, storage means such as a memory for storing calculation result information, a control means such as a CPU, and the like. The arithmetic control unit 3 performs operation control such as image processing of a captured image captured by the image capturing unit 1 and generation of a video to be presented to the video presenting means 4, and the operation of the entire pupil detection device according to the present invention. Take control. A specific example of operation control will be described later.

  In addition, various functions can be added to the pupil detection device as necessary. For example, when used as a nystagmus inspection device, analysis for performing rhythm fluctuation analysis from time-series data of the measured movement of the eyeball E Function, estimation function to estimate whether nystagmus is an abnormality caused by peripheral disease of the subject's body or an abnormality caused by central system disease, display that displays the analysis result and estimation result as diagnostic support data, for example Functions and the like can be added. An analysis function, an estimation function, and the like can be provided in the arithmetic control unit 3 or a separate device can be provided. In order to provide a display function, for example, a printing device such as a printer, or a display device 14 such as a CRT display or a liquid crystal display can be provided.

  The operation of the above pupil detection device will be described below.

  First, the video display means 3 displays a video with the subject wearing the measuring device 10. As the video, a visual target for inducing reciprocation of the eyeball E of the subject, for example, a display in which a striped pattern having a plurality of vertical lines moves with time is displayed.

  A part of the light emitted from the image display means is reflected by the half mirror 6 toward the concave mirror 5 side, and further reflected by the concave mirror 5 and reflected by the concave mirror 5, as indicated by solid arrows in FIG. Is transmitted through the half mirror 6 and enters the eyeball E of the subject. At this time, the subject sees an enlarged virtual image.

  Further, the infrared light irradiated from the infrared light irradiating means 9 toward the eyeball E of the subject is transmitted through the pupil P of the eyeball E, but is reflected at a portion other than the pupil P as shown by a broken line in FIG. Most of the reflected light is reflected by the half mirror 6 and received by the imaging unit 1, whereby an image of the eyeball E is captured.

  An image captured by the imaging unit 1 is displayed on the display unit 2. A person who operates the apparatus selects an arbitrary position in the region corresponding to the pupil P portion of the captured image on the screen of the display unit 2 by the selection unit based on the display result on the display unit 2.

  When the selection is made, the arithmetic control unit 3 first extracts pixels in the captured image corresponding to the selected position on the screen of the display unit 2.

  Then, the position where the density value of the pixel connected to the extracted pixel greatly changes is determined as the boundary of the pupil region, and the inside of the boundary is extracted as the pupil region.

  Specifically, for example, as shown in FIG. 4, first, the density value of a pixel along a reference line (for example, the horizontal scanning line L, see FIG. 4A) that passes through the pixel extracted by the selection operation described above. In the change waveform (see FIG. 4B), if the change in the density value of the pixel starting from the pixel extracted first is within a predetermined range, it is determined as a pupil region. When the density value of the pixel changes greatly beyond the predetermined range after reaching the area just outside the pupil area (area corresponding to the iris I), it is determined that the area has reached the outside of the boundary of the pupil area. . Then, a value (for example, an intermediate value of the respective density values) between the density value of the pixel in the pupil region and the density value of the pixel when reaching the outside of the boundary is selected as the threshold value B. Then, binarization is performed using such a threshold B, and pixels constituting the pupil region are extracted.

  Here, when extracting the pixels constituting the pupil region based on the threshold value, only one continuous region composed of a plurality of pixels having connectivity with the pixels extracted by the selection operation. Select to extract. For example, the captured image is scanned starting from the pixel extracted by the selection operation, and the pixels exceeding the threshold are extracted and labeled as pixels constituting the pupil region. At this time, the pixel extracted first by the selection operation is used. Only pixels having connectivity with the pixels are extracted. As a result, when extracting the pupil region, a portion having a low density value other than the pupil region (for example, mole, eyelash, spot, etc.) is not extracted.

  In deriving the pupil center position from the pupil region thus extracted, a known appropriate method can be adopted. For example, a circle inscribed in the region of the extracted pupil P portion is derived and this The center of the circle can be derived as the pupil center position, or the centroid position of the extracted pupil P portion region can be derived, and the centroid position can be derived as the pupil center position. In this case, the pupil center position can be quickly and easily derived even if the extracted pupil area is not a perfect circle or the outline of the pupil area is wobbled.

  After the pupil center position is derived as described above, the pupil center position can also be derived repeatedly in succession. In this way, when the eyeball E moves, the movement of the eyeball E can be measured by monitoring the displacement of the center position of the pupil, and can be used for nystagmus examination.

  When the derivation of the pupil center position is continuously repeated, the above processing can be repeatedly performed.

  In addition, when the extraction of the pupil region and the derivation of the pupil center position are performed as described above, the threshold used for extraction of the pupil region, the shape of the pupil region, and the positional relationship of the pupil center position with respect to the pupil region are stored. It is also possible to derive the pupil center position for the second and subsequent times by pattern matching processing using these.

  That is, for example, after the derivation of the first pupil center position is completed, after a predetermined time has elapsed, the image of the eyeball E is first imaged by the imaging unit 1, and the obtained captured image is binarized using the threshold value. . At this time, the pupil region is extracted by the binarization process, but a portion having a low density value other than the pupil region (for example, a mole, an eyelash, a stain, etc.) may be extracted at the same time.

  Next, only the pupil region is extracted from the region extracted by the above binarization processing by performing pattern matching processing based on the pupil region shape at the time of deriving the first pupil center position stored in advance. To do. Thereby, even when regions other than the pupil region (for example, moles, eyelashes, spots, etc.) have been extracted during the binarization process, it is possible to extract only the pupil region by removing them. If partial matching is allowed during pattern matching, the pupil region can be extracted even when a part of the pupil P is hidden behind the eyelid.

  Next, based on the positional relationship of the pupil center position with respect to the pupil region at the time of deriving the first pupil center position, the pupil center position is derived from the pupil region extracted by the pattern matching.

  By repeatedly performing such processing, it is possible to detect a temporal change in the pupil center position based on the eye movement, and thereby the movement of the eye E can be measured.

  When performing nystagmus examination, rhythm fluctuation analysis is performed from the measured time series data of the movement of the eyeball E, and the result is displayed on a printing device such as a printer, or a display device 14 such as a CRT display or liquid crystal display. Can be displayed. In addition, it may be estimated whether the nystagmus is an abnormality caused by a peripheral disease of the subject's body or an abnormality caused by a central disease, and the result may be displayed in the same manner as described above.

  The pupil detection device according to the present invention can be suitably applied to nystagmus examination as described above, but is not limited thereto.

It is a schematic block diagram which shows an example of embodiment of this invention. It is a perspective view which shows the external appearance of a measuring apparatus same as the above. It is a front view which shows the external appearance of a control apparatus same as the above. (A) is a front view which shows an example of a mode that the captured image is displayed on a display part, (b) is a graph which shows the change of the density value of the pixel along the horizontal scanning line in the captured image of (a). .

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging part 2 Display part 3 Computation control part E Eyeball P Pupil

Claims (5)

  An imaging unit that images the eyeball, a display unit that displays the captured image, a selection unit that selects an arbitrary position in the captured image displayed on the display unit, information on the captured image, and position information on a selection position by the selection unit And a calculation control unit that extracts only the region corresponding to the pupil from the information of the captured image when the selected position corresponds to the pupil in the captured image.   The arithmetic control unit determines a position where the density value of the pixel connected to the pixel corresponding to the selection position by the selection unit in the captured image changes beyond a predetermined range as the boundary of the region corresponding to the pupil, and this boundary The pupil detection device according to claim 1, wherein the inside is extracted as a region corresponding to the pupil.   The pupil detection device according to claim 1, wherein the arithmetic control unit derives a center position of an area corresponding to the extracted pupil.   The arithmetic control unit derives a circle inscribed in an area corresponding to the extracted pupil, and derives a center position of the circle as a center position of the area corresponding to the pupil. Item 4. The pupil detection device according to Item 3. The calculation control unit derives the center of gravity of the region corresponding to the extracted pupil, and derives the center of gravity of the circle as the center position of the region corresponding to the pupil. The pupil detection device described.

Images