KR20220025199A - Fixtureless lensmeter system - Google Patents

Abstract

The lensmeter system may include a mobile device with a camera. The camera may capture a first image of the pattern through the lens separated from the camera while the lens is in contact with the pattern. The mobile device may determine the size of the lens based on the first image of the pattern and the known features. The camera may capture a second image of the pattern while the lens is in an intermediate position between the camera and the pattern. The second image can be transformed into an ideal coordinate system and processed to determine the distortion of the pattern due to the lens. The mobile device may measure a characteristic of the lens based on the distortion. The properties of the lens may include spherical power, astigmatism power, and/or astigmatism angle.

Description

FIXTURELESS LENSMETER SYSTEM

The field of technology relates generally to determining the prescription of corrective lenses, and more particularly, in one aspect, to mobile device lensmeters and methods of operating such lensmeters.

Ophthalmologists, spectacle lens manufacturers, and others who work with lenses often use a conventional lensmeter to determine the prescription of unknown corrective lenses (including spherical power, astigmatism power, and astigmatism axis). Such lensmeters typically include shining a light source through a pattern, illuminating a corrective lens mounted to a fixture of the lensmeter, and viewing the light in an eyepiece opposite the light source. Observing the distorted shape of the pattern through the eyepiece can correlate the distortion with prescriptions known to produce these distortions.

The fixture holds the pattern, corrective lens, and eyepiece to one another at appropriate spacing and configuration. However, fixtures are typically large and heavy, which makes such devices inconvenient and undesirable for home or field use. This conventional method of determining a prescription for corrective lenses does not provide a convenient way to communicate prescription information to others, such as an ophthalmologist or lens manufacturer. For example, information may be communicated over the phone, but the risk of copy errors or other issues increases, making it less attractive for an individual to decide to prescribe corrective lenses in a convenient environment, such as at home or at work. Therefore, those who decide to prescribe unknown corrective lenses will have to visit an ophthalmologist or other specialist, adding additional delays and costs to the process.

According to one aspect, the process of determining the characteristic of the lens comprises obtaining a captured image of the pattern through a corrective lens, transforming the captured image into an ideal coordinate system, and the entire process from the reference pattern to the pattern in the captured image. processing the captured image to determine distortion, determining distortion of the captured pattern due to the corrective lens, and measuring at least one characteristic of the corrective lens. According to one embodiment, the captured image comprises a pattern and comprises a first area generated by light passing through the corrective lens and a second area produced by light that does not pass through the corrective lens, wherein the captured image is attributable to the corrective lens Determining the distortion of one captured pattern is performed at least in part by reference to the second region of the corrective lens. According to a further embodiment, the pattern is a checkerboard pattern and the second region comprises a border. According to another embodiment, transforming the captured image into an ideal coordinate system includes detecting a plurality of captured reference landmarks in a second region of the captured image, and a plurality of captured reference landmarks from the plurality of ideal reference landmarks. It involves determining a transform into a macro, and applying the transform to the captured image.

According to another embodiment, the pattern is a first pattern, the corrective lens is a first corrective lens, and obtaining a captured image of the pattern through the corrective lens is through the first corrective lens and the first pattern through the second lens. and acquiring a captured image of the second pattern.

According to another embodiment, processing the captured image to determine an overall distortion from a reference pattern to a pattern in the captured image comprises detecting a plurality of captured pattern landmarks in the captured image, and a plurality of ideal pattern lands and determining a transformation from the mark to the plurality of captured pattern landmarks, and, for the corrective lens, determining, from the transformation, a spherical power measurement, an astigmatism power measurement, and an astigmatism angle measurement. According to a further embodiment, the transform is a refractive power matrix.

According to another embodiment, acquiring the captured image of the at least one pattern with respect to the corrective lens is performed at a first position of the camera lens with respect to the at least one pattern, wherein the second position of the camera lens with respect to the at least one pattern is performed. At the location, capturing a second captured image of the at least one pattern through the corrective lens, detecting, in the second captured image, a plurality of captured pattern landmarks; determining a second transformation into the captured pattern landmark; determining, for the corrective lens, from the second transformation, a spherical power measurement, an astigmatism power measurement, and an astigmatism angle measurement; a spherical power among the first transformation and the second transformation The measurement and the measurement of the astigmatism power further comprise selecting a desired transformation having an extreme value.

According to another embodiment, the captured image is captured by a camera having a camera lens, and the corrective lens is placed in a known position relative to the camera lens and pattern. According to a further embodiment, determining the distortion of the captured image due to the corrective lens comprises determining the distance between the camera lens and the pattern and at least one of the corrective lens with reference to the distance, spherical power measurement and astigmatism power measurement. Determining the focal length.

According to one embodiment, measuring at least one characteristic of the corrective lens comprises determining a prescription for the corrective lens, the prescription comprising at least a spherical power value, an astigmatism value and an astigmatism axis value. According to another embodiment, acquiring the captured image of the pattern through the corrective lens comprises obtaining the captured image of the first pattern through the first corrective lens and the second pattern through the second corrective lens through the camera lens. wherein the two patterns pass through the first corrective lens and through the first corrective lens and the second corrective lens when the first corrective lens and the second corrective lens are disposed in known positions relative to the camera lens and the first and second patterns. spaced apart from each other to obtain a captured image of the second pattern.

According to another embodiment, the process comprises, from the captured image, determining a first position of a camera lens of a lensmeter from which the captured image was captured, identifying an orientation with respect to a second position relative to the first position; guiding a user of the lensmeter to a second position, and capturing a second captured image of the pattern through the corrective lens.

According to another aspect, the lensmeter is connected to the camera, the visual display and the camera, and through a corrective lens, acquires a captured image of the pattern, converts the captured image into an ideal coordinate system, from a reference pattern to the pattern of the captured image. and a processor configured to process the captured image to determine an overall distortion of the corrective lens, determine a distortion in the captured pattern due to the corrective lens, and measure at least one characteristic of the corrective lens.

According to one embodiment, the captured image includes a pattern and includes a first area generated by light passing through the corrective lens and a second area generated by light not passing through the corrective lens. According to a further embodiment, the processor detects a plurality of captured reference landmarks in the second region of the captured image, determines transformations from the plurality of ideal reference landmarks to the plurality of captured reference landmarks, the captured image and further configured to transform the captured image into an ideal coordinate system by being configured to apply a transform to .

According to another embodiment, the processor detects a plurality of captured pattern landmarks in the captured image, determines transformations from the plurality of ideal pattern landmarks to the plurality of captured pattern landmarks, and, for a corrective lens, from the transformations , further configured to process the captured image to determine an overall distortion from the reference pattern to the pattern of the captured image by being configured to determine a spherical power measurement, an astigmatism power measurement, and an astigmatism angle measurement. According to a further embodiment, the processor is further configured to acquire a captured image of the at least one pattern through the corrective lens at a first position, wherein the processor is further configured to obtain a second capture of the at least one pattern through the corrective lens at a second position captured image, detect in the second captured image a plurality of captured pattern landmarks, determine a second transformation from the plurality of ideal pattern landmarks to a plurality of captured pattern landmarks, for a corrective lens , determine, from the second transformation, the spherical power measurement, the astigmatism power measurement, and the astigmatism angle measurement, and select a preferred transformation in which the spherical power measurement and the astigmatism power measurement have extrema among the first transformation and the second transformation. According to another embodiment, the captured image is captured through a camera lens of the camera, and the processor determines a distance between the camera lens and the pattern, and refers to the distance, the spherical power measurement and the astigmatic power measurement at least one of the corrective lenses. and determine a distortion of the captured image due to the corrective lens by being configured to determine a focal length of

According to an embodiment, the processor is further configured to determine a prescription of the corrective lens, thereby measuring at least one characteristic of the corrective lens, the prescription comprising at least a spherical power value, an astigmatism value and an astigmatism axis value. . According to another embodiment, the pattern is printed on a physical medium. According to another embodiment, the pattern is displayed on an electronic display device.

According to some aspects, a method of operating a lensmeter system is provided, the method comprising using a camera of the lensmeter system to capture a first image of a pattern through a corrective lens in contact with the pattern. The method also includes, using a computing device of the lensmeter system, determining a size of the corrective lens based on the first image and the pattern. The method also includes capturing, using the camera, a second image of the pattern through the corrective lens while the corrective lens is in an intermediate position between the camera and the pattern. The method also includes, using the determined size of the corrective lens, determining, using the computing device, a distortion due to the corrective lens of the pattern in the second image. The method also includes measuring, using the computing device, at least one characteristic of the corrective lens based on the determined distortion.

According to another aspect, there is provided a mobile device comprising a camera and a processor configured to obtain, from the camera, a first image of a pattern through a corrective lens in contact with the pattern. The processor is further configured to determine a size of the corrective lens based on the first image and the pattern. The processor is further configured to obtain, from the camera, a second image of the pattern through the corrective lens at an intermediate position between the camera and the pattern. The processor is further configured to use the determined size of the corrective lens to determine a distortion due to the corrective lens of the pattern in the second image. The processor is further configured to measure at least one characteristic of the corrective lens based on the determined distortion.

According to another aspect, a lensmeter system is provided that includes a pattern having sized features and a mobile device. The mobile device includes a camera, a memory that stores information associated with the size of the pattern and features, and a processor configured to capture, using the camera, a first image of the pattern through a corrective lens in contact with the pattern. The processor is further configured to determine a size of the corrective lens based on the first image and the pattern using the information associated with the size of the pattern and the feature. The processor is further configured to capture, using the camera, a second image of the pattern through the corrective lens while the corrective lens is in an intermediate position between the camera and the pattern. The processor is further configured to use the determined size of the corrective lens to determine a distortion due to the corrective lens of the pattern in the second image. The processor is further configured to measure at least one characteristic of the corrective lens based on the determined distortion.

Still other aspects, embodiments, and advantages of these exemplary aspects and embodiments are discussed in detail below. Moreover, it is to be understood that both the foregoing information and the following detailed description are illustrative examples of various aspects and embodiments only, and are intended to provide an overview or framework for understanding the nature and characteristics of the claimed subject matter. "one embodiment", "an example", "another embodiment", "other example", "some embodiments", "some examples", "other embodiments", "alternative embodiments", "various embodiments" Specific references to examples and embodiments, such as , “one embodiment,” “at least one embodiment,” “this embodiment and other embodiments,” and the like, are not necessarily mutually exclusive, and the specific feature, structure, or characteristic is not necessarily implemented in the practice of the particular feature, structure, or characteristic. It is described in connection with an example or example and is intended to indicate that the embodiment or example may be included in that embodiment or example and other embodiments or examples. The appearances of these terms herein are not necessarily all referring to the same embodiment or example.

Further, in the event of any inconsistency in the usage of terms between this document and the document incorporated herein by reference, the use of the term in the incorporated reference shall supplement the use of the term in this document; First of all. In addition, the accompanying drawings are included to provide illustration and a further understanding of the various aspects and embodiments, and are incorporated in and constitute a part of this specification. The drawings, together with the remainder of the specification, serve to explain the principles and operation of the described and claimed aspects and embodiments.

Embodiments of the present invention are not limited to the details of the structure and arrangement of components described in the following description or shown in the drawings. Embodiments of the present invention may be practiced or practiced in various ways. Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “comprising” or “having”, “comprising”, “accompanying” and variations thereof herein is meant to include the hereinafter listed items and their equivalents, as well as additional items.
Various aspects of at least one embodiment are discussed below with reference to the accompanying drawings, which are not intended to be drawn to scale. The drawings are included to provide illustration and a further understanding of the various aspects and embodiments, and are incorporated in and constitute a part of this specification, but are not intended as definitions of limitation of any particular embodiment. The drawings, together with the remainder of the specification, serve to explain the principles and operation of the described and claimed aspects and embodiments. In the drawings, each identical or nearly identical component shown in the various figures is denoted by the same number. For clarity, not all components are labeled in all drawings. From the drawing:
1 is a diagram of a prior art lensmeter.
2 is a block diagram of a lensmeter system in accordance with one or more embodiments.
3 is a block diagram of a mobile device lensmeter in accordance with one or more embodiments.
4 is a flow diagram of a method for operating a mobile device lensmeter in accordance with one or more embodiments.
5A is an illustration of a reference pattern group in accordance with one or more embodiments.
5B is an illustration of a captured image of the reference pattern group of FIG. 5A in accordance with one or more embodiments.
Fig. 5c is the captured image of Fig. 5b after transformation to the ideal coordinate system.
6 depicts a number of pattern landmarks for a reference pattern group and a pattern in a captured image, in accordance with one or more embodiments.
7 shows a perspective view of a fixed lensmeter system in a first configuration in accordance with various aspects of the present invention.
8 shows a top perspective view of a fixed lensmeter system in a second configuration in accordance with various aspects of the present invention.
9 illustrates a mobile device user-interface view during fixed lensmeter operation in accordance with various aspects of the present invention.
10 is a flow diagram of exemplary operations that may be performed for fixed lensmeter operation in accordance with various aspects of the present invention.
11 shows an exemplary architecture for a fixed lensmeter system suitable for practicing some implementations of the present invention.

In accordance with one or more embodiments, the disclosed processes and systems allow a lensmeter device, such as a cell phone, to determine a characteristic, such as a prescription for one or more corrective lenses. In some embodiments, an image of one or more patterns is captured by a camera device through a corrective lens, and distortion of the pattern is measured to determine a characteristic of the corrective lens by a computing device connected as specialized software. Embodiments discussed herein describe a lensmeter as a device configured to measure the properties of one or more corrective lenses without requiring specific spacing and placement required by known lensmeters and implemented by fixtures into which they incorporate. . The present lensmeter may be a smartphone or tablet device with specialized software (eg, an app) installed to perform the claimed method. In some operating scenarios, all processing to determine the properties of the corrective lens is performed on a smartphone or tablet (eg, by the smartphone or tablet's processor(s)). In other operating scenarios, data such as camera images or image metadata may be communicated between a smartphone or tablet and one or more remote processors such as one or more cloud servers that perform some or all of the processing to determine characteristics of corrective lenses. there is. In such other operating scenarios, information indicative of the determined characteristic may be communicated from the remote processor(s) to a smartphone, tablet and/or other device or system. In some implementations, the lensmeter has a fixed location (eg, a camera embedded in a wall or fixture and communicatively coupled to one or more local and/or remote processors) such that the corrective lens and pattern are precisely spaced and positioned relative to the lensmeter. It may include one or more components capable of measuring the properties of a corrective lens without need. Such an arrangement may be suitable, for example, in a retail environment such as an optician or an eyeglass retailer.

The pattern may be displayed on paper or displayed on the display of another device such as a laptop computer. In some embodiments, a mobile device (ie, a mobile lensmeter) and another device (eg, another device displaying a pattern) may be paired to allow the devices to communicate and interact during the measurement process. The examples herein depicting a mobile lensmeter as a mobile device itself are for illustrative purposes only, and the functionality discussed herein with respect to a “mobile lensmeter” may be implemented on or in conjunction with other such devices as part of a mobile lensmeter system. It will be understood that this can be done.

In some embodiments, the two patterns are configured spaced apart so that the corrective lenses are visible to the mobile lensmeter - each through one of the pair of corrective lenses of the frame - when the corrective lenses are positioned approximately halfway between the pattern and the lensmeter and properly oriented. This arrangement allows for easy and intuitive positioning of mobile lensmeters, patterns and corrective lenses. Further, the mobile lensmeter is configured to determine the distance to the pattern and take that measurement into account when determining the prescription. This design facilitates manual positioning of the elements, eliminating the need for fixtures. In one embodiment, the pattern is a rectangle displayed on a physical medium or computer display. In some embodiments, the pattern is surrounded by a boundary having reference landmarks or other features used to orient the captured image.

In accordance with one or more embodiments, the disclosed processes and systems transform captured images into an ideal coordinate system to compensate for orientation of a lensmeter with respect to a pattern during the image capture process. In some embodiments, the transformation is made by referencing the location of the reference landmark in the captured image to the location of the reference landmark in the reference pattern group.

In accordance with one or more embodiments, the disclosed processes and systems process the captured image to determine overall distortion by detecting and determining the location of a plurality of captured pattern landmarks in the captured image. The system determines a transformation describing the distortion from the positions of a number of reference pattern landmarks (in an ideal coordinate system) with respect to corresponding captured pattern landmarks in the captured image. The representation of the transformation (eg, the power matrix) may be used to determine measurements of the corrective lens including spherical power, astigmatism power, and astigmatism angle. The portion of total distortion due to the corrective lens (opposite the lens of the lens meter) may be determined in part by determining the focal length of at least one of the corrective lenses. Other properties of corrective lenses may also be measured. This embodiment is not limited to a spherical column lens, but may be suitable for a lens having other characteristics, such as a monofocal lens, a bifocal lens, a trifocal lens, a progressive lens, an adjustable focus lens, or a lens for correcting higher order aberrations. there is.

According to one or more embodiments, a plurality of images may be captured and analyzed to identify, for example, images captured at a preferred location and/or orientation where the corrective lens is closest halfway between the lensmeter and the pattern.

In accordance with one or more embodiments, a lensmeter is provided that includes a camera, a visual display, and a processor configured to perform the processes described herein. The lensmeter may be a dedicated lensmeter or it may be a mobile device (eg, a smartphone or tablet device) running lensmeter software such as a downloadable app.

1 shows a conventional optical lensmeter system 100 for determining prescription and/or other unknown properties of a corrective lens 130 . The light source 110 is ocular through a pattern 120 (eg, a transparent target having a printed pattern having known dimensions and arrangements) and a corrective lens 130 (and a number of standard and objective lenses, not shown). It is directed to the lens 140 . A viewer whose eyes engage eyepiece 140 can observe how corrective lens 130 distorts light passing through pattern 120 . By measuring the distortion effect of the corrective lens 130 , the user can determine certain characteristics of the corrective lens 130 , including measurements of the spherical power, the astigmatism power, and the astigmatism axis of the corrective lens 130 . The lensmeter system 100 requires a fixture 150 that includes a lens holder 152 to hold the pattern 120, corrective lens 130, and eyepiece 140 in a precisely spaced and oriented arrangement. . The optical principles underlying the operation of the lensmeter system 100 require that a certain spacing and orientation be maintained by the fixture 150 .

Similarly, a digital lensmeter may be used to image a pattern through a single corrective lens, and the distortion of the image may be used to determine the prescription and/or other unknown properties of the corrective lens. As with conventional optical lensmeters, currently available digital lensmeters require corrective lenses, lensmeter lenses, and fixtures to hold the pattern in a precisely spaced and oriented arrangement.

2 illustrates a block diagram of a lensmeter system 200 in accordance with one or more embodiments. In the embodiment shown in FIG. 2 , the system 200 includes a lensmeter 210 , a corrective lens 220 and a pattern 230 . In operation, lensmeter 210 captures an image of pattern 230 through corrective lens 220 . The corrective lens 220 distorts the light emitted or reflected by the pattern 230 with the lens meter 210, and the distortion effect of the corrective lens 220 includes measurements of spherical power, astigmatism power and astigmatism axis. One or more unknown properties may be measured to determine.

The captured image of pattern 230 is normalized by transforming it into an ideal coordinate system using reference landmarks near pattern 230 . Normalization compensates for changes in rotation, tilt, or distance in spacing and orientation between lensmeter 210 , corrective lens 220 , and pattern 230 . Therefore, the lens meter system 200 does not require a fixture. The normalized pattern 230 can be compared with a reference pattern even in an ideal coordinate system, and the distortion effect of the corrective lens can be distinguished from the distortion effect of the lens of the lens meter 210 itself.

In some embodiments, pattern 230 is displayed on an electronic display (not shown), such as a computer monitor, tablet, or other mobile device, or projected onto a surface by a projector. For example, the pattern 230 may be provided on a website accessible by the lensmeter system 200 , or may be provided by or through a mobile app running on a mobile device. In another embodiment, the pattern 230 is printed on a physical medium such as paper or plastic.

In some embodiments, more than one pattern may be used to simultaneously determine the properties of two or more corrective lenses. In the preferred embodiment, two spaced-apart patterns are used, each pattern 230 being a rotation-transforming checkerboard grid of alternating black and white squares, wherein the number of rows in the checker differs by one from the number of columns. This rotation-strain quality allows the lensmeter 210 to determine whether the pattern 230 is being viewed in the correct upright position or rotated alternately to its side. In one embodiment, pattern 230 is a black and white checkerboard design with 8 rows and 7 columns. In another embodiment, pattern 230 has 16 rows and 15 columns. Other configurations or color combinations are possible.

3 is a block diagram of a lensmeter 210 in accordance with some embodiments. In some embodiments, the lensmeter 210 is a consumer mobile device (eg, a smartphone or tablet device) or computer (eg, a laptop computer) running specialized software to perform the operations described herein. am. In another embodiment, lensmeter 210 is a dedicated lensmeter device. The lensmeter 210 includes a camera 310 having a lens 312 and further includes a processor 320 , a user interface 330 , a network interface 340 , a memory 350 and lensmeter software 360 . include In some embodiments, camera 310 is an integral component of lensmeter 210 . In other embodiments, camera 310 may be an additional component or accessory. Processor 320 is coupled to camera 310 and executes software 360 for image capture functions performed by lensmeter 210 . In some embodiments, the functions of lensmeter 210 may be performed with other devices as part of a broader lensmeter system. For example, in embodiments where the functions of lensmeter 210 are performed by the user's smartphone, the smartphone may be paired with the user's laptop computer to control the display of the pattern. In that example, lensmeter 210 may be considered to include both a user's smartphone and a laptop computer.

User interface 330 receives input from a user of lensmeter 210 and provides output to that user. In some embodiments, user interface 330 displays to the user an image currently viewed through lens 312 of camera 310 , allowing the user to adjust the position or orientation of lensmeter 210 . In some embodiments, user interface 330 provides the user with a physical or on-screen button to interact with to capture an image. In another embodiment, the image is automatically captured when a pattern 230 is detected in the image and certain alignment, size, lighting and resolution conditions are met.

User interface 330 also provides an indication to the user to move any of lensmeter 210 , corrective lens 220 , and pattern 230 to different absolute positions or orientations, or to different positions or orientations relative to each other. can do. For example, the user interface 330 may display the lens meter 210 until the user positions the lens meter 210 such that the corrective lens 220 is positioned in the best known position relative to the lens meter 210 and pattern 230 . "Move forward", "Move backward" until lensmeter 210, corrective lens 220, and pattern 230 are aligned so that pattern 230 can be viewed through corrective lens 220 at 210. , instructions such as "tilting the lensmeter forward", instructions conveyed by way of graphics and illustrations, or other such instructions. In some embodiments, the user interface 330 and/or other components of the lensmeter 210 are proportional (or inversely proportional) to the distance of the lensmeter 210 from a precise location or by, for example, a recorded voice command. It is possible to provide such a command to be audible by an audible tone emitted at a frequency. In another embodiment, the user interface 330 may display a "green light" or thumbs-up icon when the user correctly positions the lensmeter 210, corrective lens 220, and pattern 230, for example. icon) to provide an indication to the user. User interface 330 may also enable the user to interact with other systems or components by providing commands to the user's physician to send corrective lens prescription information.

In some embodiments, network interface 340 allows access to downloads and upgrades of software 360 . In some embodiments, one or more steps of the process described below may be performed on a server (not shown) or other component separate from lensmeter 210 , via network interface 340 , lensmeter 210 . ) and data can be passed between the server. Network interface 340 may further allow for automatically uploading lens characteristics or prescription information to another entity, eg, the user's optometrist or other corrective lens provider.

Some or all of the processes described herein, as well as other functions, may be performed by lensmeter software 360 running on lensmeter 210 or other system in communication with lensmeter 210 (e.g., a network interface ( 340)).

4 is a flow diagram of a process 400 for determining a characteristic of a corrective lens in accordance with one or more embodiments. Such an embodiment may be implemented using a system such as that shown in FIGS. 2 and 3 .

The process begins at step 410 .

In step 420, a captured image of the pattern is acquired through a corrective lens. The image is captured by a camera (eg, camera 310 ). In some embodiments, the camera is part of or attached to a dedicated lensmeter device. In another embodiment, the camera is part of a mobile device (eg, a smartphone or tablet device). In some embodiments, the user is instructed to hold the mobile device with the camera oriented towards the pattern so that the pattern is viewed through the corrective lens. Then the pattern image is captured by the camera. An image may be captured in response to a user indication, such as clicking a physical button or interface element on a screen of the mobile device. In another embodiment, an image may be captured automatically once a stable, relatively static image is acquired and focused, with the lensmeter, corrective lens and pattern properly aligned. For example, an accelerometer of a mobile device may be used to determine if the camera is relatively stationary. If a focused image can be obtained, the system may attempt to identify the presence of a pattern in the image using known image processing and detection techniques. In some embodiments, a plurality of images may be captured, and an image may be selected from among the plurality for further processing based on criteria such as the image on which the pattern is most focused, whether elements are properly aligned, and the like. .

In some embodiments, the object may be an image displayed on a computer display. The object may be a pattern with known geometry and easily detectable feature points. According to some embodiments, a checkerboard pattern is used.

5A shows a pattern group 500 in which patterns 510 and 520 are arranged to be used to simultaneously detect characteristics of two corrective lenses, such as two spectacle lenses in a frame. Patterns 510 and 520 are located within boundary 530 . Boundary 530 includes boundary reference landmarks 532 , 533 , 534 , 535 at known locations relative to boundary 530 . Boundary 530 and/or boundary reference landmarks 532 , 533 , 534 , 535 are used in subsequent steps to correct the orientation of the captured image. In a preferred embodiment, four boundary reference landmarks are used, although some embodiments may use as few as two boundary reference landmarks. In one embodiment, boundary reference landmarks 532 , 533 , 534 , 535 are disposed at each of the four inner corners of boundary 530 . Boundary reference landmarks 532 , 533 , 534 , 535 may be recognizable markers in images captured using computer vision techniques or by computer vision techniques and/or by reference to known geometries of pattern group 500 . It may be a unique landmark detected. For example, if boundary 530 is known to be a rectangle having four inner corners, then the four inner corners may be placed and used as boundary reference landmarks 532 , 533 , 534 , 535 .

Patterns 510 and 520 also include a plurality of pattern reference landmarks 512 . The positions of the plurality of pattern reference landmarks 512 in the captured image are used in a subsequent step to determine the nature of the distortion introduced by the corrective lens. In some embodiments, pattern reference landmarks 512 are located at adjacent corners of squares within the checkerboard pattern. The pattern reference landmark 512 may be a recognizable marker in an image captured using computer vision techniques, or a land detected by computer vision techniques and/or with reference to the known geometry of the pattern group 500 . could be a mark.

The locations of boundary reference landmarks 532 , 533 and pattern reference landmark 512 are known in pattern group 500 . This known location allows the pattern group 500 to be used as a reference pattern group in a subsequent step, and can be compared to the location of the same point in the captured image.

In some embodiments, the lensmeter is configured to operate when the corrective lenses in the frame are intermediate between the lensmeter and the patterns 510 , 520 . Patterns 510 , 520 are configured and spaced such that each of patterns 510 , 520 aligns with both the lensmeter and one of the corrective lenses when the corrective lens is halfway between the lensmeter and the pattern 510 , 520 . Such an arrangement may be achieved, for example, when the distance between the centers of the patterns 510 and 520 is twice the distance between the centers of the corrective lenses. For example, if the distance between the centers of the corrective lenses in the frame is 77.5 mm, the patterns 510 and 520 may be spaced apart such that the distance between the centers of the patterns 510 and 520 is 77.5×2=155 mm.

The patterns 510 , 520 and/or borders 530 are sized and configured such that when a lensmeter captures an image of the pattern through a corrective lens, the aperture of a normal sized frame completely encloses the pattern in the captured image; , which means that the pattern is completely covered with corrective lenses. The opening of the eyeglass frame is in turn completely contained within the boundary 530 . The captured image may include one or more first areas generated from light passing through (ie, distorted by) the corrective lens and one or more second areas generated from light passing around (ie, not distorted by) the corrective lens. It can be considered as having an area.

A captured image 550 illustrating this configuration can be seen in FIG. 5B . The pattern 510 is completely contained within the opening 542 of the eyeglass frame 540 , and the pattern 520 is completely contained within the opening 544 of the eyeglass frame 540 . Patterns 510 , 520 in captured image 550 were distorted by corrective lenses in frame 540 . The spectacle frame 540 is completely contained within the boundary 530 . By using this configuration, distortion of the patterns 510 , 520 in the captured image due to the corrective lens can be measured while the boundary reference landmarks 532 , 533 remain undistorted.

It will be appreciated that the pattern group 500 shown in FIGS. 5A and 5B is for purposes of illustration, and that different configurations, sizes, or types of patterns and/or borders may be utilized or omitted altogether without departing from the scope of the present invention. In some embodiments, more than one image may be captured, each image being cropped or limited to contain only one pattern. In another embodiment, one image of both patterns is captured, and the image is split into two images for parallel processing for each pattern in a subsequent step. In another embodiment, a video clip of the pattern may be captured, or a plurality of static images may be captured in rapid succession.

It will also be appreciated that lenses with different properties will distort the pattern of the captured image in different ways. For example, a lens with a positive power will magnify the pattern in the captured image, which makes the pattern appear larger through the corrective lens. In such a situation, the pattern may be sized so that the pattern in the captured image is not too large to be completely bounded by the corrective lens. Likewise, a lens with a negative power will reduce the pattern in the captured image, which makes the pattern appear smaller through the corrective lens. In such a situation, the pattern may be sized such that the pattern in the captured image is not too small to be identified and processed at a later stage. Accordingly, in some embodiments the pattern may be displayed on a display device such that the pattern size may be configured according to the characteristics of the lens or other considerations. In some embodiments, the user may be provided with an interface (via the display device or lensmeter 210 ) to adjust the size of the pattern or select a characteristic of the lens and have the appropriately sized pattern displayed. In other embodiments, the pattern may be automatically resized by the system to be the correct size in the captured image.

As can be seen in captured image 550 of FIG. 5B , boundary 530 is rotated counterclockwise from the horizontal rectangular configuration shown in FIG. 5A , and patterns 510 , 520 and boundary 530 of FIG. 5B are rotated counterclockwise. ) is smaller than the counterpart in Fig. 5a. This deformation makes it difficult to directly compare the pattern group 500 of the captured image 550 with the reference pattern group 500 .

Accordingly, returning to Figure 4, in step 430, the captured image is transformed into an ideal coordinate system. In some embodiments, the captured image is transformed into an ideal coordinate system represented by reference pattern group 500 of FIG. 5A . This transformation may include rotating, resizing, cropping and skewing the image to remove any distortion or inaccuracy introduced due to the image capture process. In some embodiments, the captured image detects boundary reference landmarks 532', 533', 534', 535' in the captured image 550 and manipulates the captured image 550 using image manipulation techniques. By transforming the boundary reference landmarks 532', 533', 534', 535' into the corresponding boundary reference landmarks 532, 533, 534, 535 in the reference pattern group 500 of FIG. 5A in the captured image, It is transformed into an ideal coordinate system by making it appear at the same location. Boundary reference landmarks 532', 533', 534', 535' may be detected by computer vision techniques, and boundary reference landmarks 532', 533', 534', 535' or pixels constituting them may be configured to have a shape, color, or other characteristic suitable for performing these computer vision techniques.

In some embodiments, the matrix transform may include boundary reference landmarks 532 , 533 , 534 , 535 within captured image 550 and corresponding boundary reference landmarks 532 , 533 , 532 , 533 within reference pattern group 500 of FIG. 5A , 534 and 535) are determined from the distance between them. A matrix transform is then applied to some or all of the pixels of the captured image 550 to perform the transform.

A captured image 550 that appears after transformation to an ideal coordinate system can be seen in FIG. 5C . Boundary reference landmarks 532', 533', 534', 535' in this transformed captured image 550 are boundary reference landmarks 532, 533, 534, 535 within reference pattern group 500 of FIG. 5A. ) is in the same position as

In step 440, the captured image is processed to determine an overall distortion from the reference pattern to the pattern in the captured image. The overall distortion (ie, the distortion introduced by the corrective lens as well as the lens of the camera used to capture the image) is determined by comparing the patterns 510 , 520 in the captured image 550 with the patterns in the reference pattern group 500 . can be determined by In some embodiments, the comparison is performed by comparing the plurality of pattern reference landmarks 512 ′ in the captured image 550 to the plurality of pattern reference landmarks 512 in the reference pattern group 500 .

6 illustrates the location of a plurality of pattern reference landmarks 512 ′ in an exemplary captured image 550 overlaid over the locations of the plurality of pattern reference landmarks 512 within a reference pattern group 500 . The distance between each pattern reference landmark 512a', 512b' in the captured image and the corresponding reference landmark 512a, 512b in the reference pattern group is a distortion (i.e. transformation) from the ideal coordinate system to the captured image. can be used to determine the refractive power matrix P that describes

Prentice's Rule describes the amount of prism induced in a lens. P can be used to express Prentice's law in matrix form as x _test = Px _ref , where x _test is the matrix of the positions of the pattern reference landmarks 512a', 512b' in the captured image, and x _ref is the reference A matrix of positions of corresponding reference landmarks 512a, 512b within a pattern group.

The refractive power matrix P is given as

[1]

[One]

[2]here,

[2]

[3]

[3]

[4]

[4]

Solving algebraically allows the determination of values for S, which is a value related to the spherical power of the lens, C, a value related to the astigmatism of the lens, and the astigmatism angle θ of the lens.

The values of S and C account for the distortion introduced into the image captured by both the corrective lens and the lens of the lensmeter's camera. Accordingly, in step 450, a distortion of the pattern in the captured image due to the corrective lens is determined. In particular, the focal lengths f _θ and f _θ+90° of the corrective lens along two orthogonal axes corresponding to θ and θ+90° are determined by the following equation when the corrective lens is placed at an intermediate position between the camera and the pattern do.

[5]

[5]

[6]

[6]

where l is the distance between the lens meter's camera lens and the pattern. To determine the value of l , parameters of the camera and/or lens may be determined or accessed directly from a data store. In some embodiments, the focal length f of the camera lens may be determined from metadata within the captured image or from configuration information for the lensmeter. The height h of the pattern is known. Distance l can be determined from f and other parameters. A method and system for determining a distance from an object (eg, a pattern) is described in U.S. Patent Application Serial No. 14/996,917, filed January 15, 2016, entitled “SMARTPHONE RANGE FINDER,” the entire disclosure of which is The content is incorporated herein by reference in its entirety.

In step 460, at least one characteristic of the corrective lens is determined. In some embodiments, spherical power, astigmatism, and astigmatism axis measurements of the corrective lens may be determined, which may result in the prescription of the corrective lens being determined. The spherical power measurement represents the refractive power of a lens measured in diopters. Corrective lenses may prescribe certain spherical power values to correct for nearsightedness or farsightedness in all meridians of the eye. In some embodiments, spherical power values may be coded, with negative values representing myopia prescriptions and positive values representing farsightedness prescriptions.

The astigmatism value represents the prescribed lens power for astigmatism. Astigmatism power value can be zero if no correction is prescribed for astigmatism. The astigmatism measurements indicated that the corrective lens had a first meridian with no added curvature and a second meridian perpendicular to the first meridian and containing the maximum lens curvature for astigmatism correction.

The astigmatism axis value describes the direction of the second meridian of the astigmatism power. The astigmatism axis value may range from 1° to 180°, with 90° corresponding to the vertical meridian of the eye and 180° corresponding to the horizontal meridian.

Other values may also be determined for corrective lenses, such as an ADD value representing an additional magnification applied to the underside of a multifocal (eg, bifocal or trifocal) lens.

In some embodiments, the spherical power, astigmatism power, and astigmatism axis measurement of the corrective lens may be determined by the following equation.

[7]

[7]

[8]

[8]

[9]

[9]

AXIS's decisions are made from the point of view of the wearer of corrective lenses.

The values of SPH, CYL, and AXIS may be displayed on the lensmeter's screen, stored in the lensmeter's memory (eg, in a database or file), and/or affiliated with the corrective lens owner to confirm or create a prescription. It can be communicated through the lensmeter's network interface to another party, such as an ophthalmologist. For example, a person who has glasses but does not know the prescription for those glasses may perform the process. Information obtained through the methods discussed herein may be transmitted to a person's eyewear specialist who can use that information to order a new set of appropriately prescribed eyeglasses.

Process 400 ends at step 470 .

In some embodiments, the requirement that the corrective lens be positioned halfway between the lensmeter and the pattern may be relaxed. The lensmeter and/or corrective lens may instead be moved relative to each other and the pattern, and the lensmeter to capture a plurality of images. For each image, the values of S and C may be determined as discussed above. S and S+C are known to have extremes (ie, minimum or maximum) when the corrective lens is positioned halfway between the lensmeter and the pattern. The image in which S and S+C produce extrema can be used as a basis for the process described herein.

Although the examples provided herein include corrective lenses in the form of spectacles, assuming that contact lenses can be secured in orientations and positions suitable for carrying out the claimed process, the processes and systems may be applied to other types of corrective lenses, such as contact lenses. You will understand that you can.

In some embodiments, the captured image is not transformed into an ideal coordinate system. Rather, two images are captured: a first image with a corrective lens disposed between the lensmeter and the pattern as discussed in the various embodiments herein, and a second image identical to the first image except that the corrective lens has been removed. 2 “Reference” images. Since the distortion effect of the lens is not present in the second image, the first image can be directly compared to the second image to determine the amount of distortion in process 400 using the techniques discussed with respect to step 440 .

As previously discussed (eg, with respect to equations 1-9), it may be helpful to operate a fixed lensmeter such that the corrective lens is positioned halfway between the lensmeter camera and the displayed pattern. However, it can be difficult for the user to find a suitable intermediate location.

As further noted earlier, if the lensmeter and/or corrective lens are moved relative to each other and the pattern, the requirement that the corrective lens be positioned halfway between the lensmeter and the pattern can be relaxed, and the lensmeter is located in S and S Capture a plurality of images during movement for identification of extrema for +C to identify one image in which the corrective lens is in an intermediate position among the plurality of images.

However, capturing, storing, and/or processing multiple images to identify extremes of S and S+C can be computationally expensive (e.g., in terms of CPU cycles, memory usage, and/or power). there is.

According to various aspects of the present invention, the distance l between the lens and the pattern of the camera of the lensmeter and the distance l1 between the pattern and the corrective lens and/or the distance l2 between the corrective lens and the pattern (eg, l - l1 ) may be determined. An improved lens meter system and method are provided. Using one or more of these determined distances, the user may be guided to place the corrective lens in an intermediate position (eg, l1 = l2 ) before the analysis image is captured and/or the lensmeter may cause the lensmeter, corrective lens, and pattern of Relative position can be compensated.

The distances l , l1 , and/or l2 first determine the absolute size and shape of the corrective lens using additional measurement images (sometimes referred to herein as contact images or first images) captured with the corrective lens in contact with the pattern. can be determined by 7-11 illustrate various aspects of fixed lensmeter operation using contact images.

For example, FIG. 7 shows a system 200 in which a lensmeter 210 , two corrective lenses 220 mounted to a frame 708 of eyeglasses 704 and a pattern 230 are positioned for capture of a contact image. shows a perspective view of In the example of FIG. 7 , pattern 230 is displayed in display image 700 on laptop computer 702 . Laptop computer 702 may be paired with lensmeter 210 (implemented as a mobile phone in this example). Lensmeter 210 may obtain display features for display of laptop 702 from laptop 702 , and may issue instructions to control display of image 700 and/or pattern 230 to laptop 702 . can be provided to

For example, the lensmeter 210 may be associated with the display of the laptop 702 , such as dimensional information indicative of the size (eg, height and width) of the display of the laptop 702 and/or the pixel size and density of the display. information can be obtained. Using this information, lensmeter 210 may determine and/or instruct laptop 702 to adjust the absolute physical size of features of pattern 230 . In the example of FIG. 7 , a single screen-wide pattern 230 is displayed with an embedded reference landmark 706 . In this example, reference landmark 706 is a boundary reference landmark (eg, for orientation and transformation of captured images) as described above with respect to boundary reference landmarks 532 , 533 , 534 and 535 . can be used as In this example, pattern reference landmarks, such as pattern reference landmarks 512 , may be provided within pattern 230 and/or features of the pattern (eg, interfacing corners of a pattern square or other shape) may be identified as pattern reference landmarks. It can be used as a landmark.

In the operation shown in FIG. 7 , while corrective lens 220 is in contact with pattern 230 (eg, the display of laptop 702 is in contact with the display while displaying pattern 230 ), the lens meter ( 210 captures contact images of corrective lens 220 , rim 708 and pattern 230 outside of rim 708 through corrective lens 220 . In the illustrated configuration, since corrective lens 220 is in contact with pattern 230 , pattern 230 is substantially free of distortion by corrective lens 220 . Accordingly, the size of the pattern feature of the contact image is known (eg, stored in memory 350 ) and at the location of the edge of corrective lens 220 relative to pattern 230 (eg, frame 708 ). (identified by the portion of pattern 230 occluded by ) can be used to determine the absolute size and shape of each corrective lens.

As shown in FIG. 7 , a command 703 may be provided to the display of the lensmeter 210 . In another implementation, the instructions 703 may be provided in whole or in part on a display of a laptop 702 or other device. 8 shows an example of a command 703 that may be provided during lensmeter operation using lensmeter 210 . As shown in FIG. 8 , instructions 703 provide a representation 800 of a boundary reference landmark 706 , and a representation of the glasses 704 in place relative to the representation 800 of the boundary reference landmark 706 . representation 802 (including representations 804 and 806 of frame 708 and lens 220 ), all based on one or more images captured by a camera of lensmeter 210 . As shown, the text command 801 and/or the graphic command 803 prompt the user to be flat against the pattern 230 (eg, flat against the screen of the laptop 702 ) as in the configuration of FIG. 7 . to instruct to place the glasses 704 and capture an image (eg, take a picture).

As shown in FIG. 8 , a lens boundary indicator 810 indicating an outer boundary of each corrective lens 220 may be displayed. For example, the glasses 704 may be segmented from the pattern 230 displayed in the image captured by the lensmeter 210 , and the shape corresponding to the lens 220 results in the relative size and position of the connected components. can be inferred based on The lens boundary indicator 810 may be generated based on the inferred lens shape, which may then be superimposed on the camera view of the smartphone as shown in FIG. 8 to provide feedback to the user that the lens is correctly positioned.

8 also shows how the lensmeter 210 can determine whether one or both of the lenses 220 are partially obscured (eg, by a user's finger or a temple of the glasses). For example, occlusion may be detected by comparing the lens shape for the two lenses 220 in the spectacles 704 using a shape mirroring and/or alignment procedure. Occlusion may be detected when the mirrored shape of one lens is substantially different (eg, less than a threshold) from the shape of the other lens.

The exact location and shape of the occlusion can be determined by the lensmeter 210 by selecting a point on the occluded lens shape that does not match the unoccluded lens shape. The occlusion boundary 814 defined by this selected point is then superimposed on the camera view of the smartphone to inform the user of the nature and location of the occlusion so that occlusion can be resolved by the user. Additional instructions 821 may be provided to instruct the user to remove the occlusion (if occlusion is detected) or to remind the user not to occlude the lens.

As described above, when a contact image is captured using system 200 in the configuration of FIG. 7 in which corrective lens 220 is flat against pattern 230 , a boundary (indicated in FIG. 8 by boundary indicator 810 ) ) may be compared to the known size of the pattern features of the pattern 230 to determine the absolute size and shape of each corrective lens 220 .

After capturing the contact image (eg, in response to a user action or automatically as described above), ask the user to move the corrective lens 220 to a position approximately intermediate between the lensmeter 210 and the pattern 230 . Instruction 703 may be modified to instruct to do so.

9 shows a top perspective view of the system 200 in a configuration in which the user has moved the glasses 704 such that the corrective lenses 220 are positioned at an intermediate position 900 between the lensmeter 210 and the pattern 230 . . 9 shows the distance l1 between the pattern 230 and the corrective lens 220 , the distance l2 between the corrective lens 220 and the pattern 230 and the total distance l between the camera and the pattern.

Images captured during movement and positioning of the corrective lens 220 may be processed in real time to identify the shape and size of the lens 220, and compared to the shape and size of the contact image to determine the distances l1 and l2 there is. One or more of these determined distances and/or graphical indicators of absolute and/or relative distances are reported back to the user in real time (eg, using the display of the lensmeter 210 ) so that the user can put the glasses on the pattern 230 . ) and a desired intermediate position 900 , such as a center position where the distance l1 between the corrective lens 220 and the distance l2 between the corrective lens 220 and the pattern 230 is substantially equal (eg, to within a difference threshold). ) can help guide you.

Lens boundary indicator 810 and/or occlusion indicator 814 may also be displayed by lensmeter 210 in real time during movement and positioning of corrective lens 220 to position the corrective lens relative to reference landmark 706 . can be displayed to the user and notify the user if one or more lenses are obscured.

If it is determined from processing of the contact image that distances l1 and l2 are approximately equal (eg, to within a difference threshold) based on the known size of lens 220 (eg, no occlusion is detected), the lens Meter 210 may provide an indication that the user has correctly positioned lensmeter 210 , corrective lens 220 and pattern 230 , for example by displaying a “green light” or thumb up icon. When the desired positioning of the corrective lens 220 has been achieved (eg, in an intermediate position), the user is provided with instructions to capture a lensmeter image (eg, a lens dioptric image) or the lensmeter image is automatically can be captured as The properties of the lens 220 are measured (eg, as described above with respect to FIG. 3 ) using captured lensmeter images.

In this way, the information obtained during the positioning of the lensmeter 210 and the lens 220 can be computed and powered in real time (e.g., compared to iteratively calculating and comparing S and S+C as described above). reduced) to the user to help the user interactively adjust the placement of the glasses and smartphone to achieve the desired imaging conditions. This information (eg, relative distance) may be displayed to the user after capturing the lensmeter image. The lensmeter may then provide the option to retake or accept the lensmeter image in some implementations.

7-9 also show a common pattern across the display of laptop 702 , however, in various scenarios, multiple patterns (eg, patterns 510 and 520 for two lenses of glasses) It should also be noted that two patterns or more than two patterns for progressive or multifocal lenses) may be used. However, the single pattern shown and described in connection with FIGS. 7-9 may also be used to determine various characteristics of a multifocal, progressive, or other complex lens arrangement (eg, sub-region analysis of a captured image). Although single contact images and single lensmeter images are described with respect to FIGS. 7-9, in some situations, multiple contact images and/or multiple lensmeter images may be captured and aggregated, averaged or It should also be noted that they may be combined and/or co-processed. For example, a plurality of images may be stacked or combined for analysis, and/or measurement results from the plurality of images may be averaged or combined. For example, the lens power estimation operation described herein is affected by several quantities of measurement accuracy, such as the distortion of the test pattern of the captured image, the position and orientation of the pattern and lens relative to the lensmeter device. These measurements can be inaccurate, being partially uncorrelated from one moment to the next. By acquiring a series of similar images and aggregating the derived estimated measurements, the effects of temporally uncorrelated noise sources can be mitigated.

10 depicts a flow diagram of an exemplary process for lensmeter operation in accordance with various aspects of the present technology. For purposes of illustration, the exemplary process of FIG. 10 is described herein with reference to the components of FIGS. 2 , 3 and 7-9 . Also for illustrative purposes, some blocks of the exemplary process of FIG. 10 are described herein as successively or continuously occurring. However, multiple blocks of the exemplary process of FIG. 10 may occur concurrently. Further, the blocks of the example process of FIG. 10 need not be performed in the order shown and/or one or more of the blocks of the example process of FIG. 10 need not be performed.

In the illustrated exemplary flow diagram, at block 1000 , a pattern such as pattern 230 is provided. As noted above, the pattern may be a printed pattern or a lens such as a lensmeter device 210 implemented by a computing device such as a laptop 702 (eg, by the computing device itself and/or as a mobile phone). in cooperation with the meter device).

At block 1002 , instructions are given to place the corrective lens in contact (eg, flat contact) with a pattern (eg, see FIGS. 7 and 9 ) (eg, using a display of a lensmeter device) provided Although the example of FIG. 10 is described with respect to a single corrective lens, the operation of FIG. 10 may be performed with more than one corrective lens, such as a pair of frame mounted corrective lenses.

At block 1004 , a first image (eg, a contact image) of the pattern is acquired (eg, by a camera of a lensmeter) through a corrective lens in contact with the pattern. The first image may be obtained by capturing the image automatically or in response to an action of a user using a lensmeter device as described herein.

At block 1006, a lensmeter (eg, segmenting a corrective lens and/or a rim for a corrective lens from the pattern in the first image and identifying locations of connected components within the segmented image to a point corresponding to an outer boundary) ) to identify the outer boundary of the corrective lens using the first image. Boundary indicators, such as boundary indicators 810 of FIG. 8, may be displayed as representations of corrective lenses as described above.

At block 1008, the lensmeter determines whether the corrective lens is occluded. Determining whether a corrective lens is obscured may include identifying an irregular edge of a single corrective lens or identifying a shape difference between the mirrored shapes of two lenses of the spectacles (see, e.g., FIG. 8 and related discussion). ).

If it is determined that the corrective lens is occluded, the lensmeter may display at block 1010 a first image having an occlusion indicator, such as occlusion indicator 814 of FIG. 8 associated with representation 812 of the user's finger occluding the lens. . While occlusion is detected, the lensmeter may continuously return to blocks 1004 , 1006 , 1008 and 1010 until the occlusion is removed.

If it is determined that the corrective lens is not occluded, then at block 1012 the first image and the identified boundary may be stored (eg, in a memory such as memory 350 of the lensmeter).

At block 1014, the size and shape of the corrective lens is determined based on the identified boundaries and patterns. For example, the pattern may be provided with features having a known shape and absolute size, such that the absolute dimensions of the corrective lens may be determined compared to the features of the pattern in the contact image. For example, if the pattern comprises a checkerboard pattern of squares each X mm wide and the largest dimension of the corrective lens spans Y squares, then the largest dimension of the corrective lens has a length of X*Y mm. One or more additional dimensions of the lens may also be measured. Using this information, the length of the largest dimension of the lens in image pixels in any subsequent image can be used to determine the distance between the camera and the corrective lens (eg, if the pixel solid angle is known). In combination with pattern features of known size, all distances l , l1 and l2 described above can be determined from the second image.

At block 1018, a second pattern different from the first pattern may be provided. The second pattern may include a plurality of patterns, such as patterns 510 and 520 for a plurality of lenses, and/or one or more additional patterns for a multifocal or progressive lens. However, if desired, the pattern indicated in block 1000 may still be displayed rather than displaying the second pattern.

At block 1020 , the lensmeter may provide an instruction to place the corrective lens at an intermediate position 900 between the camera and the pattern of the lensmeter device, eg, at a central position.

At block 1022 , a second image of the pattern through the corrective lens (eg, the first pattern provided at block 1000 or the second pattern provided at block 1018 ) is provided with the corrective lens in an intermediate position. acquired by a lens meter. The outer boundary of the corrective lens may also be identified in the second image.

At block 1024, the lensmeter determines whether the corrective lens is occluded.

If it is determined that the corrective lens is occluded, at block 1026 the lensmeter may display a second image having an occlusion indicator, such as occlusion indicator 814 of FIG. 8 associated with representation 812 of the user's finger occluding the lens. . While an occlusion is detected, the lensmeter may continuously return to blocks 1022 , 1024 and 1026 until the occlusion is removed.

If it is determined that the corrective lens is not occluded, then at block 1028 (eg, using the known size of the corrective lens determined at block 1014 , the second image, and/or the known size of the feature in the pattern) A distance l1 between the camera and the corrective lens and a distance l2 between the corrective lens and the pattern (eg, l-l1 ) are determined.

At block 1030 , the lensmeter determines whether the distance between the lensmeter and the corrective lens and the distance between the corrective lens and the pattern are similar to within a difference threshold (eg, whether the distance l1 is approximately equal to the distance l2 ). can be judged. In this way, it can be determined whether the corrective lens is in a desired intermediate position (eg, a central position).

If it is determined that the corrective lens is not in the desired intermediate position, then at block 1032, a reposition command may be provided by the lensmeter. For example, the reposition command may include a text-based and/or graphical display of absolute and/or relative distances l1 and l2 and/or an indicator such as a graphical arrow or text instructing the user to move the corrective lens and/or lensmeter. may include

While the corrective lens is not in the intermediate position (as determined, for example, by calculation and comparison of distances l1 and l2 ), the lensmeter blocks 1022, 1024, 1026, 1028, until the corrective lens is in the desired intermediate position. Operations 1030 and/or 1032 may continue to be repeated.

If it is determined at block 1030 that the corrective lens is in the desired intermediate position, then at block 1034 the lensmeter may store a second image of the pattern through the corrective lens. At block 1030 , the lensmeter may also provide an indication to the user that the corrective lens is correctly positioned in the desired intermediate position, for example, by providing a “green light” or thumb-up icon. The second image may then be automatically captured and stored in response to detection at that location or in response to a subsequent image capture operation by the user. In this way, the operation of blocks 1030 and 1032 may help guide the user to place the corrective lens in an intermediate position between the camera and the pattern for optimal measurements. However, the operation of blocks 1030 and 1032 may be omitted or reduced in some scenarios, and the distance(s) determined at block 1028 may be the distance(s) of the pattern caused by the position of the corrective lens at different positions between the camera and the pattern. It should also be noted that it can be used to account for differences in distortion.

At block 1036, the second image may be transformed into an ideal coordinate system. For example, the second image may be transformed into an ideal coordinate system indicated by the reference pattern group 706 of FIG. 7 . This transformation may include rotating, resizing, cropping, and tilting the second image to remove any distortion or inaccuracy introduced by the image capture process. In some embodiments, the second image detects boundary reference landmarks 706 in the second image, and transforms the second image using image manipulation techniques so that the boundary reference landmarks in the second image are referenced in the second image. It is transformed into an ideal coordinate system by making it appear at the same location as the corresponding boundary reference landmark 706 in the pattern group. Boundary reference landmarks in the second image may be detected by computer vision techniques, wherein the detected boundary reference landmarks or pixels constituting them are configured to have a shape, color, or other characteristic suitable for performing such computer vision techniques. can be

In some embodiments, the matrix transform is determined from the distance between a boundary reference landmark in the second image and a corresponding boundary reference landmark 706 in the reference pattern group. A matrix transform is then applied to some or all of the pixels of the second image to perform the transform.

At block 1038, the second image is processed to determine an overall distortion from the reference pattern to the pattern in the captured image. The overall distortion (ie, the distortion introduced by the corrective lens as well as the lens of the camera used to capture the image) can be determined by comparing the pattern in the second image to the pattern in the reference pattern. In some embodiments, the comparison is performed by comparing a plurality of pattern reference landmarks, such as 512 ′ in the second image, to a plurality of pattern reference landmarks, such as a plurality of pattern reference landmarks 512 , in a reference pattern group. The pattern in the captured image attributable to the corrective lens is then (eg, using the determined distance l , where l is determined using the known size of the feature in the second image and pattern and/or the determined size of the corrective lens) can be determined. The pattern distortion of the captured image due to the corrective lens is the determined distance l , and when the corrective lens is at an intermediate position between the pattern and the camera (eg, l 1 = l 2 = l /2), the equation above It can be determined using 5 and 6. If the corrective lens is not in the intermediate position (eg, l 1 ≠ l 2 ), the equation that determines the distortion of the pattern in the captured image due to the corrective lens determines the distortion of the pattern in the captured image due to the corrective lens It is modified as understood by one of ordinary skill in the art to

At block 1040, at least one characteristic of the corrective lens is determined by the lensmeter. For example, the spherical power, astigmatism, and axis measurements of the corrective lens may be determined to determine the prescription for the corrective lens. Other values may also be determined for corrective lenses, such as an ADD value representing an additional magnification applied to the underside of a multifocal (eg, bifocal or trifocal) lens. In some embodiments, the spherical power, astigmatism power and astigmatism axis measurements of the corrective lens may be determined by Equations 7, 8 and 9 above. Accordingly, at least one characteristic of the corrective lens may be determined, at least in part, based on the measured size of the corrective lens.

The values of SPH, CYL, and AXIS may be displayed on the lensmeter's screen, stored in the lensmeter's memory (eg, in a database or file), and/or affiliated with the corrective lens owner to confirm or create a prescription. It can be communicated through the lensmeter's network interface to another party, such as an ophthalmologist. For example, a person who has glasses but does not know the prescription for those glasses may perform the process. Information obtained through the methods discussed herein may be transmitted to a person's eyewear specialist who can use that information to order a new set of appropriately prescribed eyeglasses.

Although various examples are described herein in which lensmeter operation is performed by lensmeter device 210 (eg, implemented with a mobile phone or smartphone), some or all of the lensmeter operation is captured by the mobile device. and remotely by the server using the transmitted images and/or other information. For example, FIG. 11 shows a lensmeter 210 implemented as a mobile phone, a laptop computer 702 displaying a pattern 230 , and a lensmeter 210 and/or laptop computer 702 over a network 1150 . Illustrated is an implementation of a lensmeter system that includes a server 1130 in communication with

As discussed above, aspects and functions disclosed herein may be implemented as hardware or software in one or more of such computer systems. There are many examples of computer systems in use today. Examples of these include network appliances, personal computers, workstations, mainframes, networked clients, servers, media servers, application servers, database servers and web servers, among others. Other examples of computer systems may include mobile computing devices such as cell phones and personal digital assistants, and network equipment such as load balancers, routers and switches. Further, aspects may be deployed on a single computer system or distributed among a plurality of computer systems coupled to one or more communication networks.

For example, various aspects and functionality may be distributed among one or more computer systems configured to provide services to one or more client computers. Additionally, aspects may be performed in client-server or multi-tier systems comprising components distributed among one or more server systems that perform various functions. Accordingly, the examples are not limited to running on any particular system or group of systems. Further, aspects may be implemented in software, hardware or firmware, or any combination thereof. Accordingly, aspects may be implemented within processes, operations, systems, system elements and components using various hardware and software configurations, and examples are not limited to any particular distributed architecture, network, or communication protocol.

Referring back to FIG. 3 , the lensmeter 210 may be interconnected with and exchange data with other systems, such as the server 1130 , via a network interface 340 connected to a network, such as network 1150 . The network may include any communication network through which computer systems may exchange data. In order to exchange data using a network, the lensmeter 210 and the network may, among others, Fiber Channel, Token Ring, Ethernet, Wireless Ethernet, Bluetooth, IP, IPV6, TCP/IP, UDP, DTN, HTTP, FTP, SNMP , SMS, MMS, SS7, JSON, SOAP, CORBA, REST and web services are available for a variety of methods, protocols and standards. To ensure the security of data transmission, the lensmeter 210 may transmit data over a network using various security means, including, for example, TSL, SSL, or VPN.

The various aspects and functions may be implemented as specialized hardware or software running on one or more computer systems. As shown in FIG. 3 , the lens meter 210 includes a camera 310 , a processor 320 , a user interface 330 , a network interface 340 , a memory 350 , and lens meter software 360 . .

Processor 320 may perform a series of instructions that cause data to be manipulated. Processor 320 may be a commercially available processor such as an Intel Xeon, Itanium, Core, Celeron, Pentium, AMD Opteron, Sun UltraSPARC, IBM Power5+, or IBM mainframe chip, but may be any type of processor, multiprocessor, or controller. can Processor 320 is coupled to other system elements including memory 350 , camera 310 , and the like.

The memory 350 may be used to store programs and data during the operation of the lens meter 210 . Accordingly, the memory 350 may be a relatively high performance volatile random access memory, such as dynamic random access memory (DRAM) or static memory (SRAM). However, memory 350 may include any device that stores data, such as a disk drive or other non-volatile storage device. Various examples may configure memory 350 with specialized and, in some cases, unique structures to perform the functions disclosed herein.

Memory 350 may also include computer-readable and writable non-volatile (non-transitory) data storage media having stored thereon instructions defining a program that may be executed by processor 320 . Memory 350 may also include information written to or written to the medium, and this information may be processed by processor 320 during program execution. More specifically, information may be stored in one or more data structures specifically configured to conserve storage space or increase data exchange performance. The instructions may be persistently stored as encoded signals, and the instructions may cause the processor 320 to perform any of the functions described herein. The medium may be, for example, an optical disk, a magnetic disk or a flash memory, inter alia. Various components may manage the movement of data between the storage medium and other memory elements, and examples are not limited to specific data management components. Also, the examples are not limited to specific memory systems or data storage systems.

The lensmeter 210 also includes one or more user interfaces 330 . User interface 330 may receive input or provide output. More specifically, the output device may render information for external presentation. The input device may receive information from an external source. Examples of interface devices include keyboards, mouse devices, trackballs, microphones, touch screens, printing devices, display screens, speakers, network interface cards, and the like.

Although lensmeter 210 is shown by way of example as a type of computer device in which various aspects and functions may be practiced, aspects are not limited to being implemented on lensmeter 210 as shown in FIGS. 2 and 3 . . The various aspects and functions may be practiced in one or more computers having architectures or components other than those shown in FIG. 3 . For example, lensmeter 210 may include specially programmed special-purpose hardware, such as, for example, an application specific integrated circuit (ASIC) customized to perform the specific operations disclosed herein. Another example is the Motorola PowerPC processor, which uses a grid of several general-purpose computing devices running MAC OS System X and several specialized computing devices running proprietary hardware and operating systems to perform the same functions.

The lens meter 210 may include an operating system that manages at least some of the hardware elements included in the lens meter 210 . In general, a processor or controller, such as processor 320, includes, for example, a Windows-based operating system such as Windows NT, Windows 2000 (Windows ME), Windows XP, Windows Vista or Windows 7 operating system available from Microsoft; The MAC OS System X operating system available from Apple Computers, one of many Linux-based operating system distributions, for example, the Enterprise Linux operating system available from Red Hat Inc., the Solaris operating system available from Sun Microsystems, or from various sources. Runs an operating system, which may be a UNIX operating system available from Many other operating systems may be used, and the examples are not limited to any particular implementation.

The processor 320 and the operating system together define a computer platform on which application programs can be written in a high-level programming language. These component applications may be executable, intermediate, bytecode or interpreted code that communicates over a communications network, eg, the Internet, using a communications protocol such as TCP/IP, for example. Similarly, aspects can be implemented using object-oriented programming languages such as .Net, SmallTalk, Java, C++, Ada, or C# (C-Sharp). Other object-oriented programming languages may also be used. Alternatively, a functional, scripting or logical programming language may be used.

Additionally, various aspects and functions may be implemented in a non-programmed environment, such as, for example, documents generated in HTML, XML, or other formats that render aspects of a graphical-user interface and perform other functions when viewed in the window of a browser program. can be implemented. In addition, various examples may be implemented as programmed or unprogrammed elements, or any combination thereof. For example, a web page can be implemented using HTML and data objects called within the web page can be written in C++. Accordingly, the examples are not limited to a particular programming language and any suitable programming language may be used. Accordingly, functional components disclosed herein may include various elements, such as executable code, data structures, or objects configured to perform the described functions.

The embodiments described above use a process for determining properties of corrective lenses using a camera of a mobile device. Other embodiments include detecting defects in a lens, comparing properties of two different lenses, determining a structural property of a lens based on an amount of diffraction (ie, distortion) detected, or determining a property of a lens It can be used to determine the properties of lenses in a number of different applications, including other applications that require

Having thus described various aspects of at least one example, it should be understood that various changes, modifications, and improvements will readily occur to those skilled in the art. For example, examples disclosed herein may be used in other contexts as well. Such changes, modifications, and improvements are intended to be a part of this invention and are intended to be within the scope of the examples discussed herein. Accordingly, the foregoing description and drawings are by way of example only.

Claims (26)

A process for determining the properties of a lens, comprising:
capturing a first captured image of the pattern through the corrective lens while the corrective lens is at a first distance from the pattern;
capturing a second captured image of the pattern through the corrective lens while the corrective lens is at a second distance from the pattern;
processing the first captured image to determine a first spherical power measurement;
processing the second captured image to determine a second spherical power measurement;
selecting an extreme spherical power measurement from among the plurality of spherical power measurements from a plurality of spherical power measurements comprising the first spherical power measurement and the second spherical power measurement;
determining the lens refractive power of the corrective lens with reference to the extreme spherical power measurement
containing,
process.
According to claim 1,
The extreme spherical power measurement is the largest absolute value among the plurality of spherical power measurements when the corrective lens is a converging lens, and the extreme spherical power measurement is the smallest among the plurality of spherical power measurements when the corrective lens is a diverging lens absolute value,
process.
According to claim 1,
The first captured image and the second captured image are captured by a camera having a camera lens, the camera lens having a fixed distance from the pattern during capture of the first captured image and the second captured image located in,
process.
According to claim 1,
The process further comprises determining a prescription for the corrective lens, wherein the prescription includes at least a spherical power value, an astigmatism power value, and an astigmatism axis value.
process.
According to claim 1,
the pattern is a first pattern, the corrective lens is a first corrective lens,
capturing the first captured image comprises capturing, in the first captured image, a second pattern through a second corrective lens;
The first pattern and the second pattern may be determined in the first captured image in the first captured image when the first corrective lens and the second corrective lens are disposed in known positions relative to the first pattern and the second pattern. spaced apart from each other so that the pattern and the second pattern can be captured,
process.
According to claim 1,
processing the first captured image to determine the first spherical power measurement comprises transforming the first captured image into an ideal coordinate system;
wherein processing the second captured image to determine the second spherical power measurement comprises transforming the second captured image to the ideal coordinate system;
process.
According to claim 1,
processing the first captured image to determine the first spherical power measurement comprises:
determining an overall distortion from a reference pattern to the first captured image;
determining distortion of the captured pattern due to the corrective lens;
containing,
process.
According to claim 1,
wherein each of the first captured image and the second captured image includes a first region comprising the pattern and generated by light passing through the corrective lens, and a first region that is generated by light that does not pass through the corrective lens. comprising a second region;
determining a distortion of the captured pattern due to the corrective lens is performed at least in part with reference to the second region;
process.
According to claim 1,
determining, from the first captured image, a first distance from a camera lens of a lensmeter at which the first captured image was captured to the pattern;
identifying an orientation relative to a second location relative to the first location;
guiding the user of the lens meter to the second position;
capturing the second captured image of the pattern through the corrective lens at the second location;
further comprising,
process.
According to claim 1,
processing the first captured image to determine the first spherical power measurement comprises:
detecting a plurality of captured pattern landmarks in the first captured image;
determining a transformation from a plurality of ideal pattern landmarks to said plurality of captured pattern landmarks;
determining, for the corrective lens, from the transformation, the first spherical power measurement;
containing,
process.
7. The method of claim 6,
Transforming the first captured image into an ideal coordinate system comprises:
detecting a plurality of captured reference landmarks in a second region of the first captured image;
determining a transformation from a plurality of ideal reference landmarks to the plurality of captured reference landmarks;
applying the transform to the first captured image.
containing,
process.
8. The method of claim 7,
the first captured image is captured by a camera having a camera lens;
Determining the distortion of the captured pattern due to the corrective lens comprises:
determining a distance between the camera lens and the pattern;
determining at least one focal length of the corrective lens with reference to the distance and spherical power measurements;
containing,
process.
9. The method of claim 8,
The pattern includes a checkerboard pattern, and the second region includes a border,
process.
10. The method of claim 9,
guiding the user of the lens meter from the first position to a third position;
capturing a third captured image of the pattern through the corrective lens at the third location;
further comprising,
wherein the third position is located substantially intermediate between the lensmeter and the pattern;
process.
11. The method of claim 10,
further comprising, for the corrective lens, determining, from the transformation, an astigmatism power measurement and an astigmatism angle measurement;
process.
11. The method of claim 10,
wherein the transformation is a refractive power matrix,
process.
As a lens meter,
a camera having a camera lens;
visual display;
a processor connected to the camera
Including, wherein the processor,
capturing a first captured image of the pattern through the corrective lens while the corrective lens is at a first distance from the pattern;
capturing a second captured image of the pattern through the corrective lens while the corrective lens is at a second distance from the pattern;
processing the first captured image to determine a first spherical power measurement;
processing the second captured image to determine a second spherical power measurement;
selecting an extreme spherical power measurement from among the plurality of spherical power measurements from a plurality of spherical power measurements comprising the first spherical power measurement and the second spherical power measurement;
configured to determine the lens power of the corrective lens with reference to the extreme spherical power measurement;
lens meter.
18. The method of claim 17,
The extreme spherical power measurement is the largest absolute value among the plurality of spherical power measurements when the corrective lens is a converging lens, and the extreme spherical power measurement is the smallest among the plurality of spherical power measurements when the corrective lens is a diverging lens absolute value,
lens meter.
18. The method of claim 17,
wherein the first captured image comprises a first area comprising the pattern and produced by light passing through the corrective lens, and a second area produced by light that does not pass through the corrective lens;
lens meter.
18. The method of claim 17,
The processor is
determining the distance between the camera lens and the pattern;
configured to determine at least one focal length of the corrective lens with reference to the distance and spherical power measurements;
further configured to determine distortion of the captured pattern due to the corrective lens;
lens meter.
18. The method of claim 17,
The processor is
determine, from the first captured image, a first distance from the camera lens to the pattern;
identify a direction for a second location relative to the first location;
guide the user of the lens meter to the second position,
to capture the second captured image of the pattern
more composed,
lens meter.
18. The method of claim 17,
The processor is
detecting a plurality of captured pattern landmarks in the first captured image;
determine a transformation from the plurality of ideal pattern landmarks to the plurality of captured pattern landmarks;
configured to determine, for the corrective lens, from the transformation, the first spherical power measurement;
configured to process the first captured image to determine the first spherical power measurement;
lens meter.
23. The method of claim 22,
wherein the processor is further configured to determine, for the corrective lens, from the transformation, an astigmatism power measurement and an astigmatism angle measurement.
lens meter.
23. The method of claim 22,
wherein the transformation is a refractive power matrix,
lens meter.
A process for determining the properties of a lens, comprising:
capturing a plurality of captured images of the pattern through the corrective lens while the corrective lens is moved relative to the pattern;
processing each captured image of the plurality of captured images to determine a plurality of spherical power measurements, each of the plurality of spherical power measurements being determined from one of the plurality of captured images;
selecting an extreme spherical power measurement from among the plurality of spherical power measurements;
determining the lens refractive power of the corrective lens with reference to the extreme spherical power measurement
containing,
process.
26. The method of claim 25,
The extreme spherical power measurement is the largest absolute value among the plurality of spherical power measurements when the corrective lens is a converging lens, and the extreme spherical power measurement is the smallest among the plurality of spherical power measurements when the corrective lens is a diverging lens absolute value,
process.

Images