KR20150128916A - Estimating bilirubin levels - Google Patents

Abstract

Systems, methods and devices for estimating bilirubin levels are provided. In an aspect, a method for estimating a level of a patient's bilirubin comprises receiving image data for at least one image comprising a color correction target and a region of the skin of a patient. The color-balanced image data for the skin region is generated based on the skin region and a subset of the image data corresponding to the color correction target. The patient's bilirubin level is estimated based on color-balanced image data for the skin area.

Description

ESTIMATING BILIRUBIN LEVELS < RTI ID = 0.0 >

This application claims priority to U.S. Provisional Application No. 61 / 777,097, filed March 12, 2013, which is incorporated herein by reference.

An estimated 60-84% of newborn babies have neonatal jaundice, which causes yellowing of the skin caused by the accumulation of excess bilirubin (hyperbilirubinemia), a natural compound that is produced by the destruction of red blood cells. Although these conditions are typically harmless and naturally healed within days, highly elevated levels of bilirubin can cause tremendous damage to the nucleus jaundice, which leads to hearing loss, cerebral palsy, deep developmental delay, or even death, It can lead to neurological lesions.

Current approaches to monitoring bilirubin levels in infants typically require repeated testing in hospital settings. The serum concentration of bilirubin may be determined by a transcutaneous bilirubinometer (TcB) measurement achieved using a noninvasive or expensive instrument, or by total serum bilirubin (TSB) measured from a blood sample. These tests are often not available in resource-poor settings, thus delaying early detection and treatment of nuclear jaundice. Frequently used visual assessments as an alternative to these tests are often inaccurate and can be confused by factors such as lighting or skin color. Thus, an improved approach is needed to provide non-invasive and cost-effective screening for excess bilirubin levels.

Systems, methods and devices for estimating bilirubin levels are provided. In many embodiments, the mobile device is used to capture image data of a color correction target and a skin of a patient. Image data is used to generate an estimate of the bilirubin level. Image processing may include converting image data to a plurality of different color spaces to facilitate evaluation of the overall yellowness of the skin while compensating for color differences caused by lighting, skin color, and other potential confounding factors . The screening techniques described herein may be performed by a user (e.g., a patient, a professional medical staff, a community health worker) at an outpatient clinic (e.g., a patient's home) without the need for specialized medical equipment , Improving the convenience, accessibility and cost-effectiveness of bilirubin monitoring.

Thus, in a first aspect, a method is provided for estimating the level of bilirubin in a patient. The method includes receiving image data for at least one image comprising a color correction target and a region of the skin of a patient. Color-balanced image data for a skin region is generated based on a skin region and a subset of image data corresponding to the color correction target. The patient's bilirubin level is estimated based on color-balanced image data for the skin area. In many embodiments, the image data may be obtained with any suitable imaging device. The imaging device may acquire image data irrespective of additional attachments or equipment, such as external lenses, filters, or other specialized hardware.

The bilirubin level can be estimated using only the image data of the patient's skin color at a specific point in time or by comparing the skin color image data with the basic skin color image data. For example, the method may further comprise receiving basic skin color data for the corresponding patient when the patient has a baseline bilirubin level (e.g., about 0 when basic data is acquired within 24 hours after birth) . The bilirubin level can be estimated based on one or more differences between the color-balanced image data for the skin region and the primary skin color data. Basic skin color data for a patient can be generated by capturing basic image data for a patient when the patient has a baseline bilirubin level. The basic image data may correspond to at least one image including a basic color correction target and a skin area. Based on the skin region and a subset of the base image data corresponding to the base color correction target, color-balanced basic image data for the skin region may be generated. Basic skin color data may be generated based on color-balanced basic image data for the skin region.

A standardized color correction target may be used to facilitate the color balancing process. The color correction target may include a plurality of normalized color areas including white areas. The normalized color gamut may include black, gray, light beige, cyan, magenta, yellow and dark brown. The color correction target may at least partially define an aperture configured to expose a skin region to enable capturing of image data for the skin region. The standardized gamut can be arranged in a known arrangement surrounding the opening. Thus, the process for generating color-balanced image data for the skin region may include the steps of: receiving the received image data to identify a subset of the image data corresponding to the exposed skin region and a subset of the image data corresponding to the white region; Processing the received signal. The white region data can be processed to determine the observed color values for the white region. Color-balanced image data for the exposed skin area may be generated based on the observed color values for the white areas. The observed color values for the white region may include any suitable color space values such as red, green, and blue (RGB) color space values.

The image data may be converted into a plurality of different color spaces to detect yellow discoloration of the skin. For example, the color-balanced image data for the skin region may include RGB color space data, and the method for estimating the level of bilirubin in the patient may include color- Converting the RGB color space data into at least one other color space to produce the balanced image data. The at least one color space comprises (a) cyan, magenta, yellow and black (CMYK) color spaces; (b) a YCbCr color space; And / or (c) a Lab color space.

A plurality of achromatic and chromatic features may be generated based on the image data. In some instances, the received image data may include images obtained using flash illumination and images obtained without using flash illumination. The step of estimating the bilirubin level may comprise processing a plurality of normalized achromatic and chromatic features to select a first estimated range of bilirubin levels from one of a plurality of different bilirubin ranges. The feature may be processed using an approach based on a selected first estimated range of bilirubin levels to produce a final estimate of the bilirubin level. The plurality of different bilirubin ranges may include a lower range, an intermediate range, and a higher range. The plurality of features may include selected color values of the skin region for a plurality of different color spaces. In some instances, the plurality of features may include a calculation of a color gradient across the skin region.

Estimation of the level of bilirubin may involve carrying out one or more regression steps. For example, processing features to select a first estimated range of bilirubin levels may include (a) linear regression, (b) encapsulated k-nearest neighbor regression ), (c) lasso regression, (d) LARS regression, (e) elastic net regression, (f) support vector regression using a linear kernel. vector regression), (g) support vector regression analysis that assigns high weights to higher-rated bilirubin values, and (h) random forest regression analysis. And < / RTI > Processing the features to select a first estimated range of bilirubin levels comprises the steps of: (a) linear regression analysis, (b) encapsulated k-near neighborhood regression analysis, (c) lacrosse regression analysis, (d) LARS regression analysis , support vector regression analysis using a linear kernel, (g) support vector regression analysis that assigns high weights to high-grade bilirubin values, and (h) random forest regression analysis. And performing a series of regression analyzes. Using the processing approach based on the selected first estimated range of bilirubin levels, the step of using a selected first estimated range of bilirubin levels and a final result using a plurality of normalized achromatic and chromatic features as features for a final random forest regression analysis And performing a random forest regression analysis. In some instances, estimating the bilirubin level comprises determining a color space value for the patient's skin and using a processing approach based on the determined color space value to estimate the bilirubin level. Regression analysis formulas may also include features from the base image (e.g., color space values from one or more color spaces).

In another aspect, a mobile device configured to estimate a level of bilirubin in a patient is provided. The device includes a camera operable to capture image data for a field of view, a processor operatively coupled to the camera, and a data storage device operatively coupled to the processor. The data storage device may store instructions that when executed by the processor cause the processor to receive image data for an image captured by the camera, the image including a color correction target and a region of the skin of the subject . Command causes the processor to generate color-balanced image data for the skin region based on the skin region and a subset of the image data corresponding to the color correction target and to generate color-balanced image data for the skin region based on the color- To estimate the patient's bilirubin level. In many embodiments, the mobile device may be used to estimate the bilirubin level regardless of any external attachment (e.g., a lens, filter) to the mobile device or any other specialized mobile device equipment.

The color correction target may at least partially define an aperture configured to expose a skin region and may include a plurality of normalized color regions including a white region to enable capturing of image data for the skin region . The instructions may cause the processor to process the received image data to identify a subset of the image data corresponding to the exposed skin region and a subset of the image data corresponding to the white region. The white region data can be processed to determine the observed color values for the white region. Based on the observed color values for the white region, color-balanced RGB image data for the exposed skin region may be generated. By converting the color balanced RGB image data into at least one other color space, color-balanced image data of the exposed skin region for at least one other color space can be generated. To select a first estimated range of bilirubin levels from one of a plurality of different bilirubin ranges, a plurality of normalized achromatic and chromatic features may be processed. The feature may be processed using an approach based on a selected first estimated range of bilirubin levels to produce a final estimate of the bilirubin level.

Alternatively or additionally, color-balanced image data for the exposed skin region may be processed to determine a color space value for the patient's skin. The plurality of normalized achromatic and chromatic features may be processed using an approach based on the determined patient's skin color space value to estimate the bilirubin level.

The mobile device described herein may further comprise a flash unit operable to selectively illuminate a field of view. The received image data processed to estimate the bilirubin level may include the captured image with the field of view illuminated by the flash unit and the captured image with the field of view not illuminated by the flash unit.

In yet another aspect, a method is provided for estimating a level of bilirubin in a patient. The method includes receiving image data from a mobile device for an image including a color correction target and a skin area of a patient. The color-balanced image data for the skin region may be generated via one or more processors based on a skin region and a subset of the image data corresponding to the color correction target. The patient ' s bilirubin level can be estimated based on color-balanced image data for the skin region, via one or more processors. The estimated bilirubin level may be transmitted to the mobile device. In many embodiments, at least one of receiving the image data and transmitting the estimated bilirubin level is performed using text messaging service (SMS) text messaging. The image data may be obtained by the mobile device without using external attachments, hardware add-ons, or any other specialized equipment.

Other objects and features of the present invention will become apparent from a review of the specification, the claims, and the accompanying drawings.

The novel features of the invention are set forth with particularity in the appended claims. The features and advantages of the present invention will be more readily understood with reference to the following detailed description and the accompanying drawings, which illustrate exemplary embodiments in which the principles of the invention may be utilized.

Figure la shows a guideline for using phototherapy to treat neonatal jaundice, according to an embodiment;

Figure IB shows a guideline for using transfusion transfusions to treat neonatal jaundice, according to an embodiment;

Figure 1c shows a Bhutani nomogram for assessing the risk associated with neonatal jaundice;

Figure 2 illustrates the use of a mobile device to capture image data used to estimate a patient's bilirubin level, according to many embodiments;

Figures 3A-3D illustrate a color correction target for use with a mobile device to capture image data used to estimate a bilirubin level, in accordance with multiple embodiments;

Figures 4A-4C illustrate an exemplary user interface of a mobile application for estimating a bilirubin level, in accordance with multiple embodiments;

Figures 5A-5F illustrate exemplary image data collected for use in estimating a bilirubin level, in accordance with multiple embodiments;

Figure 6 illustrates a method for estimating the bilirubin level of a patient, according to many embodiments;

Figure 7 illustrates a method for generating color-balanced image data for a skin of a patient, in accordance with multiple embodiments;

Figure 8 shows the identification of a normalized color gamut in an image, according to many embodiments;

9A illustrates a method for estimating a patient's bilirubin level, according to many embodiments;

Figure 9B illustrates another example of a method for estimating the bilirubin level of a patient, according to many embodiments;

10 illustrates a method for estimating a patient's bilirubin level, in accordance with multiple embodiments;

11A illustrates a method for estimating a bilirubin level using basic skin color data, according to many embodiments;

11B illustrates a method for generating basic skin color data, in accordance with multiple embodiments;

12 illustrates a mobile device for estimating a bilirubin level, according to many embodiments;

Figure 13 illustrates a mobile device in communication with a data processing system for estimating a bilirubin level, in accordance with multiple embodiments.

The systems, devices and methods described herein provide an improved approach for estimating the level of bilirubin in a patient (e. G., Infant or adult). A mobile device configured with appropriate software may be used to capture a standardized color correction target and an image of the patient ' s skin. Skin and target image data may be used to generate color-balanced image data that is analyzed to estimate a patient's bilirubin level. For example, color-balanced image data can be transformed into a plurality of different color spaces to extract features indicative of skin yellowing, and these features can be used in multiple regression analyzes to generate bilirubin estimates . In contrast to conventional bilirubin level measurements that rely on invasive blood testing or utilize expensive instrument equipment, the systems, devices, and methods described herein are readily performed by non-medical personnel using personal mobile devices Which is convenient, portable, and inexpensive, which can be achieved, and thus improves the accessibility and cost-effectiveness of bilirubin monitoring. Advantageously, the method described herein can be performed on a mobile device without requiring the use of external attachments, hardware add-ons, or any other specialized mobile device equipment. In particular, the disclosed technique provides an accurate bilirubin level estimate over a wide range of bilirubin concentrations, in contrast to the less accurate TcB measurement at high bilirubin levels. Additionally, the method described herein improves the accuracy and flexibility of non-invasive bilirubin level estimation, taking into account the diversity of different lighting states as well as skin color. In addition, the use of a mobile device software platform enables easy and rapid updating of the estimation methods and algorithms described herein, thus allowing improvements and upgrades to be made as needed.

Referring now to the drawings, FIG. 1A illustrates a guideline for using phototherapy to treat neonatal jaundice. Similarly, Figure IB illustrates a guideline for using exchange transfusions to treat neonatal jaundice. These guidelines may be used by specialists to determine the appropriate course of treatment based on the age of the infant, total serum bilirubin (TSB), number of pregnancies (eg, over 35 weeks), and other risk factors. Risk factors may include allograft hemolytic disease, G6PD deficiency, asphyxia, significant nausea, body temperature instability, sepsis, acidosis, or albumin levels below 3.0 grams per deciliter. The curves shown in FIGS. 1A and 1B indicate exemplary treatment thresholds for low-risk, medium-risk, and high-risk child.

Figure 1c illustrates a Bhutani nomogram 110 for assessing the risk associated with newborn jaundice. Butaninomogram (110) can be used by professional medical personnel to assess the risk of infants to develop hyperbilirubinemia based on the age of the infant and the bilirubin level. For example, the Butanomyogram 110 may be used to define a low risk zone 114, a low intermediate risk zone 116, a high intermediate risk zone 118 and a high risk zone 120 The percentile curve 112 of FIG.

Figure 2 illustrates a mobile device-based estimate of bilirubin level, in accordance with multiple embodiments. An imaging device, such as a mobile device 202 (e.g., a smartphone, tablet) is used to capture images of the color correction target 208 and the skin area 204 of the patient 206. Suitable skin areas for the approaches described herein include forehead and sternum as well as any other significantly flat areas of the skin that are likely to be uniformly illuminated. Additionally, since jaundice typically appears first in the forehead and proceeds slowly down the body, a region closer to the forehead may potentially be more beneficial for diagnosis. Thus, in many embodiments, the color correction target 208 is on the abdomen of the patient 206 near the sternum. Mobile device 202 may include a mobile software application ("mobile app" or "app") for estimating bilirubin levels based on image data as well as a camera (not shown) And a display 210 used to display a user interface (UI) The mobile device 202 may be a personal device of a user (e.g., a parent, a professional medical staff, a community health worker, etc.), and thus a user may select a color correction target 208, It is necessary to install only the mobile app on its own personal device without requiring any other instrument or hardware. In particular, the mobile device 202 may be used to implement the methods described herein regardless of any additional attachments or accessories of the mobile device, for example, external lenses, filters, or other specialized mobile device equipment. Additional details of software and hardware components suitable for mobile device 202 are provided in more detail below.

Figures 3A-3D illustrate a color correction target that may be used with a mobile device to estimate a bilirubin level, in accordance with multiple embodiments. The color correction target (also referred to as a "color calibration card " or" color card ") described herein may be used to determine the color balance 3A, a color correction target 300 may be provided on a rectangular card 302. The card 302 may be provided to a patient (e. G., ≪ RTI ID = 0.0 > For example, approximately the size of light, suitable for placement on the skin of a child (such as a child). In some instances, the card 302 may be used to prevent the spread of pathogens among patients Germicidal or disposable. The color correction target 300 may include a plurality of standardized colored regions 304 that may be printed on the card 302. The color region 304 may be of any suitable size, number or type (E.g., square, rectangular, polygonal, circular ellipse, etc.). For example, the color correction target 300 includes eight equal-sized square color areas 304a-h, Each of the standardized color gamuts 304 may have a different color (e.g., black, gray, white, cyan, magenta, yellow, light beige, dark brown) 3A, 304a is a black region, 304b is a gray region (e.g., 50% gray), 304c is a white region, and 304d is a white region 304e is a cyan area, 304f is a magenta area, 304g is a yellow area, and 304h is a dark brown area (for example, a second skin color). Other arrangements and combinations may also be used. [0064] In addition, the back side of the card 302 Search) can include at least one adhesive region in which a card 302 is to be removably attached to the patient's skin. The backplane may also include related commands, such as user instructions, on how to download the attached mobile app on the user's personal mobile device.

FIG. 3B illustrates an alternative embodiment of the color correction target 320. FIG. The color correction target 320 is substantially similar to the color correction target 300 except that the color correction target 320 also defines an aperture 322. [ The opening 322 can be positioned such that the area of interest skin is exposed through the opening 322 when the color correction target 300 is placed on the patient. The opening 322 is defined (e.g., completely enclosed) or partially defined (e.g., partially surrounded) by the target 320. A plurality of normalized color regions 324 including white regions may be positioned around openings 322 in a known arrangement. The embodiment of FIG. 3b further includes light gray, medium brightness gray, and dark gray areas for a total of ten different normalized color areas. FIG. 3C illustrates a color correction target 340 provided on a square card 342, in which a plurality of normalized color areas 344 surround square openings 346. FIG. Some of the color regions 344 may include two or more colors, such as a color region 348 including a center black square with white boundaries. FIG. 3D illustrates a color correction target 360 having a plurality of normalized color gamut 362 surrounding aperture 364. FIG. The opening 364 may be adjacent to some or all of the gamut 362. Color gamut 362 is shown in Figure 3d as a series of rectangular regions with different aspect ratios. The color, geometry, and arrangement of the color gamut of the color correction target described herein may be selected based on the image processing and analysis method to be used, as in the embodiments discussed below.

Figures 4A-4C illustrate an exemplary user interface (UI) of a mobile app for estimating bilirubin levels, in accordance with many embodiments. 4A illustrates a summary UI 400 for displaying various statistics and metrics relating to a patient. The UI 400 may include patient identification information 402 (e.g., patient name, study ID number), birth time 404 and any available bilirubin level results 406 (e.g., TSB and / or TcB Results). UI 400 may also include a button 408 or other interactive element that allows a user to acquire patient image data (e.g., a video sample or a photo sample). In some instances, a video sample may be advantageous to eliminate the motion blur problem. 4B illustrates a command UI 420 for helping the user to capture data already in the patient. UI 420 may include a graphical and / or textual instruction 422 that directs the user to perform the appropriate steps. For example, the command 422 may instruct the user to place a color correction target on the abdomen of the patient below the sternum. In many embodiments, the instructions 422 are adapted to be easily understood by an untrained individual, allowing non-medical professionals to operate the mobile app. 4C illustrates a live preview 440 that is displayed to the user during the image capture process. UI 440 may include a video preview 442 of the current view field of the camera of the mobile device. The UI 440 may include one or more positioning targets 444 that help the user position the mobile device. For example, the positioning target 444 may be a box, frame, or gamut overlaid on the video preview 422 to indicate that the color correction target is placed at the proper position. The size and position of the positioning target 444 may be selected to limit the distance of the mobile device from the patient to within an optimal range and also to ensure that the field of view adequately captures the appropriate skin area and color correction target. Once the fields of the view are correctly aligned, the user can record the image data by tapping the Record button 446. Additional details regarding the image acquisition and analysis process are provided below.

Figures 5A-5F illustrate exemplary image data collected by a mobile app according to a number of embodiments. After the recording process, the mobile app can display image data to the user for review and approval. In addition, in order to ensure that the captured image has sufficient quality for processing and analysis, the mobile app may include a function for detecting image quality, and may include an image that does not satisfy the quality threshold and needs re- The user can be notified. Figure 5A illustrates an image 500 of sufficient quality that the lighting is satisfactory and clearly visible to both the patient and the color correction target. FIG. 5B illustrates a defective image 502 where a portion of the image is interrupted by flashing light. FIG. 5C illustrates a defective image 504 where the image is too bright. 5D illustrates a defective image 506 in which the color correction target is partially obstructed. FIG. 5E illustrates a defective image 508 in which the image is partially blurred by shadows. FIG. 5F illustrates a defective image 510 where the image is too dark.

FIG. 6 illustrates a method 600 for estimating a patient's bilirubin level in accordance with multiple embodiments. Method 600, as with all of the other methods described herein, may be performed by any of the devices and systems described herein, e.g., a mobile computing device (e.g., a remote server) ≪ / RTI > The specific steps of method 600, like all other methods described herein, may be optional or may be combined with or used in place of appropriate steps of other methods described herein.

At operation 610, image data for at least one image including a color correction target and a region of the skin of the patient is received. The image data may be collected using the camera of the mobile device as previously described herein. The image data may comprise photographic data, video data or any suitable combination thereof. The photo data and video data may be captured sequentially or simultaneously (e.g., the image is captured during video recording). Image data may be obtained using flash illumination and / or without using flash illumination. In some cases, the flash illumination may be used to offset the ambient illumination, and thus the resulting image illumination is determined only by the contribution of the flash illumination. This may be advantageous in situations where it is advantageous to create more consistent lighting or where the environment lighting is lane (e.g., too dark, color intensive, etc.).

In many embodiments, the image data is collected using a mobile app that controls the number and sequence of captured images as well as the flash unit of the mobile device. For example, the app may turn on the flash unit during initial positioning of the mobile device so that the user may evaluate the amount of light in the image (e.g., via video preview) to reduce flashing, or The device can be repositioned as needed to remove it. During the writing process, the app turns on the flash unit to acquire the image data first, then turns off the flash unit to acquire the image data to create two sets of images each having flash illumination and no flash illumination The mobile device can be controlled. For example, when the user initiates the image capture processor (by pressing the write button), the app is configured to turn on the flash unit during the first half and turn off the flash unit during the second half to record the video of the patient and calibration target . The app can also capture two photos taken during the first and second half of the video record, respectively. The overall length of the video may be any suitable time, for example approximately 10 seconds. The timing of the image capture process may be further configured to ensure that the image sensor (e.g., a charge-coupled device (CCD) array) of the mobile device stabilizes before the next image is captured. For example, an app may include a certain amount of delay time (e.g., 3 to 4 seconds) before recording each set of images.

As previously discussed, once the image capture sequence is complete, the mobile app can already analyze the data to determine if the image data has sufficient quality for subsequent use. For example, an app may implement image analysis techniques (e.g., computer vision) suitable for evaluating image quality. The exemplary procedure may include extracting a color correction target from the captured image data (e.g., to determine whether the standard deviation of pixel values for each gamut falls below a predetermined threshold) Checking color consistency across each color gamut, and recommending that the user retake any image that does not pass such a test. In some instances, an app may be configured to capture multiple sets of image data per session to maximize the likelihood that at least a portion of the image data will meet quality standards. Thereafter, as will be described in greater detail below, the authorized image may be processed on the mobile device or transmitted to the separate computing system for processing.

In operation 620, color-balanced image data for the skin region is generated based on the skin region and a subset of the image data corresponding to the color correction target. Color balancing can be used to compensate for different lighting conditions because the color content of the image data can vary significantly depending on the environmental lighting (e.g. intensity, color, type (halogen fluorescent lighting, natural lighting, etc.) . In many embodiments, the image data is initially normalized by dividing the three red, green, and blue (RGB) color channels into the full luminescence of the image. In addition, as discussed below, the observed pixel color values for one or more of the normalized color gamuts of the color correction target can be used to determine the color balancing adjustment to be applied to the skin region in the image data.

At act 630, the patient's bilirubin level is estimated based on color-balanced image data for the skin region. Because hyperbilirubinemia results in yellow discoloration of the skin, the bilirubin level can be determined based on the amount of yellow color in color-balanced skin area image data. This determination can be performed using any suitable technique, for example, by converting the color-balanced image data into a plurality of different color spaces selected to facilitate quantifying the total yellowing of the skin. Image features generated from the transformed image data can be entered into a series of machine learning regression designed to estimate the concentration of bilirubin in the patient's body. This approach is discussed in more detail below.

FIG. 7 illustrates a method 700 for generating color-balanced image data in accordance with a number of embodiments. The method 700 may be applied to photographic and / or video data obtained from a skin area of a patient and a color correction target, as discussed previously. In operation 710, the received image data is processed to identify image data corresponding to the exposed skin area and image data corresponding to the white area. For example, the image data may be segmented to extract pixel values of the skin region of interest (sternum, forehead) and the color gamut of the color correction target. For the color correction target, if the mobile app UI includes a positioning target (e.g., the positioning target 444 in FIG. 4C) for aligning the correction target, the approximate pixel coordinates of the color correction target in the image data are pre- So that the search space for the color gamut can be limited. The positioning of the gamut can be determined by identifying at least two gamut regions on the correction target.

Figure 8 illustrates the identification of a normalized color gamut in an image, according to many embodiments. The gamut segmentation process may be implemented through an appropriate algorithm configured to apply a predetermined threshold to an image. In many embodiments, the algorithm initially attempts to identify at least two of these regions because the cyan, magenta, and yellow regions have distinct colors and relatively high saturation. The algorithm may convert the image data to color, saturation, value (HSV) color space and apply an empirically determined threshold to the color and saturation channel to obtain a threshold color image 800 and a threshold saturation image 802 . An "AND" operation may be performed on the two threshold images to separate the specific color gamut from the rest of the image, thereby creating a segmented image 804. In addition, since the approximate size of each gamut is predetermined, this information can be used to distinguish the gamut from the noise of the image data (e.g., using edge detection, morphological operation, etc.) Can be used. The algorithm may then use contour detection to identify boundaries of the gamut when contour smoothing is performed using an appropriate technique such as the Douglas-Peuker algorithm. Since the entire arrangement of the gamut is known, once the position of the two gamut is determined, the overall orientation of the correction target can be calculated and used to extrapolate the position of the other gamut including the white gamut.

에 대하여, 피부 영역에 대한 조절된 색 값(R, G, B)는 하기의 식에 의해 획득될 수 있으며, At operation 720, the white area data is processed to determine the observed color value for the white area. At operation 730, color-balanced image data for the exposed skin area is generated based on the observed color values for the white areas. The observed color values for the white region may include RGB color values that can be used to adjust the RGB values of the image data for the skin region. For example, the observed color value for the white region

, The adjusted color values (R, G, B) for the skin region can be obtained by the following equation,

는 색 보정 타겟상의 백색 영역의 미가공의 관찰된 색 값이다. here,

Is the raw observed color value of the white region on the color correction target.

9A illustrates a method 900 for estimating a patient's bilirubin level in accordance with multiple embodiments. The method 900 may be practiced using color-balanced RGB image data obtained as discussed previously herein. In operation 910, the color-balanced RGB image data is converted to at least one other color space to produce color-balanced image data of the exposed skin region of at least one other color space. As discussed above, the yellowing of the color-balanced skin area image data correlates with the bilirubin level of the patient's body. In many embodiments, an image sensor (e.g., a CCD or CMOS sensor) of the mobile device interpolates the reflected light into red, green, and blue wavelengths, which clearly indicates that the image sensor captures the reflection of the yellow wavelength band . Thus, the color-balanced RGB data may include a plurality of different color spaces, for example, cyan, magenta and yellow (CMY); Cyan, magenta, yellow and black (CMYK); YCbCr; Or Lab color space. Any suitable number and combination of color spaces may be used. For example, in many embodiments, RGB image data may be converted to CMYK, YCbCr, and Lab color spaces. Alternatively, RGB image data may be converted to CMY, YCbCr, and Lab color spaces.

At operation 920, a plurality of normalized achromatic and chromatic features are processed to select a first estimated range of bilirubin levels from a plurality of different bilirubin ranges. The features may be achromatic and / or chromatic (e.g., luminescence) values for the skin region, and each feature corresponds to the color space value of the color space used. In some instances, the feature may include the calculation of one or more color gradients across the skin region. Any suitable number and combination of features may be used. For example, three features may be extracted from each of the four color spaces (e.g., RGB, CMY, YCbCr, Lab; or RGB, CMYK, YCbCr, Lab) to obtain twelve achromatic and chromatic features . In addition, the features can be individually extracted from the image data obtained with and without flash illumination, respectively, resulting in a total of 24 chromatic and achromatic features. The feature may be normalized based on the overall luminescence of the image as previously described herein with respect to operation 620 of method 600. [ The extracted features can be used as inputs to machine learning regression analysis used to estimate bilirubin levels. Regression analysis may utilize some or all of the extracted features and the optimal subset of features to be used is selected based on machine learning regression analysis. The machine learning regression analysis described herein may be trained for an appropriate set of data, such as clinical patient data, to incorporate any suitable number and combination of parametric and non-parametric regression models . An exemplary regression analysis suitable for use with the methods described herein is provided below. In many embodiments, the initially computed feature and the output of the selected regression analysis are used to classify the estimated bilirubin level into one of a plurality of different bilirubin ranges, e.g., one of low, middle, and high range.

In operation 930, the feature is processed using a processing approach based on a selected first estimated range of bilirubin levels to produce a final estimate of the bilirubin level. Similar to operation 920, the normalized achromatic and chromatic features can be used to signal one or more machine learning regression analysis. The input to the regression analysis may be different based on whether the first estimated range of bilirubin levels is low, intermediate, or high, as provided in more detail below. For example, the classification of the first estimated range may be used as an input to machine learning regression analysis. The results of the initial regression analysis performed at operation 920 may also be used as input. This "two-tiered" approach to bilirubin estimation can be used to generate more accurate estimation results compared to direct estimates.

FIG. 9B illustrates a method 950 for estimating a patient's bilirubin level according to a number of embodiments. The method 950 may be implemented in combination with the method 900 to obtain a more accurate estimate of the bilirubin level. Similar to method 900, method 950 may be implemented using color-balanced RGB image data obtained as previously described herein. At operation 960, the color-balanced RGB image data is converted to color-balanced image data of at least one other skin space, as discussed above with respect to operation 910 of method 900, Lt; RTI ID = 0.0 > color space. ≪ / RTI >

At operation 970, the color-balanced image data for the exposed skin region is processed to determine a color space value for the patient's skin. The color space value for the patient ' s skin can be used to classify the patient ' s skin color into one of a plurality of different skin color types, for example, bright skin, medium skin and black skin. The skin color type may be associated with the racial and / or ethnic color of the patient. The skin color type may be determined based on the color correction target and / or the skin color value obtained from the color-balanced RGB image data. For example, the skin region may be compared to one or more normalized color regions (e.g., first and second skin color regions) of the color correction target to determine the skin color type.

At operation 980, a plurality of normalized achromatic and chromatic features are processed using an approach based on the skin color determined to estimate the bilirubin level. The normalized achromatic and chromatic features may be extracted from the color-balanced image data for one or more different color spaces as described above in connection with the method 900. The feature may be used as an input to an appropriate machine learning regression analysis along with the determined skin color, similar to operation 930 of method 900.

Figure 10 illustrates a method 1000 for estimating a patient's bilirubin level according to a number of embodiments. Method 1000 includes obtaining an image 1002 from a camera, color balancing image data 1004, performing feature extraction from color-balanced image data 1006, Performing a regression analysis (1008), and generating a bilirubin estimate (1010).

Image data may be obtained from the mobile device's camera under the control of the appropriate mobile app (act 1002). Some of the image data may be acquired with flash illumination, and some of the image data may be acquired without flash illumination. Once the app has verified that the image is of sufficient quality for processing and analysis, the image data may be color balanced (act 1004). Color balancing is performed by identifying the subset of image data corresponding to the color correction target and automatically segmenting the standardized color gamut of the target using the previously described thresholding method (act 1012) ). The segmented white areas may be used to white balance the image data (operation 1014) to produce color-balanced image data. The color-balanced image data may be RGB image data. (Act 1006). The feature extraction process may include converting (1016) the image data from the RGB color space to a plurality of different color spaces as previously described herein, ). The transformed image data may then be used to calculate a plurality of normalized achromatic and chromatic features (act 1018).

Some or all of the extracted features may be used as inputs to the machine learning regression analysis (act 1008). Any suitable number and / or combination of regression analysis may be used. For example, an initial set of machine learning regression analysis may include five different regression analyzes (operations 1020, 1022, 1024, 1026, and 1028). For example, the first regression analysis may include one or more encapsulated k-Nearest Neighbor (kNN) regression analysis (act 1020). This regression analysis can utilize a database of known characteristics and bilirubin values. When an unknown test vector is analyzed, a convex hull may be found around the test vector of the feature's database. The feature points from the convex hull can then be used in the linear regression analysis. A new regression analysis can be built every time a new test point is analyzed. The parameters for generating the convex hull may be a "green" channel (e.g., from the RGB color space) and a normalized value of the luminosity (e.g., from the YCbCr color space transformation). This can be used to ensure that the convex hull computation occurs only in two dimensions, thus ensuring that the number of convex hull points is manageable (e.g., approximately four to six points).

The second regression analysis may include one or more lasso regression and / or one or more Least Angle Regressions (act 1022). LARS can help determine which features are most useful among the total set of extracted features using a variant of the forward feature selection. For example, the best predictor (s) from a feature set may be selected by developing a single-feature linear regression analysis from each feature. The most correlated output may be selected as a "first" feature. This prediction may be subtracted from the output to obtain the residual. The algorithm may differ from other forward feature selection algorithms in that it attempts to find other features that have a correlation to the residual that is roughly the same as the first characteristic for the output. The algorithm can then find an "equiangular" direction between the two estimates and find a third feature that maximizes the correlation to the new residual along the isochronous direction. A new angle can then be found from the new features added to the previous feature and set. The feature may be added in this manner until the desired accuracy is satisfied.

The third regression analysis may include one or more Elastic Net regression analysis also known as an elastic net algorithm (act 1024). Elastic-net regression analysis is a combination of lasso regression and ridge regression. However, instead of using only forward feature selection, the algorithm may also use the L1 and L2 norms in its objective function. This allows the algorithm to be associated with LARS and Lasso but with a certain "backoff" adjustment so that the algorithm becomes more stable. The parameters can be cross-verified using a stepwise exhaustive process.

The fourth regression analysis may include one or more support vector (SV) regression analysis (act 1026). Two SV regression analyzes can be used to capture the possible non-linear relationship between the image data and the bilirubin level. SV regression analysis can be used to find a linear regression analysis function in a high dimensional feature space. The input data can then be mapped into space using a potentially non-linear function. The first SV regression analysis may use a linear kernel and the second SV regression analysis may assign a higher weight to the high-grade bilirubin value using a non-linear radial basis function.

The fifth regression analysis may include one or more random forest regression (act 1028). For example, the fifth regression analysis can use random forest regression analysis with 500 trees. Random forest is a collection of estimates. This can use many "classifying" decision trees for various sub-samples of the data set. The output of these trees can be averaged to improve prediction accuracy and control over-fitting. Each tree can be generated (alternately) using a random sub-sample.

In many embodiments, in addition to or in addition to one or more of the regression analyzes in the regression analysis described herein, other types of regression analysis may also be used. For example, one or more linear regression analysis may also be used. The method 1000 can be performed using any suitable type of regression analysis, including linear and non-linear regression analysis.

After the initial regression analysis, one or more multi-layer classifiers may be used (act 1030). For example, as previously described herein, an initially computed feature is to classify the first estimated range of bilirubin levels into one of a plurality of different bilirubin ranges, along with an output of an initial set of regressors Can be used. The classifier may include a random forest classifier, a support vector machine, and a k-Nearest Neighbor (k = 3). The results of all classifiers as well as the log-likelihood for each class can be used as a feature for final stacked regression which can be a final random forest regression analysis 1032). The results of the initial regression analysis and the original extracted features may also be used as features for this final stage regression analysis. The final regressor can be trained using an appropriate machine learning algorithm such as AdaBoost. The final regressor may be used as a final estimate 1010 of the bilirubin level (e.g., measured in milligrams per deciliter). To prevent overfitting, Leave-One-Out Cross-Validation (LOOCV) can be used at all levels of learning.

11A illustrates a method 1100 for estimating a patient's bilirubin level using primary skin color data, in accordance with various embodiments. Method 1100 is similar to method 600, except that the patient's current skin color data is compared to the primary skin color data to generate a bilirubin estimate. This approach can be used to compensate for factors that can make image analysis wrong, for example, different skin colors due to the patient's race or ethnic color.

In operation 1110, basic skin color data for the patient is received, and the primary skin color data corresponds when the patient has a baseline bilirubin level. The baseline bilirubin level may be a known bilirubin level (e.g., based on TSB or TcB testing). If the patient is an infant, basic skin color data may be collected within 24 hours after the infant's birth, typically when the bilirubin level is typically very low, for example, when the bilirubin level is approximately zero. A suitable method for collecting and generating basic skin color data is described below.

In operation 1120, image data for at least one image including a color correction target and a region of the skin of the patient is received. In operation 1130, color-balanced image data for the skin region is generated based on a skin region and a subset of the image data corresponding to the color correction target. The image data collection and color-balancing process may be similar to the processes previously described herein in connection with operations 610 and 620 of method 600, respectively. Operations 1120 and 1130 may be performed at any time after the basic skin color data is received and may be performed in any suitable time period (e.g., 10 minutes) to generate a sequential image data set used to determine whether the skin is more yellow For example, the first 4 to 5 days of the infancy period).

FIG. 11B illustrates a method 1150 for generating basic skin color data in accordance with many embodiments. At act 1160, the base image data for the patient is captured when the patient has a baseline bilirubin level, and the base image data corresponds to at least one image comprising a base color correction target and a skin region. The collection of basic image data may be similar to the collection procedure previously described herein with respect to operation 610 of method 600. The basic color correction target may be the same as the color correction target used to collect normal image data or it may be a different color correction target. Similarly, the base image data may include the same skin region or different skin regions as the normal image data. As described above, the baseline bilirubin level may be any known bilirubin level, for example, a bilirubin level determined by testing or taken within 24 hours after birth.

At act 1170, color-balanced basic image data for the skin region is generated based on the base image data. The generation of color-balanced basic image data may be performed using any of the techniques described herein previously in connection with normal image data. At act 1180, basic skin color data is generated based on color-balanced basic image data for the skin region. As discussed above, this process involves extracting features from color-balanced basic image data and using features for machine learning regression analysis.

12 illustrates a mobile device 1200 for estimating a patient's bilirubin level in accordance with multiple embodiments. The mobile device 1200 includes a camera 1202 suitable for capturing image data and a flash unit 1204 suitable for providing flash illumination. The camera 1202 and the flash unit 1204 may be built-in hardware of the mobile device 1200. Alternatively, the camera 1202 and the flash unit 1204 may be provided separately from the mobile device 1200 and coupled to the mobile device 1200 (e.g., via wired or wireless communication) have. In some instances, the mobile app described herein may detect whether the camera 1202 is able to capture an image with a sufficiently high resolution for subsequent image analysis, and notify the user if such criteria are not met have.

Mobile device 1200 also includes an input unit 1206 for receiving input from a user and a display 1208 for displaying content to a user. The input unit 1206 may include a keyboard, a mouse, a touch screen, a joystick, and the like. Input unit 1206 may also be configured to accept voice commands or gesture commands. Display 1208 may include a monitor, a screen, a touch screen, and the like. In some instances, input unit 1206 and display 1208 may be implemented across shared hardware (e.g., the same touch screen is used to accept input and display output). As previously described, the display 1208 may display to the user a UI of the mobile app for estimating one or more appropriate UIs, e.g., bilirubin levels.

The mobile device 1200 includes one or more processors 1210, memory or other data storage device 1212 or communication unit 1216 that stores one or more software modules 1214 as well as image data. The processor 1210 may be operably coupled to the camera 1202 and / or the flash unit 1204 to control one or more functions (e.g., a write function, a zoom function, a flash illumination function). Processor 1210 may also be operably coupled to memory 1212 such that processor 1210 may receive and execute instructions provided by software module 1214. [ The software module 1214 may be implemented as part of the mobile app described herein and may provide instructions for performing one or more operations of the previously discussed method. For example, the software module 1214 may enable the mobile device 1200 to capture image data of the patient and the correction target. In some instances, the software module 1214 may perform some or all of the image processing and analysis tasks described above (e.g., color balancing, feature extraction, machine learning regression analysis). Software module 1214 may be adapted to a plurality of different types of mobile devices. In addition, the mobile device 1200 receives and installs software updates, e.g., updates to improve one or more image capture, processing and analysis algorithms, thus allowing the mobile apps to be configured to be easily and quickly upgraded as needed.

The communication unit 1216 of the mobile device 1200 may communicate data (e.g., image data, bilirubin estimates, software updates, etc.) between the mobile device 1200 and an individual device or system, And / or < / RTI > The communication unit may use any suitable combination of wired or wireless communication methods including Wi-Fi communication. In some instances, communication between mobile device 1200 and an individual device may be performed using text messaging service (SMS) text messaging. The communication unit 1216 is also operably coupled to the processor 1210 so that data communication to and from the mobile device 1200 can be controlled based on commands provided by the software module 1214. [ .

13 illustrates a mobile device 1300 in communication with a data processing system 1302 for estimating a bilirubin level in accordance with multiple embodiments. Mobile device 1300 may include any of the components previously described herein in connection with mobile device 1200 of FIG. The components of data processing system 1302 may be implemented over any suitable combination of physical and / or virtual computing resources (e.g., virtual machines) including distributed computing resources (of the "cloud & have. In many embodiments, data processing system 1302 is a remote server configured to communicate with a plurality of user mobile devices, including mobile device 1300. The communication may utilize any suitable wired or wireless communication method as described above.

Data processing system 1302 includes one or more processors 1304, a memory or other data storage device 1306 that stores one or more software modules 1308, and a communications unit 1310. Communication unit 1310 may be used to communicate data (e.g., image data, bilirubin estimates, software updates, etc.) between system 1302 and mobile device 1300 (e.g., via SMS text messaging) . For example, the communication unit 1310 may receive image data provided by the mobile device 1300, e.g., image data that is not yet color-balanced. The data obtained from mobile device 1300 may be stored in memory 1306. [ The software module 1308 may include instructions executable by the processor 1304 to process and analyze image data (e. G., Color balancing, feature extraction, etc.) by performing one or more operations of, for example, Machine learning regression analysis). Processor 1304 may output an estimate of the patient ' s bilirubin level, which may be stored in memory 1306 and / or transmitted to mobile device 1300. [ In some cases, depending on user preferences, the results may also be sent to a third party, e.g., a professional medical staff who can review the results and provide additional instructions to the user as needed.

In an alternative embodiment, the mobile device described herein can be configured to illuminate the patient's skin with light of different wavelengths (e.g., 460 nm, 540 nm) and capture the image of the illuminated skin using the camera. The timing and sequence of the illumination can be controlled by the mobile app. The mobile app can analyze the collected image data to measure the intensity of light of different wavelengths reflected from the skin area. In many embodiments, absorption at different wavelengths will vary depending on the color of the skin, including yellow discoloration. Some wavelengths can be influenced by the level of bilirubin, so the intensity of the wavelength indicates the amount of bilirubin in the patient's body. Thus, an appropriate machine learning regression analysis and / or model can be developed to allow wavelength absorption to be used as an input to estimate the patient's bilirubin level. Advantageously, this approach can be more robust against different environmental and situational conditions.

In many embodiments, the mobile device is a front-facing camera (e.g., on the same mobile device side as the screen) that can be used to capture image data (e.g., photos, video) Camera). This data provides alternative and / or additional ambient lighting information that can be used to normalize the patient ' s skin color data during estimation of the patient ' s bilirubin level.

The various techniques described herein are stored on storage media and computer readable media and may be partially or fully implemented using code executable by one or more processors of the computer system. A storage medium and a computer readable medium for containing a code or a portion of a code may be embodied in a computer-readable medium such as RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) A tape, a magnetic disk storage or other magnetic storage device, a solid state drive (SSD), or other solid state storage device, or any other medium that can be used to store the desired information and which can be accessed by the system device, Such as volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing and / or transmitting information such as computer readable instructions, data instructions, program modules, or other data Including, but not limited to, storage media and communication media. That is known or used stand may include any suitable medium. Based on the teachings and teachings provided herein, one of ordinary skill in the art will recognize other ways and / or methods for implementing various embodiments.

While preferred embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that these embodiments are provided by way of example only. Numerous variations, changes and substitutions will now occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be used in practicing the invention. It is intended that the following claims define the scope of the invention, thereby means that the methods and structures within the scope of these claims and their equivalents should be covered by the claims.

Claims (27)

CLAIMS What is claimed is: 1. A method for estimating a level of bilirubin in a patient,
Receiving image data for at least one image including a color correction target and a region of a patient's skin;
Generating color-balanced image data for the skin region based on the skin region and a subset of the image data corresponding to the color correction target; And
Estimating the patient's bilirubin level based on the color-balanced image data for the skin region. The method according to claim 1,
Further comprising receiving basic skin color data for the patient corresponding to when the patient has a baseline bilirubin level;
Wherein estimating the bilirubin level is based on at least one difference between color-balanced image data for the skin region and the primary skin color data. The method of claim 2,
Wherein the basic skin color data for the patient includes:
Capture basic image data corresponding to at least one image comprising the basic color correction target and the skin region for the patient when the patient has the baseline bilirubin level;
Generate color-balanced basic image data for the skin region based on the skin region and a subset of the base image data corresponding to the basic color correction target;
And generating the basic skin color data based on the color-balanced basic image data for the skin area,
A method for estimating the level of bilirubin in a patient being generated. The method according to claim 1,
Wherein the color correction target comprises a plurality of normalized color gamuts comprising a white region. The method of claim 4,
Wherein the normalized color gamut includes a black region, a gray region, a light yellow region, a cyan region, a magenta region, a yellow region, and a dark brown region. The method of claim 4,
Wherein the color correction target at least partially defines an aperture configured to expose the skin region to enable capturing of image data for the skin region. The method of claim 6,
Wherein the normalized color gamut is disposed in a known array surrounding the aperture. The method of claim 6,
Wherein the step of generating color-balanced image data for the skin region comprises:
Processing the received image data to identify a subset of image data corresponding to the exposed skin region and a subset of the image data corresponding to the white region;
Processing the white area data to determine an observed color value for the white area; And
And generating color-balanced image data for the exposed skin region based on the observed color value for the white region. The method of claim 8,
Wherein the observed color values for the white region include red, green, and blue (RGB) color space values. The method according to claim 1,
The color-balanced image data for the skin region comprises RGB color space data;
Wherein the method further comprises converting the RGB color space data to the at least one other color space to produce color-balanced image data of the exposed skin region for at least one other color space. A method for estimating the level of bilirubin in a subject. The method of claim 10,
Wherein the at least one other color space comprises: (a) a cyan, magenta, yellow and black (CMYK) color space; (b) a YCbCr color space; Or (c) the Lab color space. The method of claim 10,
Said at least one other color space comprises (a) a cyan, magenta, yellow (CMY) color space; (b) a YCbCr color space; Or (c) the Lab color space. The method of claim 11,
Wherein the received image data comprises an image obtained using flash illumination and an image obtained without using flash illumination. 14. The method of claim 13,
Wherein the step of estimating the bilirubin level comprises:
Processing a plurality of normalized achromatic and chromatic features to select a first estimated range of the bilirubin level from a range of one of a plurality of different bilirubin ranges; And
And processing the feature using an approach based on a selected first estimated range of the bilirubin level to produce a final estimate of the bilirubin level. 15. The method of claim 14,
Wherein the plurality of different bilirubin ranges comprises a low range, an intermediate range, and a high range. 15. The method of claim 14,
Wherein the plurality of features comprises a selected color value of a skin region for a plurality of different color spaces. 15. The method of claim 14, wherein the plurality of features comprises calculation of a color gradient across the skin region. 15. The method of claim 14,
Wherein the step of processing the feature to select a first estimated range of the bilirubin level comprises: (a) linear regression; (b) encapsulated k-nearest neighbor regression; (c) lasso regression, (d) LARS regression, (e) elastic net regression, (f) support vector regression using linear kernels, regression analysis that includes at least one of: (g) a support vector regression that assigns high weights to higher-rated bilirubin values, and (h) a random forest regression analysis. ≪ / RTI > wherein the level of bilirubin in the patient is measured. 15. The method of claim 14,
(B) encapsulated k-nearest neighbor regression analysis, (c) lacrosse regression analysis, (d) linear regression analysis, Support vector regression analysis using a linear kernel, (g) support vector regression analysis assigning high weights to high-grade bilirubin values, and (h) random forest regression. And performing a series of regression analyzes including analysis of the patient's serum bilirubin level. 15. The method of claim 14,
Wherein the step of using a processing approach based on a selected first estimated range of the bilirubin level comprises, as a feature for a final random forest regression analysis, the selected first estimated range of the bilirubin level and the plurality of normalized achromatic and chromatic characteristics And performing the final random forest regression analysis using the final random regression analysis. The method according to claim 1,
Wherein estimating the bilirubin level comprises determining a color space value for the skin of the patient and using a processing approach based on the determined patient's skin color space value to estimate the bilirubin level. A method for estimating the level of bilirubin in a subject. A mobile device configured to estimate a level of bilirubin in a patient,
A camera operable to capture image data for a field of view;
A processor operatively coupled to the camera; And
A data storage device operably coupled to the processor and storing instructions;
Wherein the instructions, when executed by the processor, cause the processor to:
Receive image data for an image including a color correction target and an area of a skin of a patient captured by the camera;
Generate color-balanced image data for the skin region based on the skin region and a subset of the image data corresponding to the color correction target;
And to estimate the patient's bilirubin level based on the color-balanced image data for the skin region. 23. The method of claim 22,
Wherein the color correction target at least partially defines an aperture configured to expose the skin region to enable capturing of image data for the skin region and includes a plurality of normalized color regions including a white region ;
Wherein the instructions cause the processor to:
Processing the received image data to identify a subset of the image data corresponding to the exposed skin region and a subset of the image data corresponding to the white region;
To process the white area data to determine an observed color value for the white area;
To generate color-balanced RGB image data for the exposed skin region based on the observed color values for the white region;
To generate color-balanced image data of the exposed skin region for at least one other color space by converting the color balanced RGB image data to the at least one other color space;
To process a plurality of normalized achromatic and chromatic features to select a first estimated range of the bilirubin level from one of a plurality of different bilirubin ranges;
And to process the feature using an approach based on a selected first estimated range of the bilirubin level to produce a final estimate of the bilirubin level. 23. The method of claim 22,
Wherein the color correction target at least partially defines an aperture configured to expose the skin region to enable capturing of image data for the skin region and includes a plurality of normalized color regions including a white region ;
Wherein the instructions cause the processor to:
Processing the received image data to identify a subset of the image data corresponding to the exposed skin region and a subset of the image data corresponding to the white region;
To process the white area data to determine an observed color value for the white area;
To generate color-balanced RGB image data for the exposed skin region based on the observed color values for the white region;
To generate color-balanced image data of the exposed skin region for at least one other color space by converting the color balanced RGB image data to the at least one other color space;
To process color-balanced image data for the exposed skin region to determine a skin color for the patient;
And to process the plurality of normalized achromatic and chromatic features using the determined skin color based approach to estimate the bilirubin level. The method of claim 23 or 24,
Further comprising a flash unit operable to selectively illuminate a field of view;
Wherein the received image data processed to estimate the bilirubin level comprises a captured image with a field of view illuminated by the flash unit and a captured image with a field of view not illuminated by a flash unit, . CLAIMS 1. A method for estimating a level of bilirubin in a patient,
Receiving image data from a mobile device for an image including a color correction target and a skin area of a patient;
Generating, via the one or more processors, color-balanced image data for the skin region based on the skin region and a subset of the image data corresponding to the color correction target;
Estimating a bilirubin level of the patient based on color-balanced image data for the skin region, via the one or more processors; And
And transmitting the estimated bilirubin level to the mobile device. 26. The method of claim 25,
Wherein at least one of receiving the image data and transmitting the estimated bilirubin level is performed using text messaging service (SMS) text messaging.

Images