TWI804506B - Systems and methods for analyzing cutaneous conditions - Google Patents

Abstract

The following disclosure discusses systems and methods of detecting and analyzing cutaneous conditions. According to one embodiment, a 2D image of the cutaneous condition and a set of 3D point clouds associated with the 2D image are captured using an image capturing device. The 2D image and the set of 3D point clouds are sent to a computing device. The computing device generates a 3D surface according to the set of 3D point clouds. Subsequently, the computing device receives a depth map for the 2D image based upon the 3D surface from another computing device such that the depth map comprises depth data for each pixel of the 2D image. The cutaneous condition may then be measured and analyzed based upon the depth map using the computing device.

Description

Systems and methods for analyzing skin condition

The present invention relates to processing two-dimensional (2D) images along with three-dimensional (3D) data, and more particularly to systems and methods for analyzing skin conditions for diagnosis and treatment.

The approaches described in this section could be practiced, but are not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.

In dermatological applications, the most common method for documenting and diagnosing a skin condition is to take a photograph of the condition with a scale visible in the photograph (typically, in the photograph taken, there is a scale next to the condition ). This makes the image acquisition process slow and cumbersome. Furthermore, since there is only one measurement reference in the image, surface irregularities and camera angle variations result in lower measurement accuracy.

Some systems are capable of capturing a complete picture of a subject's skin with 2D and 3D imaging devices, but these devices are often large and expensive, with limited measurement options. These devices often rely heavily on gestalt perception and still require careful examination by a clinician or dermatologist.

Embodiments described herein include a method for analyzing a skin condition. It should be appreciated that the specific examples can be implemented in numerous ways, such as a process, an apparatus, a system, device or method. Several specific examples are described below.

In one embodiment, a method of analyzing a skin condition is described. The method may include an operation of receiving, using a computing device, a two-dimensional (2D) image of the skin condition and a set of three-dimensional (3D) point clouds associated with the 2D image. The method may further comprise an operation of using the computing device to generate a 3D surface from the set of 3D point clouds. The method may also include an operation of receiving, with the computing device, a depth map for the 2D image based on the 3D surface such that the depth map includes a depth value for each pixel in the 2D image. The method may include an operation of using the computing device to analyze the skin condition based on the 2D image and the depth map. In one embodiment, each 3D point in each 3D point cloud of the set of 3D point clouds corresponds to at least one pixel of the 2D image.

In an embodiment, the depth map is generated using a second computing device. In an embodiment, the method may further include an operation of storing the depth map in a memory device communicatively coupled to the second computing device. In one embodiment, the depth map can be calculated by using the second computing device to implement a ray casting algorithm based on the 2D image and the 3D surface. Alternatively, the depth map can be calculated by using the second computing device to implement a ray tracing algorithm based on the 2D image and the 3D surface.

In one embodiment, the 2D image and the set of 3D point clouds are captured using an image capture device. In an embodiment, the method may further comprise the operation of resolving the image capture device along at least one axis about a subject having the skin condition to capture a set of 2D images and an associated set of 3D point clouds. In another embodiment, the method may include an operation of storing the 2D image and the set of 3D point clouds in a memory device communicatively coupled to the image capture device.

In a specific example, the image capture device may include a two-dimensional (2D) camera. The method may include using the computing device to generate a set of pixel sizes for each pixel of the 2D image based on a depth value for each pixel, an angle of a horizontal field of view of the 2D camera, and an angle of a vertical field of view of the 2D camera one of the operations. In a specific example, the 2D camera can capture a color of the skin condition 2D imagery. Alternatively, the 2D camera can capture a monochromatic 2D image of the skin condition. In one embodiment, the 2D camera can have a resolution of at least 8 megapixels.

In one embodiment, the image capture device may include a three-dimensional (3D) device. In a specific example, the 3D device can be a 3D scanner. Alternatively, the 3D device may be a 3D camera such that the 3D camera captures the set of 3D point clouds corresponding to the 2D image.

In one embodiment, the 3D surface can be an interpolated 3D surface mesh. In one embodiment, the 3D surface can outline the skin condition. In an embodiment, the 3D surface can be generated by using the computing device to filter out a set of aberrations in at least one of the set of 3D point clouds. In an embodiment, the 3D surface can be generated using the computing device using at least one interpolation algorithm.

In a specific example, analyzing the skin condition may further include an operation of measuring a magnitude of the skin condition using the computing device.

In an embodiment, analyzing the skin condition may further comprise an operation of determining a variance of the magnitude of the skin condition using the computing device.

In a specific example, analyzing the skin condition may further comprise an operation of using the second computing device to automatically diagnose the skin condition according to the magnitude of the skin condition.

In one embodiment, a system for analyzing a skin condition is disclosed. The system may include an image capture device that captures a two-dimensional (2D) image of the skin condition and a set of three-dimensional (3D) point clouds associated with the 2D image. The system can further include a computing device communicatively coupled to the image capture device and a second computing device such that the computing device can receive the 2D image and the set of 3D point clouds from the image capture device. The computing device can also generate a 3D surface from the set of 3D point clouds. The computing device may further receive a depth map for the 2D image based on the 3D surface from the second computing device, such that the depth map includes depth data for each pixel of the 2D image. The computing device can analyze the skin condition based on the depth map.

In a specific example, the image capturing device may further include a 2D camera, so that the 2D camera captures the 2D image. In a specific example, the image capture device may further include a 3D device, such that the 3D device captures the set of 3D point clouds. In a specific example, the image capture device may further include a battery such that the battery supplies power to the image capture device. Similarly, the image capture device may also include a flash device, such that the flash device includes at least one light emitting diode. The image capture device may further include a touch screen display.

In one embodiment, the system can include at least one storage device communicatively coupled to at least one of the second computing device and the computing device, such that the storage device stores the depth map. In one embodiment, the at least one storage device includes at least one of the group consisting of: an internal hard drive, an external hard drive, a Universal Serial Bus (USB) drive , a solid state hard drive and a network attached storage device.

In one embodiment, the computing device and the second computing device can include at least one of the group consisting of: a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) and Advanced Reduced Instruction Set Computing Machine (ARM).

In an embodiment, the computing device is communicatively coupled to the image capture device and the second computing device via at least one network connection. In one embodiment, the network connection can be Wi-Fi, Bluetooth, Ethernet, fiber optic connection, infrared, near field communication or coaxial cable connection.

In one embodiment, a system for automatically detecting a skin condition is disclosed. The system can include an image capture device that automatically captures a set of two-dimensional (2D) images of a subject and a set of three-dimensional (3D) point clouds associated with each 2D image of the set of 2D images. The system may also include a computing device communicatively coupled to the image capture device and a second computing device such that the computing device automatically receives the set of 2D images of the subject and the Group 3D point cloud. The computing device can also automatically generate a 3D representation of the subject based on the set of 3D point clouds. Additionally, the computing device may receive a depth map for each 2D image in the set of 2D images from the second computing device, Such that the depth map contains depth data for each pixel of each 2D image. The system may include a storage device communicatively coupled to the computing device such that the storage device automatically stores the depth map.

In a specific example, the client server may further include an analysis module, so that the analysis module can automatically generate the measurement result of the skin condition based on the depth map and the 2D image. The computing device may also automatically determine a variance of the skin condition based on comparing the status of the measurements with previously stored measurements of the skin condition. The computing device can automatically generate a diagnostic recommendation based on the variance of the skin conditions.

102: Subject

103: Skin condition

104: Image capture device

106:Computing devices

108:Processing the server

204: 2D camera

206: 3D devices

208: System Single Chip

210: Flash device

212: battery

300: Computer system

302: busbar

304: Processor

306: main memory

308: Read-only memory (ROM)

310: storage device

312: Display

314: input device

316: Cursor controller

318: communication interface

320: Network link

322: local network

324: host computer

326: Internet Service Provider (ISP)

328:Internet

330: server

502: 3D point

504: 2D pixels

506: 3D surface

508: Pixel light

602: Input collection of 3D point cloud

604: Composite 3D point cloud

606: voxel neighbor positioning

608: Filtered 3D point cloud

In the drawings: FIG. 1 illustrates an exemplary system for analyzing skin conditions according to an embodiment of the invention; FIG. 2 illustrates an exemplary image capture device according to an embodiment of the invention; FIG. An exemplary computer system of an embodiment of the invention; Figure 4 illustrates an exemplary method for analyzing skin conditions according to an embodiment of the invention; Figures 5A, 5B and 5C illustrate an embodiment of the invention Figure 6A, Figure 6B, Figure 6C, Figure 6D, Figure 6E, and Figure 6F illustrate the method for filtering a set of point clouds according to an embodiment of the present invention. Exemplary steps for data noise; FIGS. 7A, 7B and 7C illustrate exemplary steps for generating device-spatial locations of pixels of a 2D image according to an embodiment of the present invention; FIGS. 8A, 8B and 8C illustrates exemplary steps for generating pixel dimensions of pixels of a 2D image according to an embodiment of the present invention; FIGS. 9A , 9B, and 9C illustrate subscales for measuring pixels of a 2D image according to an embodiment of the present invention. Exemplary steps for the pixel area; 10 illustrates an exemplary view of an output 2D image for analyzing skin conditions according to an embodiment of the present invention.

While the inventive concepts are susceptible to various modifications and alternative forms, specific instances of these concepts have been shown by way of example in the drawings and will be described herein in detail. It should be understood, however, that there is no intention to limit the inventive concepts to the particular forms described, but on the contrary the intention is to cover all modifications, equivalents, and alternatives consistent with the scope of the invention and appended claims.

In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, that the invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.

Embodiments of the invention relate to integrated circuits. The integrated circuits may be any suitable type of integrated circuits, such as microprocessors, application specific integrated circuits, digital signal processors, memory circuits, or other integrated circuits. If desired, the integrated circuits may be programmable integrated circuits containing programmable logic circuitry. In the following description, the terms "circuitry" and "circuit" are used interchangeably.

In the drawings, specific arrangements or orderings of schematic elements, such as elements representing devices, modules, instruction blocks and data elements, may be shown for ease of description. However, those skilled in the art will understand that the specific ordering or arrangement of schematic elements in the drawings is not meant to imply a specific order or sequence of processing or separation of processes is necessary. In addition, the inclusion of an illustrative element in a drawing does not mean to imply that the element is required in all embodiments, or that in some embodiments the feature represented by the element may not be included or be combined with other elements.

Additionally, in the drawings, connecting elements such as solid or dashed lines or arrows are used to Where a connection, relationship or association is illustrated between or among two or more other illustrative elements, the absence of any such connected elements is not meant to imply that no connection, relationship or association may exist. In other words, some connections, relationships or associations between elements may not have been shown in the drawings so as not to obscure the invention. Furthermore, for ease of illustration, a single connection element may be used to represent multiple connections, relationships or associations between elements. For example, where a connected element represents the transfer of signals, data, or instructions, those skilled in the art will understand that the element may represent one or more signal paths (eg, bus bars) as may be required to effect the transfer. .

Several features are described below that can each be used independently of each other or with any combination of other features. However, any individual feature may not address any of the above-discussed problems, or may only solve one of the above-discussed problems. Some of the issues discussed above may not be adequately addressed by any of the features described herein. Although headings are provided, information related to a particular heading not found in the section with that heading may also be found elsewhere in the specification.

The following disclosure discusses systems and methods for detecting and analyzing skin conditions. In one embodiment, a method of analyzing a skin condition is described. A 2D image of the skin condition and a set of 3D point clouds associated with the 2D image are captured using an image capture device. The 2D image and the set of 3D point clouds are sent to a computing device. The computing device generates a 3D surface from the set of 3D point clouds. The 3D surface is sent to a second computing device. Subsequently, the second computing device computes a depth map of the 2D image based on the 3D surface such that the depth map includes depth data for each pixel of the 2D image. The skin condition can then be measured and analyzed based on the depth map.

As used herein, "skin condition" or "skin characteristic" refers to any medical or cosmetic condition or response (such as an allergy test or exposure response) that affects the epidermal system, that is, the skin that surrounds the body and includes Organ systems: skin, hair, nails, mucous membranes and related muscles, fat, glands and glandular activity (such as sweat and sebum) and conditions (such as dry skin, oily skin, skin temperature and symptoms such as spots, papules , nodules, vesicles, blisters, pustules, Abscesses, infections, inflammations, hives, scabs, scaling, erosions, ulcerations, atrophy, hypertrophy, poikiloderma, lichenification, moles (including melanoma and skin cancer), reactions to tests (such as allergy tests, diagnosis or testing) or other exposures associated with the medical or cosmetic condition). Conditions of the human epidermal system constitute a wide range of diseases, also known as dermatoses, as well as many non-pathological conditions. Clinically, the diagnosis of any particular skin condition is made by gathering relevant information about the presence of skin lesions, including location (e.g., arm, head, leg), symptoms (prurigo, pain), duration (acute or chronic), arrangement (isolated, generalized, circular, linear), morphology (spots, papules, vesicles) and color (red, blue, brown, black, white, yellow).

In the following disclosure, the terms "voxel", "volumetric pixel" and "3D pixel" are used interchangeably. A "voxel" or "volume pixel" as used herein refers to a value on a regular grid in three-dimensional space. Like pixels in a bitmap, a voxel itself usually does not have its location, ie, its coordinates, explicitly encoded with its value. Instead, the position of a voxel is inferred based on its position relative to other voxels, which is where the voxels form a single volume in the data structure. In one embodiment, the voxels may be based on different models, such as a latticed voxel model, a sparse voxel model, or an octo-tree voxel model.

FIG. 1 illustrates an exemplary system for analyzing skin conditions according to an embodiment of the invention. Referring now to FIG. 1 , computing device 106 receives a 2D image of a skin condition 103 affected by subject 102 and an associated 3D point cloud from image capture device 104 , and additional data from processing server 108 .

In one embodiment, subject 102 is a human being affected by a skin disorder (eg, melanoma), which manifests itself in the form of skin condition 103 (eg, cancerous mole or lesion). In a specific example, the subject 102 may be an animal, a plant or other life specimens. In yet another embodiment, the subject 102 can be a mannequin, a cadaver (human or otherwise), or a body that can be used for testing purposes. any other object. In a specific example, skin condition 103 may be a mole, injury, cut, abrasion, boil, or some other condition affecting one or more layers of the subject's skin, hair, or nails. In one embodiment, the subject 102 lies prone on a platform. In another specific example, the subject 102 stands on a platform. The platform may be an examination table, metal platform, gurney, bed, or any other structure capable of supporting the weight of subject 102 in various shapes and sizes and fabricated from different materials. In one embodiment, the platform can be mechanized or motorized, that is, the platform can be adjusted to change height (from the ground or floor the platform is resting on), orientation, inclination angle (relative to the normal ground or floor on which it rests). The motorized platform is also capable of rotation.

In one embodiment, the image capture device 104 orbits the subject 102 to capture one or more 2D images of the skin condition 103 and an associated 3D point cloud. In one embodiment, image capture device 104 is attached to a cart mounted on a track attached to a platform, and image capture device 104 orbits around subject 102 along a fixed path. Similarly, image capture device 104 can capture 2D images and 3D points of skin condition 103 by manually orbiting around subject 102 . In a specific example, the image capture device 104 is controlled by a user to revolve around the subject 102 as a robot. In one embodiment, the robot can be controlled remotely. In another embodiment, the robot can be automatically controlled and steered by computing device 106 to reduce the time to capture a set of 2D images and associated 3D point clouds. In one embodiment, image capture device 104 may be communicatively coupled to computing device 106 via Wi-Fi, Bluetooth, Near Field Communication (NFC), Ethernet cable, fiber optic cable, or some other means of transferring data .

In one embodiment, computing device 106 is similar to computer system 300 described below with respect to FIG. 3 . In a particular example, computing device 106 may be communicatively coupled to a second computing device. The second computing device may perform some computationally intensive tasks and transmit the results to computing device 106 . In one embodiment, two computing devices can be configured as a client and server system such that the computing device 106 is the client device and the other computing device is the processing server 108 . In one embodiment, processing server 108 is similar to computer system 300 described below with respect to FIG. 3 . In a specific example, processing Server 108 may be a cloud-based server for processing images and related 3D point clouds or 3D surfaces or other data related to 2D images. In a particular example, computing device 106 and processing server 108 may be one or more special purpose computing devices implementing the techniques described herein. Application-specific computing devices may be hardwired to perform these techniques, or may include digital electronics such as one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or permanently programmed Other programmable logic devices (PLDs) to implement these techniques, or may include one or more devices programmed to execute these techniques in accordance with program instructions in firmware, memory, other storage devices, or a combination General hardware processor. These special purpose computing devices can also combine custom hardwired logic, ASICs or FPGAs with custom programming to implement these technologies. A special purpose computing device may be a desktop computer system, a portable computer system, a handheld device, a network connection device, or any other device that incorporates hard-wired and/or programmed logic to implement such technologies.

Figure 2 illustrates an exemplary image capture device according to an embodiment of the present invention. Referring now to FIG. 2 , the image capture device 104 may incorporate a 2D camera 204 that is communicatively coupled to the 3D device 206 . The 2D camera 204 and the 3D device 206 may share a flash device 210 . Image capture device 104 may be powered by battery 212 and controlled or operated by SoC 208 .

In a specific example, the 2D camera 204 is a high-resolution 2D color camera. In a specific example, the 2D camera 204 is a digital camera. For example, 2D camera 204 may be a compact camera, a smartphone camera, a mirrorless camera, a digital single-lens reflex camera, an electronic viewfinder, an interchangeable lens camera, or a medium format camera. In a specific example, the 2D camera 204 may have a resolution between 8 million pixels and 2 million pixels. In a specific example, the 2D camera 204 may have more than one lens. For example, the 2D camera 204 may have a lens for capturing color images and a lens for capturing monochrome images, and the two lenses are spliced together by the SoC 208 . In another specific example, the 2D camera 204 can be an analog camera. For example, 2D camera 204 may be a movie camera or a large format camera.

In one embodiment, 2D camera 204 is communicatively coupled to 3D device 206. 3D device 206 may be a structured light depth camera utilizing infrared projection for 3D point cloud generation. The 3D device 206 can be a non-contact 3D scanner. For example, the 3D device 206 may be a time-of-flight 3D laser scanner, a triangulation-based 3D scanner, a modulated light 3D scanner, or a volumetric 3D scanner.

In one embodiment, the 2D camera 204 and the 3D device 206 are communicatively coupled to the SoC 208 . SoC 208 may be one or more application-specific computing devices as described above. In one embodiment, the SoC 208 may include microcontrollers, microprocessors, system controllers, graphics processors, memory, digital signal processors, and other integrated circuit components. In one embodiment, SoC 208 may be similar to computer system 300 described below with respect to FIG. 3 . In one embodiment, the SoC 208 is used for real-time processing of images and data obtained by the image capture device 104 . In one embodiment, SoC 208 implements an operating environment and a user interface to process and respond to feedback received from a user of image capture device 104 .

In a specific example, the battery 212 powers the image capture device 104 . The battery 212 may be a secondary battery or a rechargeable battery. For example, the battery 212 is a lithium ion battery, a lithium polymer battery or a nickel-cadmium battery. In a specific example, battery 212 is a primary battery or a non-rechargeable battery. For example, the battery 212 can be an alkaline battery, or a zinc-carbon battery. The battery 212 allows the user to wirelessly capture the set of 2D images and the set of 3D point clouds.

In one embodiment, the flash device 210 is used to illuminate the skin condition 103 . In one embodiment, the flash device 210 is an array of LED lights connected to the SoC 208 on the image capture device 104 . In another embodiment, flash device 210 is communicatively coupled to image capture device 104 , but is distinct from image capture device 104 . For example, flash device 210 may include one or more flash bulbs, electronic flash devices, high-speed flash devices, or air-gap flash devices. In one embodiment, the flash device 210 is also used to illuminate the skin condition 103 during the acquisition of the 3D point cloud. In one embodiment, the image capture device 104 can also include a touch screen display, which can be used to User control, input and feedback. The touch screen display can be a capacitive touch screen or a resistive touch screen. In one embodiment, the image capture device 104 includes a memory storage device for storing 2D images and 3D point clouds related to the skin condition 103 . The memory storage device can be a flash memory device or a non-volatile storage device, such as a Secure Digital (SD) card.

FIG. 3 illustrates a computer system 300 in which an embodiment of the present invention may be practiced. Computer system 300 includes a bus 302 or other communication mechanism for communicating information, and a hardware processor 304 coupled to bus 302 for processing information. Hardware processor 304 may be, for example, a general purpose microprocessor.

Computer system 300 also includes a main memory 306 , such as random access memory (RAM) or other dynamic storage device, coupled to bus 302 for storing information and instructions to be executed by processor 304 . Main memory 306 may also be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 304 . These instructions, when stored in a non-transitory storage medium accessible by processor 304, cause computer system 300 to function as a special-purpose machine customized to perform the operations specified in the instructions.

Computer system 300 may include a bus 302 or other communication mechanism for communicating information, and a hardware processor 304 coupled to bus 302 for processing information. Hardware processor 304 may be, for example, a general purpose microprocessor.

Computer system 300 also includes a main memory 306 , such as random access memory (RAM) or other dynamic storage device, coupled to bus 302 for storing information and instructions to be executed by processor 304 . Main memory 306 may also be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 304 . These instructions, when stored in a non-transitory storage medium accessible by processor 304, cause computer system 300 to function as a special-purpose machine customized to perform the operations specified in the instructions.

The computer system 300 further includes a read only memory (ROM) 308 or is coupled to The bus 302 is used to store other static storage devices for static information and instructions for the processor 304 . A storage device 310 such as a magnetic disk, optical disk or solid state drive is provided and coupled to the bus 302 for storing information and instructions.

Computer system 300 may be coupled via bus bar 302 to a display 312, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, light emitting diode (LED) display, or organic light emitting diode (OLED) OLED) displays for displaying information to computer users. An input device 314 including alphanumeric and other keys is coupled to bus 302 for communicating information and command selections to processor 304 . Another type of user input device is a cursor controller 316 (such as a mouse, trackball, touch-enabled display, or cursor direction keys) for communicating directional information and command selections to the processor 304 and user Cursor movement on control display 312 . This input device typically has two degrees of freedom in two axes, a first axis (eg, x) and a second axis (eg, y), which allows the device to specify a position in a plane.

According to one embodiment, the techniques herein are performed by computer system 300 in response to processor 304 executing one or more sequences of one or more instructions contained in main memory 306 . These instructions may be read into main memory 306 from another storage medium, such as storage device 310 . Execution of the sequences of instructions contained in main memory 306 causes processor 304 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions.

The term "storage media" as used herein refers to any non-transitory media that stores data and/or instructions that cause a machine to operate in a specific manner. Such storage media may include non-volatile media and/or volatile media. Non-volatile media include, for example, optical, magnetic or solid-state drives (such as storage device 310). Volatile media includes dynamic memory, such as main memory 306 . Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid state drives, magnetic tape or any other magnetic data storage medium, CD-ROM, any other optical data storage medium, Any physical media, RAM, PROM and EPROM, FLASH- EPROM, NV-RAM or any other memory chips or storage cartridges.

Storage media is distinct from, but can be used in conjunction with, transmission media. Transmission media participate in the transfer of information between storage media. Transmission media includes, for example, coaxial cables, copper wire and fiber optics, including the wires that comprise bus 302 . Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 304 for execution. For example, the instructions may initially be carried on a disk or solid state drive of the remote computer. The remote computer can load the commands into its dynamic memory and send the commands over a telephone line using a modem. A modem local to computer system 300 can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector can receive the data carried in the infrared signal and appropriate circuitry can place the data on bus 302 . Bus 302 carries the data to main memory 306, from which processor 304 retrieves and executes the instructions. The instructions received by main memory 306 may optionally be stored on storage device 310 either before or after execution by processor 304 .

The computer system 300 also includes a communication interface 318 coupled to the bus bar 302 . Communication interface 318 provides a bi-directional data communication coupling to network link 320 , which is connected to a local network 322 . For example, communication interface 318 may be an Integrated Services Digital Network (ISDN) card, cable modem, satellite modem or modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 318 may be an area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 318 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information. For example, wireless links can be implemented by using networking technologies like WiFi, Bluetooth, infrared, and near field communication (NFC), among others.

Network link 320 typically provides data communication to other data devices via one or more networks. For example, network link 320 may provide connection to host computer 324 via local network 322 Or connect to data equipment operated by an Internet Service Provider (ISP) 326. The ISP 326 in turn provides data communication services via a worldwide packet data communication network (now commonly referred to as the "Internet" 328). Local network 322 and Internet 328 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 320 and through communication interface 318 that carry the digital data to and from computer system 300 are example forms of transmission media.

Computer system 300 can send and receive data (including program code) via the network, network link 320 and communication interface 318 . In the Internet example, server 330 may transmit a requested code for the application via Internet 328 , ISP 326 , local network 322 and communication interface 318 .

Processor 304 may execute the received code upon receipt), and/or store in storage device 310 or other non-volatile storage for later execution.

Figure 4 illustrates an exemplary method for analyzing skin condition according to an embodiment of the invention. For purposes of illustrating a clear example, FIG. 4 will be discussed with respect to FIGS. 1 , 2 and 3 .

Referring now to FIG. 4, at block 402, computing device 106 receives a 2D image of skin condition 103 and a set of 3D point clouds associated with the 2D image. In one embodiment, the 2D image and the set of 3D point clouds are captured by the image capture device 104 . In one embodiment, each 3D point in each 3D point cloud of the set of 3D point clouds corresponds to at least one pixel of the 2D image. In one embodiment, when capture is triggered by the image capture device 104 , the 2D camera captures a 2D color image on the image capture device 104 and uses the SoC 208 to store the 2D color image.

In one embodiment, 3D device 206 captures multiple 3D point clouds and stores them on image capture device 104 during a capture event. Ideally, each point cloud would consist of as many points as there are pixels in the corresponding 2D image. In practice, however, only points corresponding to certain pixels are captured due to limitations of the depth imaging method due to sensor resolution constraints, projector resolution limitations, and optics. In one embodiment, the 3D device 206 acquires multiple point clouds for each individual capture event to remove noise in the data by comparing the differences between the multiple point clouds. In a specific example, the 3D device The operating frequency of 206 is determined by SoC 208 . For example, 3D device 206 may operate at a nominal frequency of sixty hertz to capture many point clouds with a minimum time interval to minimize potential misalignment caused by movement of image capture device 104 during the data acquisition process, While still providing enough data for noise filtering.

In one embodiment, image capture device 104 sends the set of 2D images and the set of 3D point clouds to computing device 106 . In one embodiment, the 2D image and 3D point cloud are sent via a high-speed wired or wireless network connection (eg, Wi-Fi, Ethernet or fiber optic connection). In one embodiment, the image capture device 104 encrypts the set of 2D images and the set of 3D point clouds, and transmits the encrypted set of 2D images and the set of 3D point clouds to the computing device 106 via a secure communication channel. For example, the image capture device 104 can encrypt the set of 2D images and the set of 3D point clouds using a cryptographic hash function. In one embodiment, image capture device 104 may compress the set of 2D images and the set of 3D point clouds before sending to computing device 106 . The compressed set of 2D images and the set of 3D point clouds can reduce resources (eg, time, bandwidth, and processing cycles) for transmitting the data to computing device 106 . Image capture device 104 may use lossy or lossless data compression techniques and compression standards, such as Motion Picture Professionals Group (MPEG) 1, 2, 4 or High Efficiency Video Coding (HEVC) H.261, H.262, H.264, H.265, Joint Photographic Experts Group (JPEG), Portable Network Graphics (PNG), Multiple Image Network Graphics (MNG) or Tagged Image File Format (TIFF).

At step 404, the computing device 106 generates a 3D surface from the set of 3D point clouds. In a specific example, the computing device 106 performs 3D point cloud filtering before generating the 3D surface. The point cloud filtering process is discussed further below with respect to FIGS. 6A-6F . In one embodiment, the 3D surface is a high resolution triangular 3D mesh comprising a set of triangles connected by common edges or corners that they derive from filtered 3D point cloud data. In one embodiment, the 3D surface is an interpolated 3D mesh.

In step 406, computing device 106 receives a depth map for the 2D image based on the 3D surface such that the depth map includes depth data for each pixel of the 2D image. in a specific In one example, computing device 106 generates a depth map. In one embodiment, the interpolated 3D mesh generated in step 404 is then used for ray casting to generate per-pixel depth data for each pixel of the 2D image. As used herein, "ray-casting" refers to a computer graphics algorithm that uses the geometric algorithm of ray tracing. The idea of ray casting is to trace a ray from the camera, one ray per pixel, and find the closest surface blocking the ray's path. In one embodiment, the depth-per-pixel data also accounts for the resolution and known camera properties of the image capture device 104 or 2D camera 204 . In another embodiment, computing device 106 offloads the generation of the depth map to processing server 108 and receives the depth map from processing server 108 via communication interface 318 and network link 320 described above with respect to FIG. 3 . In one embodiment, the 3D surface generated at step 404 is transmitted to the processing server 108 . The processing server 108 then implements a ray casting algorithm to generate per-pixel depth data for each pixel of the 2D image. The per-pixel data is then transferred to computing device 106 . In one embodiment, the processing server 108 generates per-pixel depth data for each 2D image captured during each capture event.

At step 408, the skin condition 103 is analyzed by the computing device 106 based on the depth map. In another embodiment, the skin condition 103 can also be analyzed by the processing server 108 . In one embodiment, the current measurement of skin condition 103 may be compared to older measurements of skin condition 103 stored by computing device 106 or processing server 108 for diagnostic purposes. For example, if skin condition 103 is a wound that earlier measured two centimeters wide and currently measures four centimeters wide, the clinician may suspect that skin condition 103 present on subject 102 is spreading and may require further diagnosis investigation. In one embodiment, computing device 106 or processing server 108 automatically generates recommendations based on comparing current and previously stored measurements of skin condition 103 . In one embodiment, computing device 106 or processing server may store the depth map in a storage device similar to storage device 310 described above with respect to FIG. 3 .

In one embodiment, processing server 108 may combine the 3D surfaces generated for each 2D image of each capture event to generate a 3D surface depicting a portion of subject 102's body. In one embodiment, the processing server 108 may be calibrated to automatically detect certain skin conditions. raise For example, the processing server 108 can automatically detect moles that are larger than five millimeters in size, and correct the image area in the 2D image containing the detected mole based on the camera and surface angle that provided a clean plate for image analysis.

In one embodiment, the processing server 108 can generate a malignancy score for each skin condition based on pre-programmed criteria. For example, some of the pre-programmed criteria may include: asymmetry of the skin condition 103 with respect to a particular axis, boundaries of the skin condition 103 which may indicate irregular growth, number, type and color variation of the skin condition 103, The diameter of the skin condition 103, the height of the skin condition 103 above the epithelial surface, and the evolution of the skin condition 103 over time. Those skilled in the art will appreciate that the list of pre-programmed criteria can be expanded or further defined based on evolving medical understanding regarding skin conditions and indicators of malignancy.

In one embodiment, processing server 108 generates a combined malignancy score for subject 102 based on malignancy scores for various skin conditions. In one embodiment, the various malignancy scores and combined malignancy scores are transmitted by processing server 108 to computing device 106 for further analysis by a clinician. In one embodiment, the processing server 108 compares the current various malignancy scores and combined malignancy scores with previously stored various malignancy scores and combined malignancy scores.

5A, 5B, and 5C illustrate top views of exemplary steps for generating per-pixel depth data, according to an embodiment of the invention. 5A, 5B, and 5C will be discussed with respect to FIGS. 1-4 for the purpose of illustrating a clear example.

Referring now to FIG. 5A , in one embodiment, image capture device 104 captures and stores a 2D image of subject 102 affected by skin condition 103 along with a 3D point cloud. In one embodiment, the 2D image is composed of 2D pixels 504 , and the 3D point cloud is composed of 3D points 502 . In a specific example, the image capture device 104 stores a set of 3D point clouds associated with the 2D imagery, which is then submitted to the computing device 106 for processing at a later stage.

Referring now to FIG. 5B , in one embodiment, the computing device 106 generates a 3D surface 506 using the 3D points 502 of the 3D point cloud. In one embodiment, 3D surface 506 is the interpolated surface of subject 102 3D Surface Mesh. In one embodiment, computing device 106 filters the set of point clouds of 3D points 502 before generating 3D surface 506, as described in FIGS. 6A, 6B, 6C, 6D, and 6E below. In one embodiment, computing device 106 generates a smooth surface geometry corresponding to the surface geometry of subject 102 using various interpolation techniques.

For example, the computing device 106 may interpolate the 3D surface 506 using a spline method by estimating grid cell values (by fitting the minimum curvature surface to the 3D points 502). Similarly, the inverse distance weighted (IDW) method for interpolating 3D surfaces estimates cell values by averaging the values of nearby 3D points 502 . The closer a 3D point is to the center of the cell being estimated, the more weight is given to that 3D point. Another interpolation technique that may be utilized by computing device 106 is the natural neighbor technique. This technique uses a weighted average of neighboring 3D points and creates a 3D surface 506 that does not exceed the minimum or maximum value in the 3D point cloud. Yet another interpolation technique is kriging technique. Kriging techniques involve forming weights from surrounding measured values to predict values at unmeasured locations. Those skilled in the art will appreciate that the disclosed invention may be implemented in conjunction with a wide variety of techniques for interpolating the 3D surface 506 .

Referring now to FIG. 5C , in one embodiment, computing device 106 calculates the distance of a point on 3D surface 506 corresponding to each of pixels 504 of the 2D image as viewed from the perspective of image capture device 104 . In one embodiment, computing device 106 transmits 3D surface 506 to processing server 108, and processing server 108 computes, using various techniques, each of pixels 504 corresponding to the 2D image as viewed from the perspective of image capture device 104. The distance of a point on the 3D surface 506 for one pixel. In one embodiment, the computing device 106 or the processing server 108 calculates the distance of the pixel 502 using a ray casting technique.

In one embodiment, the computing device 106 traces light rays for each pixel of the 2D image such that the light rays originate from the simulated image capture device and pass through the pixel. The ray may then intersect the 3D surface 506 and produce a depth measurement for the pixel. In one embodiment, the computing device 106 uses a ray tracing algorithm to generate depth data for each pixel of the 2D image. Those familiar with this technique should understand that The methods for calculating depth data described above are not limiting. Other surface intersection and distance measurement techniques may be used. In one embodiment, per-pixel depth data generated by computing device 106 by applying ray casting techniques may be stored in a data structure. For example, the data structure can be an array or a list. Those skilled in the art will appreciate that each pixel of depth data corresponds to an actual distance value. Based on actual distance values in units of measurement (eg, meters or inches), a value of 0.63728 would therefore correspond to 0.63728 meters or 63.728 cm. The actual distance values require no further decoding or interpolation. In one embodiment, the per-pixel depth data can be used by computing device 106 or processing server 108 for further measurement, detection and analysis of skin conditions.

6A, 6B, 6C, 6D, 6E, and 6F illustrate an exemplary method for filtering data noise in a set of point clouds, according to an embodiment of the invention. For purposes of illustrating clear examples, FIGS. 6A , 6B, 6C, 6D, 6E, and 6F will be discussed with reference to FIGS. 1-5 .

6A , 6B and 6C depict an input set 602 of 3D point clouds that may be captured by 3D device 206 of image capture device 104 . In one embodiment, each point cloud in the set of clouds is associated with a capture event and one or more pixels of the 2D image captured by the 2D camera 204 .

In one embodiment, the input set 602 of 3D point clouds is then subjected to a noise removal mechanism that utilizes voxel-based neighbor checking (similar to the natural neighbor technique) and point position averaging (similar to the above for IDW described in Figures 5A to 5C).

Referring now to FIG. 6D , in one embodiment, the computing device 106 combines the input set 602 of point clouds to produce a composite 3D point cloud 604 . In one embodiment, the generation of the composite 3D point cloud 604 is based on comparing changes in regions across the input set of point clouds 602 using voxel-based cloud partitioning across the boundaries of the input set of point clouds 602 .

Referring now to FIGS. 6E and 6F , in one embodiment, the computing device 106 generates a filtered 3D point cloud 608 by utilizing a technique of voxel neighbor positioning 606 . For example, for each voxel, the point in the voxel is determined to be valid and the average of its position is used for this region Before, a point needs to be rendered from a minimum number of point clouds. Such elimination of 3D points that may be considered data noise, since noise is typically only present in individual clouds and is probabilistically unlikely to persist across all point clouds in the input set 602 of point clouds. In one example, computing device 106 may perform additional filtering on filtered 3D point cloud 608 to remove outlier segments, smooth small noise not removed in multi-frame noise removal, and by It is more suitable for structured 3D point data in one way of triangulation. Some techniques for performing additional filtering may include plane fitting point normal estimation, simplification, and two-sided filtering, for example.

7A, 7B, and 7C illustrate exemplary steps for generating device-spatial locations of pixels of a 2D image, according to an embodiment of the invention. For purposes of illustrating a clear example, FIGS. 7A-7C will be discussed with respect to FIGS. 1-4 . As used herein, "device-space" refers to a 3D position relative to a device in a coordinate system such that the device has a position and orientation at the origin (0,0,0). For example, for FIGS. 7A-7C , the image capture device 104 is at the origin.

Referring now to FIGS. 7A-7C , in one embodiment, the per-pixel depth data or depth map generated by computing device 106 or processing server 108 in step 406 as described with respect to FIG. The server 108 calculates the size of a given pixel in the 2D image. The pixel size of the pixels capturing the skin condition 103 can be useful for analyzing the skin condition 103 .

In one embodiment, the size of a given pixel can be calculated from the depth values of the given pixel and four neighboring pixels. In one specific example, the depth values of four neighboring pixels help establish the actual or real-world positions of the four corners of the pixel. For example, referring to FIG. 7A , the size of pixel p can be calculated based on the depth values of four neighboring pixels p ₁ ^N , p ₂ ^N , p ₃ ^N and p ₄ ^N .

To calculate the device space _positions ^of pixels p , p1N _, p2N ^, p3N ^, and p4N , we first assign two _{- dimensional} ^coordinates to these pixels. For example, referring to ^FIG . 7B, pixels p , p1N , p2N ^, p3N ^, and _p4N are assigned coordinates ( _{1,1 )} , ( _0,2 ⁾ , (2,2), ( 0,0) and (2,0). Other embodiments may use other coordinates. Any arbitrary coordinates can be used as long as the same coordinate scheme is used for all pixels in the 2D image.

Referring to FIG. 7C , in a specific example, the method for calculating the size of a given pixel p can further use the known horizontal field of view Θ _H and vertical field of view Θ _V of the 2D camera 204 to calculate the distance of each pixel in 3 dimensions camera - spatial position. In one embodiment, computing device 106 or processing server 108 determines the device-space location of pixels p , p ₁ ^N , p ₂ ^N , p ₃ ^N and p ₄ ^N based on the above parameters. In one embodiment, the device-space positions of the pixels p , p ₁ ^N , p ₂ ^N , p ₃ ^N and p ₄ ^N are calculated according to the following formula:

In the _above formula, D represents the depth _of a given pixel, C represents the coordinates of the pixel as discussed above with respect to FIG. _h and _Rv represent the horizontal and vertical resolutions, respectively. Θ _H and Θ _V represent the horizontal and vertical viewing angles of the 2D camera 204 . The device-space ^positions calculated by computing device ¹⁰⁶ for pixels p , p1N , p2N ^, p3N _, and _p4N thus allow further measurements for pixel p to _be made based ^on desired area _or path measurements. Those skilled in the art will appreciate that utilizing per-pixel evaluation allows for accurate surface feature measurements independent of profile and camera or device angles. Furthermore, when calculating the size of an area covering multiple pixels (eg, the size of skin condition 103), the same process is used across all relevant pixels, where the sum of the results for each pixel represents the result for the area being measured.

8A, 8B and 8C illustrate exemplary steps for generating the pixel size of a pixel of a 2D image according to an embodiment of the present invention. Figures 8A-8C will be discussed with respect to Figures 7A-7C for the purpose of illustrating a clear example.

Referring now to FIGS. 8A-8C , in one embodiment, computing device 106 or processing server 108 calculates the pixel size of a given pixel based on the given pixel and its neighbors in FIGS. 7A-7C previously determined device-space location. Continuing the example from above, after determining the camera-space positions of pixels p , p1N , p2N ^, p3N _, and _p4N , ^computing device 106 or processing _{server 108} may use vector subtraction to generate pixel ^{- space} corners vector. As used herein, "pixel-space" refers to a 3D position relative to a corresponding 3D position of a reference pixel such that the 3D position of the reference pixel is at the origin (0,0,0).

分別指像素p及

之位置向量。參看圖8B，V _1-4指給定像素p之拐角向量。根據以下公式計算拐角向量：

Furthermore, a "corner vector" refers to the 3D position of each corner of a given pixel in pixel-space. Referring to Figure 8A, Pos and

Respectively refer to pixel p and

The position vector of . Referring to FIG. 8B, V _1-4 refers to the corner vector for a given pixel p . Corner vectors are calculated according to the following formula:

，則：

Therefore, if Pos = x ₀ , y ₀ , z ₀ and

,but:

V ₁ =( x ₁ ,y ₁ ,z ₁ - x ₀ ,y ₀ ,z ₀ )/2

V ₁ =( x ₁ - x ₀ )/2 , ( y ₁ - y ₀ )/2 , ( z ₁ - z ₀ )/2

Referring now to FIG. 8C, in one embodiment, computing device 106 or processing server 108 may use the corner vectors to compute the horizontal and vertical dimensions of a pixel. In a specific example, the dimensions are calculated according to the following formula:

Corner vectors are computed for pairs of horizontal and vertical dimensions, where h _t represents the top horizontal dimension, h _b represents the bottom horizontal dimension, v ₁ represents the left vertical dimension and v _r represents the right vertical dimension. x ₀ , y ₀ , z ₀ represent the position vector of a given pixel. In one embodiment, only two of the four sizes are required for further measurement or analysis of the skin condition 103 . For example, only an aggregated pixel width and an aggregated pixel height may be required for calculating the surface area of skin condition 103, and computing device 106 or processing server 108 may use the average of the corresponding edges to provide a single pixel width/height value combination.

9A, 9B and 9C illustrate exemplary steps for measuring sub-pixel regions of a 2D image according to an embodiment of the present invention. For purposes of illustrating a clear example, FIGS. 9A-9C will be discussed with reference to previously discussed figures. The pixel dimensions of one or more pixels generated by computing device 106 or processing server 108 as described above with respect to FIGS. 8A-8C may be used by computing device 106 or processing server 108 to measure a given pixel in a 2D image. sub-pixel area. Sub-pixel measurements of regions of the 2D image can be useful for analyzing skin conditions 103, where the skin condition 103 is not depicted by a full number of pixels, ie the skin condition 103 is included in parts of certain pixels.

Referring now to FIGS. 9A-9C , in one embodiment, a pixel-space measurement field can represent a sub-pixel area (eg, a line as depicted in FIG. 9A ), or a portion of a pixel area (eg, as in FIGS. 9B and 9C ). Figure 9C). In one embodiment, the normalized pixel edge intersections are used to derive new camera-space vectors via linear interpolation of the corner vectors discussed above with respect to FIGS. 8A-8C . In one embodiment, the length is calculated by calculating the distance between a pair of intersection point vectors. For example, in FIG. 9A, two intersection point vectors I1 and I2 are used to determine the length of the line shown in FIG _. 9A _.

及I ₁形成之三角形計算部分像素範圍。在不表示完整像素「內部」的相交三角形之情況下，使用一矩形計算其餘範圍尺寸(若需要)。舉例而言，在圖9C中，使用由相交向量I 2、

、V₃及V₄形成之矩形計算由相交向量限界的子像素區域之範圍。 Similarly, when a portion of the extent is required, the "interior" of the pixel is determined and three triangle point vectors are computed using the relevant edge intersections and inverses of trigonometric functions to compute triangle properties. For example, in Figure 9B, using the intersection vector I _2,

And the triangle formed by I ₁ calculates part of the pixel range. In the case of intersecting triangles that do not represent the "inside" of a full pixel, a rectangle is used to calculate the remaining bounds dimensions (if needed). For example, in Figure 9C, using the intersection vector I 2,

The rectangle formed by , V ₃ and V ₄ calculates the extent of the sub-pixel area bounded by the intersecting vectors.

In one embodiment, edge intersections are computed using one of the standard 2D infinite line intersection formulas given below, where resulting intersections that occur outside the normalized pixel space are discarded.

In the above formula, L ^s ₁ represents the start coordinate of line 1, and L ^e ₁ represents the end coordinate of line 1. Similarly, L ^s ₂ represents the start coordinates of line 2, and L ^e ₂ represents the end coordinates of line 2. In one embodiment, the calculations described in FIGS. The dedicated calculation and analysis module is used.

10 illustrates an exemplary view of an output 2D image for analyzing skin conditions according to an embodiment of the present invention. For purposes of illustrating a clear example, FIG. 10 will be discussed with respect to FIGS. 1-4 .

In FIG. 10, a subject 102 and at least one skin condition 103 are visible via a graphical user interface (GUI). In one embodiment, the GUI is implemented by a stand-alone application executing on computing device 106 or processing server 108 . In one embodiment, the GUI is an extension of the user interface implemented on the image capture device 104 that can be manipulated by a user of the image capture device 104 via a touch screen as described above. In one embodiment, the GUI may allow the user to interact with the final 2D image of subject 102 together with the embedded 2D image for each pixel of the 2D image by using input device 314 as described with respect to computer system 300 of FIG. 3 . In-depth data interaction. In one embodiment, the user may be able to manipulate the final 2D image of the subject 102 for measurement and analysis of the skin condition 103 . For example, a user may be able to rotate, zoom in or out, change point of view, take screenshots, measure changes to skin condition 103 . In one embodiment, the user can initiate a side-by-side comparison of the final 2D image and a previous final 2D image stored by the processing server 108 in the GUI. In one embodiment, the GUI allows the user to view the skin condition 103 at different magnifications, which is also useful for detecting other skin conditions of different sizes.

In the foregoing specification, specific examples of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than a restrictive one. The sole and exclusive indication of the scope of the invention and what the applicant intends to be the scope of the invention is: the literal scope of the collection of such claims issued from this application in the specific form in which they were issued and etc. range of effects, including any subsequent corrections. Any definitions expressly set forth herein for terms contained in these claims shall govern the meaning of such terms as used in these claims.

102‧‧‧Subjects

103‧‧‧Skin condition

104‧‧‧Image capture device

106‧‧‧Computing devices

108‧‧‧Processing the server

Claims (39)

A method of analyzing a skin condition, comprising: using a computing device to receive a two-dimensional (2D) image of the skin condition and a set of three-dimensional (3D) point clouds associated with the 2D image; The set of 3D point clouds generates a 3D surface; using a second computing device to generate a depth map for the 2D image based on the 3D surface, wherein the depth map includes a depth value for each pixel of the 2D image and using the computing device to analyze the skin condition based on the 2D image and the depth map; wherein the horizontal and vertical dimensions of each pixel on a desired area or path in the 2D image are calculated for analysis. The method as claimed in claim 1, wherein the horizontal and vertical dimensions of the desired area in the 2D image or each pixel on the path use pixels for the desired area in the 2D image or each pixel on the path -Space corner vector is calculated according to the following formula:

Among them: V ₁ , V ₂ , V ₃ , V ₄ represent the pixel-space corner vectors for a given pixel, x ₀ , y ₀ , z ₀ represent the position vectors of the given pixel, h _t represent the given pixel the top horizontal dimension of the given pixel, h _b represents the bottom horizontal dimension of the given pixel, v ₁ represents the left vertical dimension of the given pixel, v _r represents the right vertical dimension of the given pixel, wherein the desired area in the 2D image or the pixel-space corner vector of each pixel on the path is calculated using the desired area in the 2D image or the 3D camera-space position of each pixel on the path, according to the following formula:

Where: Pos represents the position vector of the given pixel,

represents the position vector of the pixel located at the corner of the given pixel, V _n represents the pixel-space vector for the given pixel, n is 1, 2, 3 or 4, wherein the desired area or path in the 2D image The 3D camera-space position of each pixel above is calculated according to the following formula:

z = D where: D represents the depth of the given pixel, C represents the coordinates of the given pixel, where C _h and C _v represent the horizontal and vertical coordinate values, R represents the resolution of the total image, where _Rh and R _v represent the horizontal and vertical resolutions respectively, and Θ _H and Θ _V represent the horizontal and vertical 2D camera viewing angles. The method according to claim 1, wherein each 3D point in each 3D point cloud of the set of 3D point clouds corresponds to at least one pixel of the 2D image. The method of claim 1, further comprising: storing the depth map in a memory device communicatively coupled to the second computing device. The method as claimed in claim 1, wherein the second computing device is used to calculate the depth map by implementing a ray-casting algorithm according to the 2D image and the 3D surface. The method of claim 1 wherein the second computing device is used to compute the depth map by implementing a ray tracing algorithm based on the 2D image and the 3D surface. The method as claimed in claim 1, wherein an image capture device is used to capture the 2D image and the set of 3D point clouds. The method of claim 7, further comprising: orbiting the image capture device along at least one axis about a subject having the skin condition to capture a set of 2D images and an associated set of 3D point clouds . The method of claim 7, further comprising storing the 2D image and the set of 3D point clouds in a memory device communicatively coupled to the image capture device. The method of claim 7, wherein the image capture device comprises a two-dimensional (2D) camera. The method as recited in claim 10, further comprising: using the computing device to generate a value for A set of pixel dimensions for each pixel of the 2D image. The method of claim 10, wherein the 2D camera captures a color 2D image of the skin condition. The method of claim 10, wherein the 2D camera captures a monochromatic 2D image of the skin condition. The method as claimed in claim 10, 11, 12 or 13, wherein the 2D camera has at least 8 megapixel resolution. The method of claim 7, wherein the image capture device comprises a three-dimensional (3D) device. The method as claimed in claim 15, wherein the 3D device is a 3D scanner. The method of claim 15, wherein the 3D device is a 3D camera, and wherein the 3D camera captures the set of 3D point clouds corresponding to the 2D image. The method of claim 1, wherein the 3D surface is an interpolated 3D surface mesh. The method of claim 1, wherein the 3D surface outlines the skin condition. The method of claim 1, wherein the 3D surface is generated using the computing device by filtering out a set of aberrations in at least one of the set of 3D point clouds. The method of claim 1, wherein the 3D surface is generated using the computing device by using at least one interpolation algorithm. The method of claim 1, wherein analyzing the skin condition further comprises using the computing device to measure a size of the skin condition. The method of claim 22, wherein analyzing the skin condition further comprises determining a variance of the magnitude of the skin condition using the computing device. The method of claim 23, wherein analyzing further comprises using the second computing device to automatically diagnose the skin condition based on the variation in the magnitude of the skin condition. A system for analyzing a skin condition, comprising: an image capture device that captures a two-dimensional (2D) image of the skin condition and a set of three-dimensional (3D) point clouds associated with the 2D image; a computing device communicatively coupled to the image capture device and a second computing device, wherein the a computing device: receiving the 2D image and the set of 3D point clouds from the image capture device; generating a 3D surface based on the set of 3D point clouds; using the second computing device to generate a depth for the 2D image based on the 3D surface map, wherein the depth map includes depth data for each pixel of the 2D image; and analyzes the skin condition based on the depth map; wherein the horizontal and vertical values of each pixel on a desired area or path in the 2D image Dimensions are calculated for analysis. The system of claim 25, wherein the horizontal and vertical dimensions of the desired area in the 2D image or each pixel on the path use pixels for the desired area in the 2D image or each pixel on the path -Space corner vector is calculated according to the following formula:

Among them: V ₁ , V ₂ , V ₃ , V ₄ represent the pixel-space corner vectors for a given pixel, x ₀ , y ₀ , z ₀ represent the position vectors of the given pixel, h _t represent the given pixel the top horizontal dimension of the given pixel, h _b represents the bottom horizontal dimension of the given pixel, v ₁ represents the left vertical dimension of the given pixel, v _r represents the right vertical dimension of the given pixel, wherein the desired area in the 2D image or the pixel-space corner vector of each pixel on the path is calculated using the desired area in the 2D image or the 3D camera-space position of each pixel on the path, according to the following formula:

Where: Pos represents the position vector of the given pixel,

represents the position vector of the pixel located at the corner of the given pixel, V _n represents the pixel-space vector for the given pixel, n is 1, 2, 3 or 4, wherein the desired area or path in the 2D image The 3D camera-space position of each pixel above is calculated according to the following formula:

z = D where: D represents the depth of the given pixel, C represents the coordinates of the given pixel, where C _h and C _v represent the horizontal and vertical coordinate values, R represents the resolution of the total image, where _Rh and R _v represent the horizontal and vertical resolutions, respectively, and Θ _H and Θ _V represent the horizontal and vertical 2D camera viewing angles. The system of claim 25, wherein the image capture device further comprises a 2D camera, wherein the 2D camera captures the 2D image. The system of claim 25, wherein the image capture device further comprises a 3D device, wherein the 3D device captures the set of 3D point clouds. The system of claim 25, wherein the image capture device further comprises a battery, wherein the battery provides power to the image capture device. The system of claim 25, wherein the image capture device further comprises a flash device, wherein the flash device comprises at least one light emitting diode. The system of claim 25, wherein the image capture device further comprises a touch screen display. The system of claim 25, further comprising at least one storage device communicatively coupled to at least one of the second computing devices, wherein the storage device stores the depth map. The system as claimed in claim 32, wherein the at least one storage device includes at least one of a group consisting of the following: an internal hard disk, an external hard disk, a Universal Serial Bus (USB ) drive machine, a solid state hard disk and a network attached storage device. The system of claim 25, wherein the computing device and the second computing device comprise at least one of the group consisting of: a microprocessor, an application specific integrated circuit (ASIC), a field Programmable Gate Array (FPGA) and Advanced Reduced Instruction Set Computing Machine (ARM). The system of claim 25, wherein the computing device is communicatively coupled to the image capture device and the second computing device via at least one network connection. The system of claim 35, wherein the at least one network connection comprises at least one of the group consisting of: Wi-Fi, Bluetooth, Ethernet, fiber optic connection, infrared, proximity Field communication or coaxial cable connection. A system for automatically detecting a skin condition, comprising: an image capture device that automatically captures a set of two-dimensional (2D) images of a subject and associated with each 2D image in the set of 2D images a set of three-dimensional (3D) point clouds; a computing device communicatively coupled to the image capture device and a second computing device, wherein the computing device: automatically receives the set of 2D images of the subject and the set of 3D point clouds for each of the 2D images; and automatically generates a 3D representation of the subject based on the set of 3D point clouds; wherein the second computing device generates a depth map for each 2D image in the set of 2D images based on the 3D rendering, wherein the depth map includes depth data for each pixel in each 2D image; and a storage device communicatively coupled to the computing device, wherein the storage device automatically stores the depth map; wherein the horizontal and vertical dimensions of each pixel on a desired area or path in the 2D image are calculated for analysis . The system of claim 37, wherein the horizontal and vertical dimensions of the desired area in the 2D image or each pixel on the path use pixels for the desired area in the 2D image or each pixel on the path - The space corner vector is calculated according to the following formula:

Among them: V ₁ , V ₂ , V ₃ , V ₄ represent the pixel-space corner vectors for a given pixel, x ₀ , y ₀ , z ₀ represent the position vectors of the given pixel, h _t represent the given pixel the top horizontal dimension of the given pixel, h _b represents the bottom horizontal dimension of the given pixel, v ₁ represents the left vertical dimension of the given pixel, v _r represents the right vertical dimension of the given pixel, wherein the desired area in the 2D image or the pixel-space corner vector of each pixel on the path is calculated using the desired area in the 2D image or the 3D camera-space position of each pixel on the path, according to the following formula:

Where: Pos represents the position vector of the given pixel,

represents the position vector of the pixel located at the corner of the given pixel, V _n represents the pixel-space vector for the given pixel, n is 1, 2, 3 or 4, wherein the desired area or path in the 2D image The 3D camera-space position of each pixel above is calculated according to the following formula:

z = D where: D represents the depth of the given pixel, C represents the coordinates of the given pixel, where C _h and C _v represent the horizontal and vertical coordinate values, R represents the resolution of the total image, where _Rh and R _v represent the horizontal and vertical resolutions, respectively, and Θ _H and Θ _V represent the horizontal and vertical 2D camera viewing angles. The system as claimed in claim 37, wherein the client server further includes an analysis module, and wherein the analysis module: automatically generates a measurement of the skin condition based on the depth map and the 2D image; based on comparing the measurement of the skin condition with previously stored measurements of the skin condition, automatically determining a variation of the skin condition; and automatically generating assessment and diagnosis outputs according to the variation of the skin condition.

Images