LesionAir: An Automated, Low-Cost Vision-Based Skin Cancer Diagnostic Tool

More than 3,000,000 nonmelanoma skin cancers [1] and 140,000 melanoma skin cancers [2] afflict people every year in the U.S.; this translates to one out of every three cancers [3]. One out of five Americans will develop skin cancer at some point in their life [4].

Identifying early-stage skin cancer before it has metastasized is critical, as prompt excision of the lesion nearly guarantees the patient's recovery [5,6]. Fortunately, skin cancer generally develops in the outermost layers of skin, making a possible malignant tumor visible in the early stage when treatment is likely to be most successful.

Tissue differentiation is critical for identifying cancerous tissue; however, without performing a biopsy, it is challenging to characterize tissue types within the same tissue structure. In most clinical settings, practitioners analyze tissue using nothing more than sight and touch. Fidelity is limited due to resolution and sensitivity constraints of the practitioner; this method is only effective when performed by experienced dermatologists [7].

The ability to accurately diagnose melanoma varies significantly between specialists (dermatologists) and nonspecialists (primary care physicians) with more experience leading to higher diagnostic accuracy. Dermatologists diagnostic accuracy ranges from 25% [8] to 89% [9], whereas nonspecialist's accuracy ranges from 4% to 80% [8–10]. This is worrisome as the majority of patients with suspicious skin lesions typically visit their primary care physician before seeing a dermatologist. Due to these facts, visual diagnosis is commonly associated with a risk of missed melanomas (false negative) and unnecessary biopsies (false positive).

Because of these limitations, the gold standard for diagnosis has been invasive biopsy and excision, followed by histological and pathological examination. Studies have found that the ratio of biopsies of benign lesions to malignant ones can vary from five to one to as high as 500 to one [11], while at the same time, one-third of skin cancers are missed [12]; this shows current practices have neither specificity nor sensitivity, and a better approach is required.

As technology has advanced, imaging systems using various modalities have been developed to aid in the diagnosis of melanoma. The most basic of these tools is a dermoscope; a specialized microscope that enables dermatologists to see subsurface features. Dermoscopes have been shown to increase diagnostic accuracy [13], but less than 60% of practitioners use them [11].

More advanced optical methods use imaging probes that give a range of views, such as total body photography [14], or increased resolutions, such as confocal microscopy [15] or optical coherence tomography [16]. Some methods incorporate multispectral imaging and leverage the chromophores to quantify melanin, blood, and collagen content within a lesion [17].

Research in this area has focused primarily on optical technologies to automate and assist the dermatologist in diagnosing skin lesions; however, current devices are complex, expensive, and designed to be used by a trained dermatologist only as a means of verifying the initial diagnosis. Most methods focus on measuring optical responses, which vary with stimulation and have limited reliability and, thus, are not applicable across all clinical populations. Technology adoption has, thus, been slow due to these issues [11].

An alternative approach exploits the mechanical properties of skin to aid in medical diagnoses. Cancer causes noticeable variations in the elastic properties of tissue [18]. The evaluation of the elastic properties of soft tissue has been performed manually via palpation to detect various types of cancers for decades. While effective, manual palpation has limited fidelity and is dependent on the practitioner. Automating stiffness measurement has the potential to increase sensitivity and consistency over current methods, turning a subjective test into a repeatable, objective test that enables cancer to be diagnosed sooner resulting in a greater chance of survival. Elastograms—or tissue stiffness maps—can be formed from several different methods but are most often constructed using acousto-optics [19], ultrasonics [20], or magnetic resonance [21]. While proven effective, these systems are far too expensive and cumbersome for clinical adoption [20], which requires a portable, cost-effective method for quantifying skin stiffness.

While stiffness is a simple mechanical property that can be calculated from force and deflection, these two properties are challenging to accurately apply and measure across a three-dimensional (3D) surface using low-cost methods.

In this paper, we present a device; we call LesionAir, a low-cost tissue characterization device for application to skin cancer detection. The device determines tissue stiffness by applying a gentle vacuum to the tissue region and measures the full-field tissue deformation through structured light triangulation. The device's camera system also enables morphological clinical features (such as asymmetry, border irregularity, color variation, and diameter—commonly referred to by the ABCD acronym) to be quantified optically. We hypothesize that future analytical techniques can combine the morphology of a lesion, its stiffness, and demographic data about the patient to provide a multimodal approach to the diagnosis of skin cancer.

LesionAir (Fig. 1) consists of two integrated systems to apply force and measure deflection of tissue: a small vacuum pump and pressure sensor to create and maintain controllable negative pressure; and a simple, low-cost structured light system to create a deflection map of the tissue as the pressure is modulated. This initial prototype connects to a computer which controls these systems, collects data, and formulates the results. These components reside in a plastic, light-tight, airtight housing that provides a self-contained, pressurized environment.

LesionAir's vision system comprises a ring light, structured-light projector, and camera. The ring light evenly illuminates the scene to provide adequate lighting for the camera to capture standard-lit images. The projector shines a fixed structured light pattern onto the tissue that the camera uses for triangulation.

The prototype device captures standard- and structured-lit images using Allied Vision's (Newburyport, MA) Manta G-145B camera with a 16 mm f/1.4 Fujinon lens. This monochrome 1.4 MP camera can capture images at 30 fps.

For standard-lit images, a generic 48-light-emitting diode (LED) macro ring light evenly illuminates the scene. The ring light, along with LesionAir's light-tight enclosure, ensures that all of the images taken will have standardized lighting and no shadows.

The structured light projector consists of a 5 W white LED, gobo plate, and an off-the-shelf 16 mm camera lens used to focus the pattern onto the patient. The gobo was fabricated from a sheet of 0.1 mm thick 304 stainless steel and is comprised of a 40 × 28 array of 40 μm diameter dots with 0.1125 mm spacing between the dots to form a 4.5 mm × 3.15 mm rectangle.

LesionAir's force application system consists of a pneumatic pump, solid state pressure sensor, and a flexible seal that acts as a junction between the device and the skin.

From finite element analysis (FEA) simulations based on modeled skin elasticity, we chose to apply a total negative pressure of 100 mbar over a 6 cm diameter area. This is certainly more force than what is typically used in manual palpation, but applying this much force causes significant changes to skin deflection, which helps mitigate noisy structured light triangulation data.

It is necessary that this pressure does not cause any pain or discomfort. The reported pain threshold and pain tolerance levels for humans is 4 bar and 5–11 bar, respectively [22]. A 100 mbar pressure is significantly below these values and also allows the user to immediately remove the device if there were to be any discomfort.

A KNF Neuberger (Trenton, NJ) NMP 05 B microdiaphragm gas sampling pump was selected to evacuate the enclosure. This pump can produce a 500 mbar vacuum at a flow rate of 0.3 L/min.

The internal pressure is monitored by an Omega PX209-30VAC5V solid-state pressure transducer with a range of 0 to −1000 mbar.

The custom seal between the device and the patient resembles a bellows profile and is capable of forming a seal on a variety of contours to enable the device to be used on different parts of the human body. The seal was fabricated from a biocompatible silicone rubber material that will not irritate the skin. Oil- or water-based lubricants can be applied to the sealing interface to help prevent any air leaks, and alcohol can be used to sterilize the seal. Significantly curved or highly variable surfaces can prevent a pressure-tight seal due to the limited compliance of the seal material.

While the camera interfaces with the computer using Ethernet, the lighting system, vacuum pump, and pressure sensor communicate with a personal computer using Treehopper, a low-cost USB-based open-source platform designed to connect computers, smartphones, and tablets to embedded electronics for signal acquisition, control, and interfacing.¹ Treehopper was selected because it can interface directly with matlab scripts (as well as programs written in Android and C#) without requiring any embedded programming. A custom printed circuit board integrates a Treehopper-enabled microcontroller, inputs from the pressure sensor, output drivers for the ring light and vacuum pump, and a constant-current boost converter to drive the projector LED. A button was added to the system to allow the user to trigger the software without using a keyboard or mouse.

The enclosure is 3D printed from a high-impact acrylonitrile butadiene styrene-like material (Somos NeXt). Through-wall connections are sealed using gaskets and O-rings. Internal ribs provide optimized enclosure strength to material volume ratio. A 6 cm aperture at the bottom of the enclosure exposes the tissue region of interest.

The enclosure serves several purposes. It rigidly holds the camera and projector systems relative to each other and the aperture, enabling a one-time calibration to be valid for all trials. It provides a pressure-regulated, self-contained internal chamber that is light-tight so that all recorded images are standardized. It also acts as a modular platform, so that components can be swapped out if necessary.

Without any attachments, the enclosure measures 14 × 11 × 25 cm. In its final form with everything attached, the device is 26 × 13 × 26 cm and weighs approximately 1.5 kg. The projector is offset to the side of the camera by approximately 8.4 cm and tilted at a 45 deg angle to maximize depth resolution. The estimated cost for the entire bill of materials of the device is $4000.

In this section, we present the procedural flow for a typical use case of LesionAir. For preparation, a clinician identifies a suspect lesion and sanitizes the region to be measured. The caregiver records a timestamp along with patient information (patient identification code, age, sex, ethnicity, and skin tone) and lesion information (location and type). For a clinical trial, the patient information is associated with an anonymous patient identification code, so the patient cannot be personally identified.

The clinician centers the device over the region of interest and places it against the skin. The data acquisition process is started by pressing the button. Once completed, the device can be removed. This process is performed twice—once for the lesion and once for assumed healthy tissue on a region of the body symmetric to the location of the lesion. The system compares the lesion and nonlesion regions to produce relative stiffness measurements. A relative measurement removes the mechanical boundary condition effects due to the seal compression from vacuum-induced forces.

The data acquisition process uses a graphical user interface so the user can see the camera's view during each stage of operation. Once the process is initiated, a series of image capture sequences begin. For each image capture sequence, the system:

1. (1)

   Records the pressure inside the chamber

2. (2)

   Turns on the structured-light projector

3. (3)

   Captures a structured-lit image

4. (4)

   Turns off the structured-light projector

5. (5)

   Turns on the ring light

6. (6)

   Captures a standard-lit image

7. (7)

   Turns off the ring light

LesionAir is set to capture images for pressures of 0–100 mbar in 20 mbar increments. After each image capture sequence, the pump is turned on until the pressure decreases 20 mbar and the image acquisition cycle begins again. This is repeated until the system captures data for all pressure levels.

When finished, the software saves the data and shows all images to the user, who ensures there are no errors before proceeding with the data processing.

The system processes the data in three phases: morphology quantification, scene reconstruction, and stiffness quantification. The initial standard-lit image is segmented to differentiate the lesion from surrounding tissue using filtering and dynamic localized thresholding. The ABCDs are then quantified to assess the morphology. To determine asymmetry, the segmented lesion's centroid is determined and the areas of the four quadrants around the centroid are compared. Border irregularity is quantified by finding the number of pixels on the boundary of the segmented lesion and comparing them to the number of pixels in a perfect ellipse determined by the major and minor diameter of the segmented lesion. Color variation is the range of values of the grayscale intensity image. The diameter of the lesion is calculated through the area of the segmented lesion.

The algorithms are designed such that the metrics are larger for an atypical morphology. A perfectly round, symmetrical lesion with uniform color will produce a zero for the A, B, and C metrics. Diameter is an absolute measurement. The research community considers a lesion with a diameter greater than 6 mm suspicious for melanoma [5].

Next, the surface profile at each pressure state is reconstructed. For each set of structured light images, the projected pattern is differentiated from the background tissue using filtering and dynamic localized thresholding. The system finds the centroids of the dots in the pattern using a connected components algorithm. The keypoints of the light pattern is ordered using the algorithm described in Ref. [23]. The camera and projector calibration matrices are then used to triangulate and create sparse reconstructions of the scene. The camera and projection matrices were determined through a calibration procedure that used Camera Calibration Toolbox for matlab [24] to determine the camera matrix; ProCamCalib [25] was integrated with a custom segmentation and ordering algorithm to determine the projection matrix.

The sparse reconstruction point clouds are then bicubic-interpolated to surfaces. Because bicubic interpolation does not handle discontinuous, limited-domain data, the surfaces must be carefully cropped in to provide accurate data.

Once the structured light images are converted into surfaces, the stiffness quantification process can begin. The ambient pressure surface for the lesion and symmetric healthy region are loaded and used as a reference for the other measurements. The next surfaces and corresponding pressures are then loaded and the pressure differential is calculated.

The script determines the overlapping data region between all of the images and crops any data outside of this region. The surfaces can only be compared where there the data is known. At high pressures, and consequently high strains, the tissue is deformed significantly, so only a small part of the high-pressure surface overlaps with the original undeflected surface, causing the map to shrink.

Stiffness is determined by dividing the pressure differential by the calculated surface displacement. The pressure differential for the lesion and nonlesion images will be slightly different, so this conversion normalizes the values, so they can be compared. While not a standards traceable value, the normalized stiffness is a relative metric created specifically for the patient. Tissue elasticity is proportional to the normalized stiffness, where the stiffness is also normalized to remove boundary condition effects by comparing the lesion stiffness map to the patient's corresponding nonlesion stiffness map.

The elasticity of the lesion can thus be qualitatively compared with the healthy surrounding tissue through analysis of the normalized stiffness map. In the future, a machine-learning algorithm could compare these elasticity measurements, ABCDs, and patient demographic information to suggest a diagnosis. This information can be used to suggest immediate treatment or it can be stored and compared to a later measurement to determine if the lesion is evolving, and therefore, potentially cancerous.

Benchtop experiments were used to evaluate LesionAir's measurement accuracy, then the system accuracy was quantified on a characterized object.

A precise curved surface and a precise flat surface were both measured to gauge the extremes. Accuracy was determined by comparing the results of the reprojection depth map to measurements made on the same surfaces by a coordinate measuring machine.

Repeatability was quantified by measuring the same surface ten different times with a 10 s delay between the measurements. The device and measured object were rigidly fixed so they wouldn't move between measurements. The measured object was again the precision ground flat plate.

The system efficacy was evaluated using a three-layer custom phantom tissue model fabricated from synthetic rubber [26].

LesionAir was clinically evaluated in a proof of concept study at the Dermatology Laser Center in San Marcos, TX. The general design of the study was to identify a patient with a potentially malignant lesion and to assess that lesion with LesionAir prior to biopsy or excision. Along with the standard of care, the excised lesion was subjected to histopathological review so that a dermatopathologist could render the diagnosis.

The clinical protocol used to perform testing with LesionAir was approved by the Committee on the Use of Humans as Experimental Subjects at the Massachusetts Institute of Technology. The study was approved under Committee on the Use of Humans as Experimental Subjects Protocol No. 1511310864 and the device and its use were determined to be a nonsignificant risk.

A trained dermatologist working in a high-volume general medical and surgical dermatology clinic identified a potential patient to participate in the study. The patient was selected from those individuals who presented for routine dermatologic visits.

For the initial accuracy assessment experiments, a common billiard ball was chosen as a precise curved reference surface and a precision ground flat plate was chosen as the precise flat surface. Both were measured by a Zeiss (Oberkochen, Germany) Eclipse C99/II coordinate measuring machine to not deviate from true shape by more than 0.025 mm.

The structured light measurement of the billiard ball resulted in a maximum measured error of 0.293 mm, 0.085 mm root-mean-square (RMS) error, and 0.051 mm standard deviation. A heat map and cumulative distribution function are shown in Figs. 2 and 3, respectively. The heat map shows the geometric distribution of error across the measured area. This was used as a diagnostic tool to identify where the largest errors were occurring. Manufacturing defects and optical distortion are likely causes of errors. The cumulative distribution function of these results reveals that approximately 50% of the data points have less than 0.05 mm error and 90% of the points have less than 0.15 mm of error.

The measurements of the precision ground flat plate resulted in a maximum measured error of 0.282 mm, 0.062 mm RMS error, and 0.041 mm standard deviation. The cumulative distribution function of these results is better than the billiard ball, with approximately 90% of the points with less than 0.1 mm of error.

It is difficult to determine the acceptable range of error for this method as the stiffness of human tissue can vary up to four orders of magnitude [27]; thus, the best way to determine the required design parameters is through an experimental study.

The results of the repeatability experiment were used to quantify the maximum and average deviation in the three principle Cartesian directions, along with the total magnitude of deviation. The total average deviation was 0.0112 mm with a maximum deviation of 0.0274 mm. Again, the results were deemed to be acceptable for the application.

The phantom models were also measured five times using LesionAir. The average device measurement deviated from FEA predictions by 10.0%, 57.8%, and 18.6% for the models with an exposed lesion, embedded lesion, and without a lesion, respectively [26].

Full-field normalized stiffness maps were also created for the exposed and embedded phantom models. Qualitative analysis of the stiffness maps clearly shows a stiffer mass in the center of the image. The mass has an approximate diameter of 1.5 cm, matching the specification.

These results indicated that the device should be able to successfully map variations in skin stiffness and that the stiffness map can be used to identify the boundaries of a lesion.

The selected pilot study patient was a 60 year old Caucasian male with Fitzpatrick type I-II skin. This patient was new to the clinic and presented for a full skin examination. It was noted that the patient had a lesion along the right jawline measuring approximately 8–10 mm in diameter. From the patient history, the patient reported that this lesion was relatively new over the last year.

Clinical examination of the lesion revealed it to be a skin-colored papule with some hyperkeratosis over the center. The lesion felt fairly indurated throughout without any evidence of hyperpigmentation. The clinical presentation of this lesion was consistent with a basal cell carcinoma, while dermatopathology diagnosed the lesion as a stage 3 malignant melanoma.

Normalized stiffness maps, an example of which is shown next to a clinical image of the lesion in Fig. 4, were created from the acquired data and overlaid onto the standard-lit images for qualitative evaluation. The normalized stiffness mapping consistently indicates a well-defined region of increased relative stiffness. This increased stiffness is virtually uniform throughout the entire lesion. This is completely consistent with the clinical assessment of induration of lesion and the histologic confirmation of a solid nodule of fibrotic tumor mass. While the lesion position was not quantitatively verified, the mapping appears to match up very well with the lesion position.

The results of the automated ABCD morphology quantification for the patient are shown in Fig. 5. The segmentation algorithm had difficulty correctly segmenting the lesion, since its appearance was not that of a typical melanoma; however, the diameter measurement was larger than 6 mm, consistent with a high cancer probability.

LesionAir accomplished the preliminary goals of the research by demonstrating an automated, repeatable, objective process to detect relative changes in the stiffness of tissue. The presented platform is simple-to-use, economical, and portable, while still providing accurate results, as demonstrated by its use in a clinical setting.

The pilot study showed that LesionAir is capable of differentiating between stiff tissue and normal tissue. The sample size of the pilot study was too small to achieve results with statistical significance, but the promising results warrant a continued investigation of the efficacy of the device and method. Subsequent studies will help determine if the device is able to definitively confirm a cancer diagnosis, or just provide additional information about a lesion that a specialist can use to make a diagnostic decision. In particular, since it takes so long to see an experienced dermatologist, trained clinician use of LesionAir could help to better triage large numbers of patients with legions.

The method described in this research has shown promise as a diagnostic tool, but additional development and testing are required as discussed earlier. Most importantly, a larger scale, longitudinal clinical trial is required. Given the four orders of magnitude that skin stiffness varies by, modeling acceptable deflection measurement error is nearly impossible; such a clinical trial would provide the needed empirical data necessary to validate the resolving capability of the proposed device. The initial intent of the method was not to evaluate the relative stiffness of a lesion versus an area of healthy tissue for an immediate diagnosis, but instead to look at the evolution of a lesion. Significant changes in stiffness and morphology over time correlate directly with an increased chance of skin cancer. A multiyear study would require lesions to be measured daily, weekly, or monthly to verify how fast they evolve and how soon this change can be identified.

A large dataset from a longitudinal study would also be beneficial in creating a robust classification algorithm. It is hypothesized that a machine learning neural network classifier would be ideal for this application. The neural network could be trained using the ABCD metrics and normalized stiffness maps as inputs. This process would provide an algorithm to give all subsequent measured lesions a score on how likely they are to be cancerous.

Finally, the method needs to be miniaturized to the initially proposed smartphone form factor with real-time processing capabilities. The device was designed with miniaturization in mind, such that all of the components necessary are already included on the phone, or easy to add through a simple hardware attachment.

• National Science Foundation (Grant No. 1122374).