INTERPRETABLE DEEP LEARNING APPLIED TO FLUORESCENCE CONFOCAL MICROSCOPY FOR INTRAOPERATIVE TISSUE ASSESSMENT IN PEDIATRIC SURGICAL ONCOLOGY.
In FCM images from 141 specimens obtained from 42 children, a convolutional neural network classified malignant, benign, and healthy tissue, detecting malignant tiles with 91.35% accuracy and producing heatmaps that highlighted cellular-level architectural distortion.
Open original publication →What the AI sees
In FCM images from 141 specimens obtained from 42 children, a convolutional neural network classified malignant, benign, and healthy tissue, detecting malignant tiles with 91.35% accuracy and producing heatmaps that highlighted cellular-level architectural distortion.
Research significance
The evidence supports feasibility for automated, spatially interpretable classification of pediatric tissue on ex vivo FCM images; it remains an inference that prospective intraoperative use could improve real-time surgical decision-making or clinical outcomes.
Source abstract
BACKGROUND: In pediatric surgical oncology, intraoperative tissue assessment is limited by small specimen size and the absence of real time histopathology. Ex vivo fluorescence confocal microscopy (FCM) provides rapid histology like imaging of fresh tissue, yet interpretation relies on expert visual assessment. We tested the hypothesis that deep learning applied directly to ex vivo FCM images enables automated, accurate, and spatially interpretable tumor detection in pediatric specimens. METHODS: FCM images were prospectively acquired during routine clinical activity from 42 children, yielding 141 surgical and biopsy specimens. For this analysis, anonymized mosaics were retrospectively collected and expert manually annotated as malignant, benign, or healthy tissue, then decomposed into 256 × 256 pixel tiles at 0.5 µm per pixel. A convolutional neural network was trained, validated, and tested on stratified datasets. Performance metrics were calculated at the tile level using accuracy, sensitivity, specificity, and F1 score. Tile level activation heatmaps were generated to localize regions influencing predictions. RESULTS: Overall, 243659 tiles were analyzed, including 124348 malignant, 48441 benign, and 70870 healthy tissue tiles. The independent testing dataset comprised 84751 tiles. Malignant tissue was identified with 91.35% accuracy, 95.88% sensitivity, 86.28% specificity, and an F1 score of 92.13%. Healthy tissue achieved 94.41% accuracy and 95.58% specificity. Benign tissue showed 90.14% accuracy and 91.78% specificity. Heatmaps consistently highlighted at a cellular level the architectural distortion in malignant tiles. CONCLUSION: This work introduces the first pediatric integration of deep learning with ex vivo FCM, demonstrating the feasibility of rapid and interpretable tissue classification and providing the basis for future prospective intraoperative validation.