Recurrent nuisance flooding is common across many parts of the globe and causes extensive challenges for drivers on the roadways. The prevailing monitoring methods for roadway flooding are costly and not automated or effective. The ubiquity of visual data from cameras and advancements in computing such as deep learning may offer cost-effective methods for automated flood depth estimation on roadways based on reference objects such as cars. However, flood depth estimation faces challenges due to the limited amount of data annotated with water levels and diverse scenes showing reference objects at various scales and perspectives. This study proposes a novel deep learning approach to automated flood depth estimation on roadways. Our proposed pipeline addresses variations in object perspective and scale. We have developed an innovative approach to generate and annotate flood images by manipulating existing image datasets of cars in various orientations and scales to simulate four floodwater levels for augmenting real flood images. Furthermore, we propose object scale normalization for our reference objects (cars) to improve water level predictions. The proposed model achieves an accuracy of 74.85% and F1 score of 74.32% for four water levels when tested with real flood data. The proposed approach substantially reduces the time and labor required for labeling datasets while addressing challenges in perspective/scale, offering a promising solution for image-based flood depth estimation.
KEYWORDS: Data modeling, Magnetic resonance imaging, Performance modeling, Education and training, Data privacy, Cross validation, Feature extraction, Tumors, Radiomics, Mixtures
Deep learning models have shown potential in medical image analysis tasks. However, training a generalized deep learning model requires huge amounts of patient data that is usually gathered from multiple institutions which may raise privacy concerns. Federated learning (FL) provides an alternative to sharing data across institutions. Nonetheless, FL is susceptible to a few challenges including inversion attacks on model weights, heterogenous data distributions, and bias. This study addresses heterogeneity and bias issues for multi-institution patient data by proposing domain adaptive FL modeling using several radiomics (volume, fractal, texture) features for O6-methylguanine-DNA methyltransferase (MGMT) classification across multiple institutions. The proposed domain adaptive FL MGMT classification inherently offers differential privacy (DP) for the patient data. For domain adaptation two techniques e.g., mixture of experts (ME) with a gating network and adversarial alignment are used for comparison. The proposed method is evaluated using publicly available multi-institution (UPENN-GBM, UCSF-PDGM, RSNA-ASNR-MICCAI BraTS-2021) data set with a total of 1007 patients. Our experiments with 5-fold cross validation suggest that domain adaptive FL offers improved performance with a mean accuracy of 69.93% ± 4.8 % and area under curve of 0.655 ± 0.055 across multiple institutions. In addition, further analysis of probability density of gating network for domain adaptive FL identifies the institution that may bias the global model prediction due to increased heterogeneity for a given input. Our comparison analysis shows that the proposed method with bias identification offers the best predictive performance when compared to different commonly employed FL and baseline methods in the literature.
Alexithymia describes a psychological state where individuals struggle with feeling and expressing their emotions. Individuals with alexithymia may also have a more difficult time understanding the emotions of others and may express atypical attention to the eyes when recognizing emotions. This is known to affect individuals with Autism Spectrum Disorder (ASD) differently than neurotypical (NT) individuals. Using a public data set of eye-tracking data from seventy individuals with and without autism who have been assessed for alexithymia, we train multiple traditional machine learning models for alexithymia classification including support vector machines, logistic regression, decision trees, random forest, and multilayer perceptron. To correct for class imbalance, we evaluate four different oversampling strategies: no oversampling, random oversampling, SMOTE, and ADASYN. We consider three different groups of data: ASD, NT, and combined ASD+NT. We use a nested leave-one-out cross validation strategy to perform hyperparameter selection and evaluate model performance. We achieve F1 scores of 90.00% and 51.85% using decision trees for ASD and NT groups, respectively, and 72.41% using SVM for the combined ASD+NT group. Splitting the data into ASD and NT groups improves recall for both groups compared to the combined model.
Automatic classification of child facial expressions is challenging due to the scarcity of image samples with annotations. Transfer learning of deep convolutional neural networks (CNNs), pretrained on adult facial expressions, can be effectively finetuned for child facial expression classification using limited facial images of children. Recent work inspired by facial age estimation and age-invariant face recognition proposes a fusion of facial landmark features with deep representation learning to augment facial expression classification performance. We hypothesize that deep transfer learning of child facial expressions may also benefit from fusing facial landmark features. Our proposed model architecture integrates two input branches: a CNN branch for image feature extraction and a fully connected branch for processing landmark-based features. The model-derived features of these two branches are concatenated into a latent feature vector for downstream expression classification. The architecture is trained on an adult facial expression classification task. Then, the trained model is finetuned to perform child facial expression classification. The combined feature fusion and transfer learning approach is compared against multiple models: training on adult expressions only (adult baseline), child expression only (child baseline), and transfer learning from adult to child data. We also evaluate the classification performance of feature fusion without transfer learning on model performance. Training on child data, we find that feature fusion improves the 10-fold cross validation mean accuracy from 80.32% to 83.72% with similar variance. Proposed fine-tuning with landmark feature fusion of child expressions yields the best mean accuracy of 85.14%, a more than 30% improvement over the adult baseline and nearly 5% improvement over the child baseline.
State-of-the-art deep learning models have demonstrated success in classifying facial expressions of adults by relying on large datasets of labeled images. Unfortunately, there is a scarcity of labeled images of child expressions. Deep learning models trained on adult data do not generalize well on child data due to the domain shift caused by morphological differences in their faces. Recent deep domain adaptation approaches align the data distribution of a target domain with the source domain using a few target domain samples. We propose that the domain adaptation may be improved by incorporating steps of deep transfer learning, such as initialization with pre-trained source weights and freezing early layers of the model. The knowledge of a few labeled examples from the child data (target domain) is incorporated into the adult data distribution (source domain) using a contrastive semantic alignment (CSA) loss. This work combines deep transfer learning and domain adaptation approaches to generate seven expression labels (‘happy’, ‘sad’, ‘anger’, ‘fear’, ‘surprise’, ‘disgust’, plus ‘neutral’) for facial images of children in reference to the source domain, adult facial expressions, using 10 or fewer samples per expression. Our hybrid approach outperforms the transfer learning model by 12% on mean accuracy using only 10 samples per expression class.
The classification of facial expression has been extensively studied using adult facial images which are not appropriate ground truths for classifying facial expressions in children. The state-of-the-art deep learning approaches have been successful in the classification of facial expressions in adults. A deep learning model may be better able to learn the subtle but important features underlying child facial expressions and improve upon the performance of traditional machine learning and feature extraction methods. However, unlike adult data, only a limited number of ground truth images exist for training and validating models for child facial expression classification and there is a dearth of literature in child facial expression analysis. Recent advances in transfer learning methods have enabled the use of deep learning architectures, trained on adult facial expression images, to be tuned for classifying child facial expressions with limited training samples. The network will learn generic facial expression patterns from adult expressions which can be fine-tuned to capture representative features of child facial expressions. This work proposes a transfer learning approach for multi-class classification of the seven prototypical expressions including the ‘neutral’ expression in children using a recently published child facial expression data set. This work holds promise to facilitate the development of technologies that focus on children and monitoring of children throughout their developmental stages to detect early symptoms related to developmental disorders, such as Autism Spectrum Disorder (ASD).
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.