Deep learning models are currently the models of choice for image classification tasks. But large scale models require large quantities of data. For many tasks, acquiring a sufficient quantity of training data is not feasible. Because of this, an active area of research in machine learning is the field of few sample learning or few shot learning (FSL), with architectures that attempt to build effective models for a low-sample regime. In this paper, we focus on the established few-shot learning algorithm developed by Snell et al.1 We propose an FSL model where the model is produced via traditional encoding with the backend output layer replaced with the prototypical clustering classifier of Snell et al.. We hypothesize that this algorithm’s encoding structure produced by this training may be equivalent to models produced by traditional cross-entropy deep learning optimization. We compare few shot classification performance on unseen classes between models trained using the FSL training paradigm and our hybrid models trained traditionally with softmax, but modified for FSL use. Our empirical results indicate that traditionally trained models can be effectively re-used for few sample classification.
Deep learning models are pervasive for a multitude of tasks, but the complexity of these models can limit interpretation and inhibit trust in their estimates of confidence. For the classification task, we investigate the induced geometric relationships between the class conditioned data distributions with the deep learning models’ output weight vectors. We propose a simple statistic, which we call Angular Margin, to characterize the “confidence” of the model given a new input. We compare and contrast our statistic to Angular Visual Hardness and Softmax outputs. We demonstrate that Angular Margin provides a superior statistic for detecting minimum-perturbation adversarial attacks and/or misclassified images than standard Softmax predictions.
Continued advancements in adversarial attacks have crippled neural network performance. These small pixel perturbations can go undetected and cause networks to misclassify with high confidence. The motivation for this paper was to investigate how various sensor modalities and network models respond to adversarial attacks. It is important to realize that the large diversity in neural network architectures makes it difficult for any analytical conclusions to be made that generalize across any given neural network. For this reason, we share the statistical analyses performed which could be applied to any network under review. General observations gained from this analysis are also shared which indicated that network classification accuracy is not just a function of the network model but the data as well.
T-distributed Stochastic Neighbor Embedding (t-SNE) has become an extremely popular algorithm for low- dimensional visualization of high dimensional data. While it is acknowledged that it is highly sensitive to its parameters, it continues to be used extensively by the machine learning community, with `intuition' an accepted basis for embedding selection. In this paper, we will illustrate and explain why t-SNE is not a distance preserving algorithm, but rather order preserving, with the cardinality of the order proportional to the perplexity parameter. We compare and contrast t-SNE with Sammon Nonlinear Mappings locally using Kruskal Stress and Spearman Rank Correlation measures.
In support of airborne radar detection missions that rely on Synthetic Aperture Radar (SAR) imagery, there is a need for extensive sets of training data. Due to a paucity of measured data from some targets of interest, there is sometimes a need to train on only simulated SAR data, and yet detect live targets with high confidence during testing. In support of this mission, many researchers have applied a variety of mathematical techniques to simulate data sets. These techniques range from template matching and simpler statistical methods to deep neural networks (DDNs). They demonstrate that with proper pre-processing, some of these methods can achieve target detection with apparently high confidence. However, for all these papers there is no exact measurement of the differences or similarities in the simulated and measured data that would provide a good predictor of the margins between decision boundaries. Thus, this paper has developed a combination of pre-processing methods and standard metrics that enable the assessment of simulated data quality independent of which target recognition algorithm will be utilized. The results show that for some pre-processing methods the differences in simulated data and measured data do not always lend themselves to the desired ability to train on simulated SAR imagery and test on measured SAR imagery.
Deep learning models are pervasive for a multitude of tasks, but the complexity of these models can limit interpretation and inhibit trust. For a classification task, we investigate the induced relationships between the class conditioned data distributions, and geometrically compare/contrast the data with the deep learning models' output weight vectors. These geometric relationships are examined across models as a function of dense hidden layer width. Additionally, we geometrically characterize perturbation-based adversarial examples with respect to the deep learning model.
Many current classification models, such as Random Kitchen Sinks and Extreme Learning Machines (ELM), minimize the need for expert-defined features by transforming the measurement spaces into a set of "features" via random functions or projections. Alternatively, Random Forests exploit random subspaces by limiting tree partitions (i.e. nodes of the tree) to be selected from randomly generated subsets of features. For a synthetic aperture RADAR classification task, and given two orthonormal measurement representations (spatial and multi-scale Haar wavelet), this work compares and contrasts ELM and Random Forest classifier performance as a function of (a) input measurement representation, (b) classifier complexity, and (c) measurement domain mismatch. For the ELM classifier, we also compare two random projection encodings.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.