Training process is crucial for the deployment of the network in applications which have two strict requirements on both accuracy and robustness. However, most existing approaches are in a dilemma, i.e. model accuracy and robustness forming an embarrassing tradeoff – the improvement of one leads to the drop of the other. The challenge remains for as we try to improve the accuracy and robustness simultaneously. In this paper, we propose a novel training method via introducing the auxiliary classifiers for training on corrupted samples, while the clean samples are normally trained with the primary classifier. In the training stage, a novel distillation method named input-aware self distillation is proposed to facilitate the primary classifier to learn the robust information from auxiliary classifiers. Along with it, a new normalization method - selective batch normal- ization is proposed to prevent the model from the negative influence of corrupted images. At the end of the training period, a L2-norm penalty is applied to the weights of primary and auxiliary classifiers such that their weights are asymptotically identical. In the stage of inference, only the primary classifier is used and thus no extra computation and storage are needed. Extensive experiments on CIFAR10, CIFAR100 and ImageNet show that noticeable improvements on both accuracy and robustness can be observed by the proposed auxiliary training. On average, auxiliary training achieves 2.21% accuracy and 21.64% robustness (measured by corruption error) improvements over traditional training methods on CIFAR100. Codes have been released on github.

In a Data-Generating Experiment (DGE), the data, X, is often obtained either from a Black-Box with inputs θ and Y, or from a Quantile function or a learning machine, f(Y, θ); θ is unknown, element of metric space (Θ, ρ), Y is random. If X has intractable or unknown c.d.f., Fθ, non-identifiability of θ cannot be confirmed and when present, among others, limits the predictive accuracy of the learned model, f(Y, \hat{θ}); \hat{θ} estimate of θ. In Machine Learning, non-identifiability of θ is ubiquitous and its extent is a criterion for selecting a learning machine. Empirical indices, EDI and PPVI, are introduced using P-values of Kolmogorov-Smirnov tests: i) to confirm almost surely, using generated data, the discrimination of θ from θ^∗, namely that the Kolmogorov distance, dK(Fθ, Fθ^∗), is positive, ii) to confirm identifiability of θ(∈ Θ) by repeating i) for θ^∗ in a sieve of Θ, since neighboring parameter values are in practice indistinguishable, and iii) most important, to compare EDI-graphs of DGEs, preferring more discrimination and less non-identifiability among parameters, and select one DGE to use. In applications, EDI-graphs confirm nonidentifiability in mixture models and in models parametrised with sums of parameters. EDI and PPVI explain why Tukey’s g-and-h model (DGE1) has better g-discrimination than the g-and-k model (DGE2), unless the sample size is extremely large; h_0 = k_0. EDIgraphs indicate that Normal learning machines have better parameter discrimination thanSigmoid learning machines and their parameters are non-identifiable.

No abstract uploaded!

No abstract uploaded!

When sample, X = x, is observed from intractable c.d.f. Fθ, or a Black-Box with input θ, an Approximate Bayesian Computation (ABC) method provides approximate posterior, π_ϵ. θ^∗ is included in the support of π_ϵ when the Matching distance ρ(S(x^∗), S(x)) ≤ ϵ; x^∗ is a sample drawn from F_{θ^∗}, θ^∗ is obtained from prior π, S is a summary statistic, ϵ > 0. ABC concerns include: the use of only one sample, x^∗, for each θ^∗; the choices of S, ρ and ϵ; π_ϵ(θ^∗), which is determined by arbitrary kernel, K(x, x^∗; ϵ), creating visual π_ϵ-artifacts. The concerns are accommodated with the introduced Fiducial(F)-ABC for all (θ^∗ drawn): M x^∗ are drawn from F_{θ^∗}; a universal S is used, the empirical measure indexed by sets which activate its sufficiency for exchangeable observations-vectors and have been neglected; a strong, probability distance ρ is used, inherently connected with S and Matching; light is thrown to ϵ’s nature and value; πϵ is obtained from the proportions of x∗ matching x. F-ABC for all posterior is closer to Bayesian philosophy, which does not use θ^∗-exclusions. Under few, mild assumptions, π_ϵ converges to the posterior, π(θ|x), when ϵ ↓ 0, and rates of concentration of T (F_{θ^∗}) to T (F_θ) are obtained when n ↑ ∞; T is a functional. Satisfactory F-ABC for all θ^∗ drawn posteriors are depicted for parametric and data generating models, including Tukey’s (a, b, g, h)-model, a 5-parameter normal mixture and a time series model. Various advantages of the F-ABC for all are presented over coarsened posteriors for observations in R^d, d ≥ 1.