In a Data-Generating Experiment, the observed sample, x, has intractable or unavailable c.d.f., F_θ, and θ’s statistical nature is unknown; θ is element of metric space (Θ, dΘ). Matching estimates of θ are introduced, learning from the “best” x-matches with samples X^∗ from F_{θ^∗}, θ^∗ ∈ Θ. Under mild conditions, these nonparametric estimates are uniformly consistent and the upper bounds on their rates of convergence in probability have the same rate and depend on the Kolmogorov entropies of an increasing sequence of sets covering Θ. When Θ ⊆ Rm and the observations are i.i.d. the upper bounds can be, \sqrt{log n}/\sqrt{n} when m is known, and \sqrt{m_n·log n}/\sqrt{n} when m is unknown; m ≥ 1, m_n ↑ ∞ at a desired rate. Upper bounds can also be obtained for dependent observations. These rates hold for observations in R^d, complementing recent results obtained for real, i.i.d. observations, under stronger assumptions and using weak probability distances; d ≥ 1. In simulations, the Matching estimates are successful for the mixture of 2 normals and for Tukey’s (a, b, g, h) and the (a, b, g, k) models. Computers’ evolution will allow for more and faster comparisons, resulting in improved Matching estimates for universal use in Machine Learning.

We investigate the use of artificial neural networks (NNs) as an alternative tool to current analytical methods for recognizing knots in a given polymer conformation. The motivation is twofold. First, it is of interest to examine whether NNs are effective at learning the global and sequential properties that uniquely define a knot. Second, knot classification is an important and unsolved problem in mathematical and physical sciences, and NNs may provide insights into this problem. Motivated by these points, we generate millions of polymer conformations for five knot types: 0, 3_1, 4_1, 5_1, and 5_2, and we design various NN models for classification. Our best model achieves a five-class classification accuracy of above 99% on a polymer of 100 monomers. We find that the sequential modeling ability of recurrent NNs is crucial for this result, as it outperforms feed-forward NNs and successfully generalizes to differently sized conformations as well. We present our methods and suggest that deep learning may be used in specific applications of knot detection where some error is permissible. Hopefully, with further development, NNs can offer an alternative computational method for knot identification and facilitate knot research in mathematical and physical sciences.

Training process is crucial for the deployment of the network in applications which have two strict requirements on both accuracy and robustness. However, most existing approaches are in a dilemma, i.e. model accuracy and robustness forming an embarrassing tradeoff – the improvement of one leads to the drop of the other. The challenge remains for as we try to improve the accuracy and robustness simultaneously. In this paper, we propose a novel training method via introducing the auxiliary classifiers for training on corrupted samples, while the clean samples are normally trained with the primary classifier. In the training stage, a novel distillation method named input-aware self distillation is proposed to facilitate the primary classifier to learn the robust information from auxiliary classifiers. Along with it, a new normalization method - selective batch normal- ization is proposed to prevent the model from the negative influence of corrupted images. At the end of the training period, a L2-norm penalty is applied to the weights of primary and auxiliary classifiers such that their weights are asymptotically identical. In the stage of inference, only the primary classifier is used and thus no extra computation and storage are needed. Extensive experiments on CIFAR10, CIFAR100 and ImageNet show that noticeable improvements on both accuracy and robustness can be observed by the proposed auxiliary training. On average, auxiliary training achieves 2.21% accuracy and 21.64% robustness (measured by corruption error) improvements over traditional training methods on CIFAR100. Codes have been released on github.

No abstract uploaded!

We develop an auto-reservoir computing framework, Auto-Reservoir Neural Network (ARNN), to efficiently and accurately make multi-step-ahead predictions based on a short-term high-dimensional time series. Different from traditional reservoir computing whose reservoir is an external dynamical system irrelevant to the target system, ARNN directly transforms the observed high-dimensional dynamics as its reservoir, which maps the high-dimensional/spatial data to the future temporal values of a target variable based on our spatiotemporal information (STI) transformation. Thus, the multi-step prediction of the target variable is achieved in an accurate and computationally efficient manner. ARNN is successfully applied to both representative models and real-world datasets, all of which show satisfactory performance in the multi-step-ahead prediction, even when the data are peturbed by noise and when the system is time-varying. Actually, such ARNN transformation equivalently expands the sample size and thus has great potential in practical applications in artificial intelligence and machine learning.