In a Data-Generating Experiment, the observed sample, x, has intractable or unavailable c.d.f., F_θ, and θ’s statistical nature is unknown; θ is element of metric space (Θ, dΘ). Matching estimates of θ are introduced, learning from the “best” x-matches with samples X^∗ from F_{θ^∗}, θ^∗ ∈ Θ. Under mild conditions, these nonparametric estimates are uniformly consistent and the upper bounds on their rates of convergence in probability have the same rate and depend on the Kolmogorov entropies of an increasing sequence of sets covering Θ. When Θ ⊆ Rm and the observations are i.i.d. the upper bounds can be, \sqrt{log n}/\sqrt{n} when m is known, and \sqrt{m_n·log n}/\sqrt{n} when m is unknown; m ≥ 1, m_n ↑ ∞ at a desired rate. Upper bounds can also be obtained for dependent observations. These rates hold for observations in R^d, complementing recent results obtained for real, i.i.d. observations, under stronger assumptions and using weak probability distances; d ≥ 1. In simulations, the Matching estimates are successful for the mixture of 2 normals and for Tukey’s (a, b, g, h) and the (a, b, g, k) models. Computers’ evolution will allow for more and faster comparisons, resulting in improved Matching estimates for universal use in Machine Learning.

We investigate the use of artificial neural networks (NNs) as an alternative tool to current analytical methods for recognizing knots in a given polymer conformation. The motivation is twofold. First, it is of interest to examine whether NNs are effective at learning the global and sequential properties that uniquely define a knot. Second, knot classification is an important and unsolved problem in mathematical and physical sciences, and NNs may provide insights into this problem. Motivated by these points, we generate millions of polymer conformations for five knot types: 0, 3_1, 4_1, 5_1, and 5_2, and we design various NN models for classification. Our best model achieves a five-class classification accuracy of above 99% on a polymer of 100 monomers. We find that the sequential modeling ability of recurrent NNs is crucial for this result, as it outperforms feed-forward NNs and successfully generalizes to differently sized conformations as well. We present our methods and suggest that deep learning may be used in specific applications of knot detection where some error is permissible. Hopefully, with further development, NNs can offer an alternative computational method for knot identification and facilitate knot research in mathematical and physical sciences.

No abstract uploaded!

We study randomly initialized residual networks using mean field theory and the theory of difference equations. Classical feedforward neural networks, such as those with tanh activations, exhibit exponential behavior on the average when propagating inputs forward or gradients backward. The exponential forward dynamics causes rapid collapsing of the input space geometry, while the exponential backward dynamics causes drastic vanishing or exploding gradients. We show, in contrast, that by adding skip connections, the network will, depending on the nonlinearity, adopt subexponential forward and backward dynamics, and in many cases in fact polynomial. The exponents of these polynomials are obtained through analytic methods and proved and verified empirically to be correct. In terms of the" edge of chaos" hypothesis, these subexponential and polynomial laws allow residual networks to" hover over the boundary between stability and chaos," thus preserving the geometry of the input space and the gradient information flow. In our experiments, for each activation function we study here, we initialize residual networks with different hyperparameters and train them on MNIST. Remarkably, our initialization time theory can accurately predict test time performance of these networks, by tracking either the expected amount of gradient explosion or the expected squared distance between the images of two input vectors. Importantly, we show, theoretically as well as empirically, that common initializations such as the Xavier or the He schemes are not optimal for residual networks, because the optimal initialization variances depend on the depth

When sample, X = x, is observed from intractable c.d.f. Fθ, or a Black-Box with input θ, an Approximate Bayesian Computation (ABC) method provides approximate posterior, π_ϵ. θ^∗ is included in the support of π_ϵ when the Matching distance ρ(S(x^∗), S(x)) ≤ ϵ; x^∗ is a sample drawn from F_{θ^∗}, θ^∗ is obtained from prior π, S is a summary statistic, ϵ > 0. ABC concerns include: the use of only one sample, x^∗, for each θ^∗; the choices of S, ρ and ϵ; π_ϵ(θ^∗), which is determined by arbitrary kernel, K(x, x^∗; ϵ), creating visual π_ϵ-artifacts. The concerns are accommodated with the introduced Fiducial(F)-ABC for all (θ^∗ drawn): M x^∗ are drawn from F_{θ^∗}; a universal S is used, the empirical measure indexed by sets which activate its sufficiency for exchangeable observations-vectors and have been neglected; a strong, probability distance ρ is used, inherently connected with S and Matching; light is thrown to ϵ’s nature and value; πϵ is obtained from the proportions of x∗ matching x. F-ABC for all posterior is closer to Bayesian philosophy, which does not use θ^∗-exclusions. Under few, mild assumptions, π_ϵ converges to the posterior, π(θ|x), when ϵ ↓ 0, and rates of concentration of T (F_{θ^∗}) to T (F_θ) are obtained when n ↑ ∞; T is a functional. Satisfactory F-ABC for all θ^∗ drawn posteriors are depicted for parametric and data generating models, including Tukey’s (a, b, g, h)-model, a 5-parameter normal mixture and a time series model. Various advantages of the F-ABC for all are presented over coarsened posteriors for observations in R^d, d ≥ 1.