In a Data-Generating Experiment, the observed sample, x, has intractable or unavailable c.d.f., F_θ, and θ’s statistical nature is unknown; θ is element of metric space (Θ, dΘ). Matching estimates of θ are introduced, learning from the “best” x-matches with samples X^∗ from F_{θ^∗}, θ^∗ ∈ Θ. Under mild conditions, these nonparametric estimates are uniformly consistent and the upper bounds on their rates of convergence in probability have the same rate and depend on the Kolmogorov entropies of an increasing sequence of sets covering Θ. When Θ ⊆ Rm and the observations are i.i.d. the upper bounds can be, \sqrt{log n}/\sqrt{n} when m is known, and \sqrt{m_n·log n}/\sqrt{n} when m is unknown; m ≥ 1, m_n ↑ ∞ at a desired rate. Upper bounds can also be obtained for dependent observations. These rates hold for observations in R^d, complementing recent results obtained for real, i.i.d. observations, under stronger assumptions and using weak probability distances; d ≥ 1. In simulations, the Matching estimates are successful for the mixture of 2 normals and for Tukey’s (a, b, g, h) and the (a, b, g, k) models. Computers’ evolution will allow for more and faster comparisons, resulting in improved Matching estimates for universal use in Machine Learning.

We investigate the use of artificial neural networks (NNs) as an alternative tool to current analytical methods for recognizing knots in a given polymer conformation. The motivation is twofold. First, it is of interest to examine whether NNs are effective at learning the global and sequential properties that uniquely define a knot. Second, knot classification is an important and unsolved problem in mathematical and physical sciences, and NNs may provide insights into this problem. Motivated by these points, we generate millions of polymer conformations for five knot types: 0, 3_1, 4_1, 5_1, and 5_2, and we design various NN models for classification. Our best model achieves a five-class classification accuracy of above 99% on a polymer of 100 monomers. We find that the sequential modeling ability of recurrent NNs is crucial for this result, as it outperforms feed-forward NNs and successfully generalizes to differently sized conformations as well. We present our methods and suggest that deep learning may be used in specific applications of knot detection where some error is permissible. Hopefully, with further development, NNs can offer an alternative computational method for knot identification and facilitate knot research in mathematical and physical sciences.

We develop an auto-reservoir computing framework, Auto-Reservoir Neural Network (ARNN), to efficiently and accurately make multi-step-ahead predictions based on a short-term high-dimensional time series. Different from traditional reservoir computing whose reservoir is an external dynamical system irrelevant to the target system, ARNN directly transforms the observed high-dimensional dynamics as its reservoir, which maps the high-dimensional/spatial data to the future temporal values of a target variable based on our spatiotemporal information (STI) transformation. Thus, the multi-step prediction of the target variable is achieved in an accurate and computationally efficient manner. ARNN is successfully applied to both representative models and real-world datasets, all of which show satisfactory performance in the multi-step-ahead prediction, even when the data are peturbed by noise and when the system is time-varying. Actually, such ARNN transformation equivalently expands the sample size and thus has great potential in practical applications in artificial intelligence and machine learning.

This paper proposes using the neural networks to efficiently solve the second-order cone programs (SOCP). To establish the neural networks, the SOCP is first reformulated as a second-order cone complementarity problem (SOCCP) with the KarushKuhnTucker conditions of the SOCP. The SOCCP functions, which transform the SOCCP into a set of nonlinear equations, are then utilized to design the neural networks. We propose two kinds of neural networks with the different SOCCP functions. The first neural network uses the FischerBurmeister function to achieve an unconstrained minimization with a merit function. We show that the merit function is a Lyapunov function and this neural network is asymptotically stable. The second neural network utilizes the natural residual function with the cone projection function to achieve low computation complexity. It is shown to be Lyapunov stable and converges globally

In a Data-Generating Experiment (DGE), the data, X, is often obtained either from a Black-Box with inputs θ and Y, or from a Quantile function or a learning machine, f(Y, θ); θ is unknown, element of metric space (Θ, ρ), Y is random. If X has intractable or unknown c.d.f., Fθ, non-identifiability of θ cannot be confirmed and when present, among others, limits the predictive accuracy of the learned model, f(Y, \hat{θ}); \hat{θ} estimate of θ. In Machine Learning, non-identifiability of θ is ubiquitous and its extent is a criterion for selecting a learning machine. Empirical indices, EDI and PPVI, are introduced using P-values of Kolmogorov-Smirnov tests: i) to confirm almost surely, using generated data, the discrimination of θ from θ^∗, namely that the Kolmogorov distance, dK(Fθ, Fθ^∗), is positive, ii) to confirm identifiability of θ(∈ Θ) by repeating i) for θ^∗ in a sieve of Θ, since neighboring parameter values are in practice indistinguishable, and iii) most important, to compare EDI-graphs of DGEs, preferring more discrimination and less non-identifiability among parameters, and select one DGE to use. In applications, EDI-graphs confirm nonidentifiability in mixture models and in models parametrised with sums of parameters. EDI and PPVI explain why Tukey’s g-and-h model (DGE1) has better g-discrimination than the g-and-k model (DGE2), unless the sample size is extremely large; h_0 = k_0. EDIgraphs indicate that Normal learning machines have better parameter discrimination thanSigmoid learning machines and their parameters are non-identifiable.