In its simplest form, convolution neural networks (CNNs) consist of a fully connected layer g composed with a sequence of convolution layers T . Although g is known to have the universal approximation property, it is not known if CNNs, which has the form g ◦ T inherits this property, especially when the kernel size in T is small. In this paper, we show that under suitable conditions, CNNs does inherit the universal approximation property and its sample complexity can be characterized. In addition, we discuss concretely how the nonlinearity of T can improve the approximation power. Finally, we show that when the target function class has a certain compositional form, convolutional networks are far more advantageous compared with fully connected networks, in terms of number of parameters needed to achieve a desired accuracy.

No abstract uploaded!

We propose a computational framework named iterative local adaptive majorize-minimization (I-LAMM) to simultaneously control algorithmic complexity and statistical error when fitting high dimensional models. I-LAMM is a two-stage algorithmic implementation of the local linear approximation to a family of folded concave penalized quasi-likelihood. The first stage solves a convex program with a crude precision tolerance to obtain a coarse initial estimator, which is further refined in the second stage by iteratively solving a sequence of convex programs with smaller precision tolerances. Theoretically, we establish a phase transition: the first stage has a sublinear iteration complexity, while the second stage achieves an improved linear rate of convergence. Though this framework is completely algorithmic, it provides solutions with optimal statistical performances and controlled algorithmic complexity for a large family

No abstract uploaded!

Alzheimer's disease (AD) is a no-cure disease that has been frustrating the scientists for many years. Analyzing the disease has become an important but challenging research topic. The shape analysis of the sub-cortical structure of AD patients has been commonly used to understand this disease. In this paper, we assess the feasibility of using shape information on the hippocampal (HP) surfaces to detect some sub-structural changes in AD patients. We propose a quasi-conformal statistical shape analysis model, which allows us to study local regional geometric changes in the HPs amongst normal control (NC) and AD groups. A shape index defined by the quasi-conformality and surface curvatures is used to characterize region-specific shape variations of the HP surfaces. Feature vectors can be extracted for each HPs, with which a classification model can be built using machine learning methods to classify HPs into NC and AD subjects. Experiments have been carried out on 99 normal controls and 41 patients with AD. Results demonstrate that the proposed quasi-conformal based model is effective for classifying HPs into NC and AD groups with high classification accuracy (with highest overall classification accuracy reaching 87.86% in a leave-one-out experiment using the whole dataset).