No abstract uploaded!

We study minimizers of the functionalwhere $B_{1}^{{\mathchoice {\raise .17ex\hbox {$\scriptstyle +$}} {\raise .17ex\hbox {$\scriptstyle +$}} {\raise .1ex\hbox {$\scriptscriptstyle +$}} {\scriptscriptstyle +}}}=\{x\in B_{1}: x_{1}>0\}$ ,$u$=0 on {$x$∈$B$_{1}:$x$_{1}=0}, $\lambda^{{\mathchoice {\raise .17ex\hbox {$\scriptstyle \pm $}} {\raise .17ex\hbox {$\scriptstyle \pm $}} {\raise .1ex\hbox {$\scriptscriptstyle \pm $}} {\scriptscriptstyle \pm }}}$ are two positive constants and 0<$p$<1. In two dimensions, we prove that the free boundary is a uniform$C$^{1}graph up to the flat part of the fixed boundary and also that two-phase points cannot occur on this part of the fixed boundary. Here, the free boundary refers to the union of the boundaries of the sets {$x$:±$u$($x$)>0}.

In this work we introduce a formulation for a non-local Hessian that combines the ideas of higher-order and non-local regularization for image restoration, extending the idea of non-local gradients to higher-order derivatives. By carefully choosing the weights, the model allows to improve on the current state of the art higher-order method, Total Generalized Variation, with respect to overall quality and particularly close to jumps in the data. In the spirit of recent work by Brezis et al., our formulation also has analytic implications: for a suitable choice of weights, it can be shown to converge to classical second-order regularizers, and in fact allows a novel characterization of higher-order Sobolev and BV spaces.

A recent line of work has studied the statistical properties of neural networks to great success from a {\it mean field theory} perspective, making and verifying very precise predictions of neural network behavior and test time performance. In this paper, we build upon these works to explore two methods for taming the behaviors of random residual networks (with only fully connected layers and no batchnorm). The first method is {\it width variation (WV)}, ie varying the widths of layers as a function of depth. We show that width decay reduces gradient explosion without affecting the mean forward dynamics of the random network. The second method is {\it variance variation (VV)}, ie changing the initialization variances of weights and biases over depth. We show VV, used appropriately, can reduce gradient explosion of tanh and ReLU resnets from \exp (\Theta (\sqrt L)) and \exp (\Theta (\sqrt L)) respectively to constant \exp (\Theta (\sqrt L)) . A complete phase-diagram is derived for how variance decay affects different dynamics, such as those of gradient and activation norms. In particular, we show the existence of many phase transitions where these dynamics switch between exponential, polynomial, logarithmic, and even constant behaviors. Using the obtained mean field theory, we are able to track surprisingly well how VV at initialization time affects training and test time performance on MNIST after a set number of epochs: the level sets of test/train set accuracies coincide with the level sets of the expectations of certain gradient norms or of metric expressivity (as defined in\cite {yang_meanfield_2017}), a measure of expansion in a random neural network. Based on insights from past works in

No abstract uploaded!