Feature distillation, a primary method in knowledge distillation, always leads to significant accuracy improvements. Most existing methods distill features in the teacher network through a manually designed transformation. In this paper, we propose a novel distillation method named task-oriented feature distillation (TOFD) where the transformation is convolutional layers that are trained in a data-driven manner by task loss. As a result, the task-oriented information in the features can be captured and distilled to students. Moreover, an orthogonal loss is applied to the feature resizing layer in TOFD to improve the performance of knowledge distillation. Experiments show that TOFD outperforms other distillation methods by a large margin on both image classification and 3D classification tasks. Codes have been released in Github[3].

We consider Schrdinger operators in L2(R3) with a singular interaction supported by a finite curve . We present a proper definition of the operators and study their properties, in particular, we show that the discrete spectrum can be empty if is short enough. If it is not the case, we investigate properties of the eigenvalues in the situation when the curve has a hiatus of length 2. We derive an asymptotic expansion with the leading term which a multiple of ln.

We propose a new basic trapdoor ℓIC (ℓ-Invertible Cycles) of the mixed field type for Multivariate Quadratic public key cryptosystems. This is the first new basic trapdoor since the invention of Unbalanced Oil and Vinegar in 1997. ℓIC can be considered an extended form of the well-known Matsumoto-Imai Scheme A (also MIA or C^∗), and share some features of stagewise triangular systems. However ℓIC has very distinctive properties of its own. In practice, ℓIC is much faster than MIA, and can even match the speed of single-field MQ schemes.

Collecting paired training data is difficult in practice, but the unpaired samples broadly exist. Current approaches aim at generating synthesized training data from the unpaired samples by exploring the relationship between the corrupted and clean data. This work proposes LUD-VAE, a deep generative method to learn the joint probability density function from data sampled from marginal distributions. Our approach is based on a carefully designed probabilistic graphical model in which the clean and corrupted data domains are conditionally independent. Using variational inference, we maximize the evidence lower bound (ELBO) to estimate the joint probability density function. Furthermore, we show that the ELBO is computable without paired samples under the inference invariant assumption. This property provides the mathematical rationale of our approach in the unpaired setting. Finally, we apply our method to real-world image denoising and super-resolution tasks and train the models using the synthetic data generated by the LUD-VAE. Experimental results validate the advantages of our method over other learnable approaches.

The longstanding open problem of approximating all singular vertex couplings in a quantum graph is solved. We present a construction in which the edges are decoupled; an each pair of their endpoints is joined by an edge carrying a potential and a vector potential coupled to the loose edges by a coupling. It is shown that if the lengths of the connecting edges shrink to zero and the potentials are properly scaled, the limit can yield any prescribed singular vertex coupling, and moreover, that such an approximation converges in the norm-resolvent sense.