Feature distillation, a primary method in knowledge distillation, always leads to significant accuracy improvements. Most existing methods distill features in the teacher network through a manually designed transformation. In this paper, we propose a novel distillation method named task-oriented feature distillation (TOFD) where the transformation is convolutional layers that are trained in a data-driven manner by task loss. As a result, the task-oriented information in the features can be captured and distilled to students. Moreover, an orthogonal loss is applied to the feature resizing layer in TOFD to improve the performance of knowledge distillation. Experiments show that TOFD outperforms other distillation methods by a large margin on both image classification and 3D classification tasks. Codes have been released in Github[3].

No abstract uploaded!

No abstract uploaded!

We consider resonant vortices around nodal points of the wave function describing electron transport through a mesoscopic device. With a suitable choice of the device geometry, the dominating role is played by single vortices with a preferred orientation. To characterize the strength of the resulting magnetic moment, we have introduced a magnetance, the quantity defined in analogy with the device conductance. Its basic properties and possible experimental detection are discussed.

The concept of matrix canonical realization of a Lie algebra is introduced. The generators of the Lie algebra of the pseudoorthogonal group n) are recurrently expressed in terms of matrices with polyno-mial elements in a certain number of quantum-mechanical canonical variables p~, qi and they depend on a certain number of free real para-meters. The realizations are, in the well-defined sense, skew-hermitean and Casimir operators are multiples of the identity element. Part of them are usual canonical realizations.