1

Our diffusion-driven adaptation method, DDA, shares its models for classification and generation across all domains. Both models are trained on the source domain, then fixed during testing. We augment diffusion with image guidance and self-ensembling to automatically decide how much to adapt.

GACT is an activation compression training (ACT) framework to support a broad range of machine learning tasks for generic neural network architectures with limited domain knowledge. By analyzing a linearized version of ACT’s approximate gradient, we prove the convergence of GACT without prior knowledge on operator type or model architecture.

We introduce AdaContrast, a novel test-time adaptation strategy that uses self-supervised contrastive learning on the target domain to exploit the pair-wise information among target samples, which is optimized jointly with pseudo labeling.

ActNN is a PyTorch library for memory-efficient training. It reduces the training memory footprint by compressing the saved activations. ActNN is implemented as a collection of memory-saving layers. These layers have an identical interface to their PyTorch counterparts.

Co-design of a deformable convolution operation on FPGA with hardware-friendly modifications, showing up to 9.76× hardware speedup. Development of an efficient DNN model for object detection with co-designed input-adaptive deformable convolution that achieves 67.1 AP50 on Pascal VOC with 2.9 MB parameters. The model is 20.9× smaller but 10% more accurate than the Tiny-YOLO. Implementation of an FPGA accelerator to support the target neural network design that runs at 26 frames per second on Pascal VOC with 61.7 AP50.

Tent equips a model to adapt itself to new and different data during testing ☀️ 🌧 ❄️. Tented models adapt online and batch-by-batch to reduce error on dataset shifts like corruptions, simulation-to-real discrepancies, and other differences between training and testing data.

We present a novel framework that jointly detects and tracks 3D vehicle bounding boxes. Our approach leverages 3D pose estimation to learn 2D patch association overtime and uses temporal information from tracking to obtain stable 3D estimation.

Monocular Plan View Networks use a monocular plan view image together with a first-person image to learn a deep driving policy. The plan view image is generated from the first-person image using 3D detection and re-projection.

We propose an object-centric perception approach to deep control problems, and focus our experimentation on au- tonomous driving. Existing end-to-end models are holistic in nature; our approach augments policy learning with explicit representations that provide object-level attention.

We present a novel neural network layer that performs differentiable rasterization of arbitrary simplex-mesh-based geometrical signals (e.g., point clouds, line mesh, triangular mesh, tetrahedron mesh, polygon and polyhedron) of arbitrary dimensions. We further provide examples of incorporating the DDSL into neural networks for tasks such as polygonal image segmentation and neural shape optimization (for MNIST digits and airfoils).