Researcher tools: code & datasets

LITMUS Predictor provides support for simulating performance in ~100 languages given training observations of the desired task-model. Each training observation specifies the finetuning-datasize + test-performance in different languages. Further, the tool provides support for constructing a data-collection strategy to maximize performance in desired targets subject to different constraints.

DistIR is an intermediate representation (IR) and associated set of tools for optimizing distributed machine learning computations (both training and inference). An IR is a format for representing programs used by compilers and software analysis tools. DistIR’s key feature is a simulator that allows the user to estimate the runtime and memory requirements of a model without running it, thereby enabling the user to quickly pick the best distribution strategy and configuration of the model…

We present a new method LiST for efficient fine-tuning of large pre-trained language models (PLMs) in few-shot learning settings. LiST significantly improves over recent methods that adopt prompt fine-tuning using two key techniques. The first one is the use of self-training to leverage large amounts of unlabeled data for prompt-tuning to significantly boost the model performance in few-shot settings. We use self-training in conjunction with meta-learning for re-weighting noisy pseudo-prompt labels.

A number of PyTorch modules (neural network components) that are broadly (re)usable. The goal of this release is to provide a library to machine learning researchers to use and advance the state-of-the-art in the area.

This PyTorch package implements Taming Sparsely Activated Transformer with Stochastic Experts.

This PyTorch package implements Very Deep Transformers for Neural Machine Translation, to stabilize the large scale language model and neural machine translation training, as described in: Very deep transformers for neural machine translation

This repository contains necessary data associated with the Analyzing the Effects of Reasoning Types on Cross-Lingual Transfer Performance, published in EMNLP 2021 Multilingual Representation Learning workshop. This data is a multi-lingual extension of ID 3860 released dataset. Similar to the ID 3397 and 3860, the broad goal for this project is to research and develop Neuro-Symbolic systems for Natural Language Inferencing (NLI) to leverage the “correctness” guarantees and interpretability of symbolic systems in neural network inference. In…

This is the implementation of the paper Meta Self-training for Few-shot Neural Sequence Labeling. MetaST is short for meta-learning for self-training.

Jacdac (Joint Asynchronous Communications; Device Agnostic Control) is a single wire broadcast protocol for the plug and play of microcontrollers (MCUs) within the contexts of rapid prototyping, making, and physical computing. The Jacdac Device Development Kit (DDK) is for 3rd party hardware designers, firmware developers and manufacturers who wish to create their own Jacdac devices. We’ll be releasing the DDK open-source.