What is Blackfish AI?

Blackfish helps researchers use state-of-the-art, open source artificial intelligence and machine learning models on high-performance computing (HPC) clusters. With Blackfish, researchers can spin up their own private, OpenAI-compatible inference APIs, and run large-scale batch inference jobs using HPC resources already available on their campus. Blackfish aims to provide an alternative to public cloud services, such as ChatGPT, Amazon Transcribe, etc. that is safe to use with restricted data, prioritizes customization and replicability, and saves research funds.

What problem does this solve?

Researchers increasingly rely on AI to unlock new sources of data and augment or replace non-specialist assistants, e.g., coding data sources. Public cloud providers have for years offered AI services that perform essential data preprocessing tasks, such as transcribing recordings, performing object detection tasks, and labeling content. While relatively easy to use, these services:

• can be prohibitively expensive,
• are opaque and not customizable,
• suffer from model creep, making results hard to reproduce,
• may not satisfy data handling requirements,
• and, for many research tasks, are no longer clearly better than open source alternatives.

For many preprocessing tasks research depends on — transcription, classification, annotation, extraction — open source models now perform comparably to commercial services, and sometimes better.¹²³ In addition, they cost nothing to run on an institutional cluster, can be inspected and pinned to an exact version, and provide complete control over sensitive data. For researchers working under an IRB protocol or a data use agreement, open source may be the only game in town.

However, the benefits of open source come with their own costs. Researchers need to learn how to work with the HPC clusters that provide access to high-powered GPUs: writing Slurm batch scripts, running Apptainer containers, navigating Hugging Face cache layouts, setting up port forwarding off a compute node, and working out which GPU will fit a 70-billion parameter model. These barriers limit adoption to researchers with the time, funding, and inclination to keep up with a fast-moving ecosystem.

Blackfish automates the tricky parts of deployment and provides standardized workflows so that launching a private inference API or running batch inference against a collection of documents is a single command — or a few clicks. Researchers should focus on their research, not on an evolving toolset. We meet researchers where they are by providing multiple ways to work, including a Python API, a command-line tool (CLI), and a browser-based user interface (UI). And because everything runs on your cluster, under your allocation, and using the model version you chose, you keep the transparency, privacy, and reproducibility benefits of open source.

Next Steps

Ready to get started? Our getting started page walks through each step in detail.

Acknowledgements

Blackfish is maintained by research software engineers at the Data Driven Social Science Initiative, a part of Data and Intelligent Systems at Princeton University.