Introduction — LLMs on NRP

Teaching: 20 min · Exercises: 0 min · Total: 20 min

Overview

The National Research Platform (NRP) makes large language models available to the research community through a managed, OpenAI-compatible inference endpoint. You do not need to rent cloud credits, install model weights, or request a GPU — you just point any OpenAI-compatible tool at NRP's URL and authenticate with a personal token.

This lesson covers what is available, how to get access, and the different ways you can interact with the models.

What NRP Provides

NRP exposes two complementary AI/LLM resources:

1. Managed LLM Service

A rotating catalog of open-weights models hosted on NRP GPUs, reachable two ways:

• Programmatically, behind a single OpenAI-compatible REST endpoint:

`` https://ellm.nrp-nautilus.io/v1 ``

You authenticate with a bearer token (see Getting Access below). The endpoint speaks the OpenAI API, so any tool that supports a custom base_url works out of the box.

• In the browser, via Open WebUI, signed in with your NRP account — no API token needed. Useful for quick experiments and sharing demos with collaborators.

Currently available models (see live list):

Evaluating models are under active testing — configuration can change without notice; stick to the others for anything you need to be stable through the workshop.

Models occasionally restart or roll over to a new version — check what's currently up at the LLM status dashboard.

2. Bring-Your-Own GPU

For workloads requiring full model control — custom weights, fine-tuning, custom quantization, or private inference — you can request your own GPU pod in your namespace and run any inference server (vLLM, TGI, Ollama, etc.). This is covered in other NRP training sessions.

Getting Access

Step 1: NRP Account

You need an NRP account associated with your institutional credentials. If you don't have one yet, follow Getting Started with NRP.

You also need to belong to a namespace with LLM access enabled — this is a per-namespace feature flag, so an account alone is not sufficient. For this tutorial, ask Daniel Diaz to add you to the CLARIPHY namespace, which already has it enabled. See Setup.

Step 2: Get an API Token

Go to https://nrp.ai/llmtoken and click Get LLM token. A personal bearer token will be sent to your email or displayed on the page.

To use it from a terminal, set your token and the endpoint URL in that terminal — they last until you close it. In a JupyterHub terminal, or on macOS, Linux or WSL:

export OPENAI_API_BASE="https://ellm.nrp-nautilus.io/v1"
export OPENAI_API_KEY="<paste-your-token-here>"

In Windows PowerShell:

$env:OPENAI_API_BASE = "https://ellm.nrp-nautilus.io/v1"
$env:OPENAI_API_KEY = "<paste-your-token-here>"

Replace the placeholder with your token. Setting these from inside a notebook works a little differently — see the setup check at the start of Lesson 2.

Step 3: Verify Access

From a bash terminal — a JupyterHub terminal, or macOS, Linux or WSL — send a real chat request. This actually exercises the model, not just the endpoint, so it's a more meaningful check than listing models:

curl -s -X POST "$OPENAI_API_BASE/chat/completions" \
     -H "Authorization: Bearer $OPENAI_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"model": "minimax-m2", "messages": [{"role": "user", "content": "What is the National Research Platform?"}]}'

You should get back a JSON response with a real answer buried in it (something about a distributed, Kubernetes-based US research cyberinfrastructure).

For a cleaner look, pipe it through Python to pull out just the reply text:

curl -s -X POST "$OPENAI_API_BASE/chat/completions" \
     -H "Authorization: Bearer $OPENAI_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"model": "minimax-m2", "messages": [{"role": "user", "content": "What is the National Research Platform?"}]}' \
  | python3 -c 'import json, sys; d = json.load(sys.stdin); print(d["choices"][0]["message"]["content"] if "choices" in d else d)'

If the token is wrong, this prints the error message from the endpoint instead of a reply.

You can also list the full model catalog:

curl -s -H "Authorization: Bearer $OPENAI_API_KEY" \
     "$OPENAI_API_BASE/models" \
  | python3 -m json.tool | head -20

On Windows, or if you'd rather stay in Python, the Setup Check at the start of the Chat notebook runs the same test from a notebook cell.

Entry Points at a Glance

How to Get Help

• NRP support chat (Slack / Matrix): https://nrp.ai/contact/
• NRP documentation: https://nrp.ai/documentation/
• LLM-specific docs: https://nrp.ai/documentation/userdocs/ai/llm-managed/

Run the Notebooks

You can run the notebooks for this training either on the NRP training hub or on your own machine (local setup) — the training hub is recommended since the packages are already installed for you. You'll still need your own personal API token either way (see Getting Access above).