Single-B300 Validation of Llama 3.3 70B + Runtime LoRA: SDPA Recovery, Telemetry and Next Benchmark Design

Product Q&A
1

Single-B300 Validation of Llama 3.3 70B + Runtime LoRA

Hello Verda community,

I am Edin Vučelj from BPM RED Academy in Bosnia and Herzegovina.

We recently completed a controlled, application-level validation of our FinC2E v2 domain-specific AI engine on a single NVIDIA B300 SXM6 AC instance provided through Verda.

Objective: verify that our actual domain adapter—not a prompt-only prototype or managed endpoint—could be loaded, executed, measured, and documented directly on NVIDIA Blackwell Ultra infrastructure.

1. Validation environment

Component Configuration
GPU 1 × NVIDIA B300 SXM6 AC
Base model Meta Llama 3.3 70B Instruct
Precision BF16
Adapter FinC2E v2 PEFT/LoRA
Adapter parameters 621,281,280
LoRA rank / alpha r = 48 / alpha = 96
Runtime PyTorch 2.11.0 + CUDA 12.8
Libraries Transformers 5.12.1, PEFT 0.19.1, Accelerate 1.14.0
Concurrency 1

2. Attempt 1: model load passed, first inference failed

The base model loaded successfully and the expected parameter count was confirmed: 

Base model parameter count: 70,553,706,496
Base model load time: 10.74 seconds
Failure stage: baseline first inference

The first inference failed with the following cuDNN Frontend error: 

RuntimeError: cuDNN Frontend error:
[cudnn_frontend] Error: No valid execution plans built.

The failed attempt was preserved as part of the evidence package rather than being overwritten. The failure was classified as a cuDNN SDPA execution-plan issue, not a model-weight or adapter-integrity problem.

3. Controlled recovery

For the second attempt, we changed only the problematic backend policy: 

torch.backends.cuda.enable_cudnn_sdp(False)

Flash SDPA: enabled
Math SDPA: enabled
cuDNN SDPA: disabled

The base model, adapter, prompts, output limits, and benchmark structure remained unchanged.

The second attempt completed successfully.

Recovery principle: isolate the failing backend while preserving the workload, model artifacts, and evaluation conditions.

4. Selected performance results

Metric Baseline Llama 3.3 70B FinC2E v2
Warm median TTFT 30.77 ms 49.81 ms
Warm median decode throughput 26.30 output tokens/s 19.31 output tokens/s
Representative warm total time ≈ 4.860 s ≈ 4.556 s
Representative output length 128 tokens 88 tokens
Complete canonical JSON outputs 0/4 4/4

FinC2E v2 was slower per generated token, but it produced shorter, complete, machine-usable outputs.

In the representative warm comparison, the adapter completed the required structured response earlier despite lower decode throughput.

The primary application-level result was not maximum token generation. It was reliable completion of the required structured-output contract.

5. GPU telemetry

Telemetry field Observed value
Average GPU utilization ≈ 90.6%
Maximum GPU utilization ≈ 91%
Maximum GPU memory used ≈ 137,578 MiB
Average power draw ≈ 570 W
Maximum power draw ≈ 585 W
Maximum temperature ≈ 52 °C
Average SM clock ≈ 2,032 MHz

The full BF16 70B model and runtime adapter occupied approximately half of the available B300 memory, leaving substantial headroom for larger KV cache, higher concurrency, or multi-adapter serving experiments.

6. Evidence integrity

The validation package contains:

  • base-model revision and staging evidence;
  • adapter configuration and SHA-256 hash;
  • model-load summaries;
  • Attempt 1 logs and traceback;
  • Attempt 2 backend policy;
  • baseline and adapter benchmark results;
  • GPU telemetry;
  • model-output hashes;
  • final checksum manifests.

Both the external package checksum and the internal evidence-manifest verification passed.

7. What this validation demonstrates

  • The full Llama 3.3 70B model can be loaded in BF16 on a single NVIDIA B300 GPU.
  • The FinC2E v2 runtime LoRA adapter can be loaded and executed successfully.
  • The adapter materially changes the model’s output behavior.
  • FinC2E v2 completed the canonical structured-output contract in 4/4 runs.
  • The failed and successful execution paths can be reconstructed from the evidence package.
  • The workload generated measurable GPU, latency, throughput, memory, power, and temperature data.

8. What this validation does not claim

  • This is not an MLPerf submission.
  • This is not a performance-leadership claim.
  • It is not directly comparable to batched vLLM, TensorRT-LLM, FP8, NVFP4, speculative decoding, or multi-GPU serving.
  • It does not yet establish full production readiness.
  • It does not yet validate the complete HumAI MightHub multi-engine fleet on B300.
Current benchmark scope:
One GPU, BF16, PyTorch Transformers + SDPA, runtime LoRA, short fixed prompts, concurrency 1.

9. Proposed next benchmark phase

Dimension Planned comparison
Serving stack Transformers vs vLLM vs TensorRT-LLM
Concurrency 1, 2, 4, 8, 16 and 32
Latency TTFT p50/p95, TPOT and end-to-end latency
Throughput Per-user output tokens/s and total system tokens/s
Quality Structured-output validity and domain evaluation
Efficiency GPU memory, power and energy per valid response
Routing Single-adapter and multi-LoRA Mission Control
Resilience Fallback, failure classification and evidence continuity

10. Questions for the Verda community and AI infrastructure engineers

  1. What is the preferred attention backend for this B300 and PyTorch workload?
  2. Would vLLM or TensorRT-LLM be the better first optimized serving path for runtime LoRA?
  3. What methodology would you recommend for isolating LoRA overhead on Blackwell Ultra?
  4. Which metrics would make this most useful as a Verda-focused B300 case study?
  5. Has anyone observed the same cuDNN SDPA “No valid execution plans built” error on B300?
  6. For multi-LoRA testing, should adapter-switching latency, concurrent adapter serving, or aggregate throughput be prioritized first?
  7. Would a public latency-throughput Pareto curve combined with structured-output quality be useful to the community?

11. Disclosure policy

We can share:

  • public-safe aggregate metrics;
  • system configuration;
  • methodology;
  • selected telemetry;
  • a redacted evidence manifest;
  • the proposed benchmark matrix.

Proprietary adapter weights, sensitive model outputs, private infrastructure information, and institutional materials will remain protected.

We are interested in technical feedback, reproducibility review, serving-stack optimization, B300 credits for the next benchmark phase, and a possible European AI case-study collaboration.

About the project

FinC2E v2 is a domain-specific financial governance and compliance engine developed within BPM RED Academy’s broader HumAI MightHub architecture.

The longer-term objective is a model-neutral, human-accountable Mission Control layer for regulated and mission-critical workflows, including structured outputs, policy controls, human review, override, model routing, and evidence provenance.

Author: Edin Vučelj
Organization: BPM RED Academy
Location: Bosnia and Herzegovina

1 Like