Qwen3.6-27B Uncensored

1. 2026-06-23

   Heretic setup on A100

   Loaded Qwen3.6-27B on an A100 80GB and wired up Heretic with harmless-alpaca good prompts and two bad sets: a broad harmful-behaviors set and a hand-built offensive-security set. Both profiles started at 100/100 refusals.

2. 2026-06-24

   Cyber needed a targeted direction

   A broad evaluation gave no gradient at gentle KL, so the cyber run could not find a direction that opened security prompts without wrecking the model. Scoped the bad and eval sets to offensive-security prompts and the Pareto front opened up. Also ran MPOA and staged-abliteration experiments that did not beat the single-direction Heretic result.

3. 2026-06-25

   Selected the trials

   Picked Cyber trial 155 (refusals 34/100, KL 0.255) and Aggressive trial 230 (refusals 41/100, KL 0.214) off the Pareto fronts. Battery on the BF16 checkpoints: perplexity 2.87 and 2.10, no degeneration on neutral prompts, category probes behaving as expected.

4. 2026-06-25

   Quantized to W4A16 on the build pod

   AWQ W4A16 asym g128 MSE with the 208-entry ignore list and the GatedDeltaNet padding monkeypatch. 256×1024 calibration after 512×2048 OOM'd the FFN. Grafted the 15 bf16 MTP tensors back in, verified, and uploaded both W4A16 quants and both BF16 sources. First public release.

5. 2026-06-25

   Served and benchmarked on 2× V100

   Served through 1Cat-vLLM with FLASH_ATTN_V100, TurboMind W4A16, fp8_e5m2 KV, and MTP. Aggressive: openness 0.417, gsm8k 0.85, 58.5 tok/s. Cyber: opens security prompts, gsm8k 0.91, 47.1 tok/s.

6. 2026-06-26

   Medium and ThinkingCap experiments

   Built a gentle Medium tier (openness 0.25) and a set on bottlecapai's ThinkingCap reasoning base. ThinkingCap Medium came out sharpest (gsm8k 0.94) but least open (0.167); a useful middle of the curve rather than a shipping flagship.

7. 2026-06-27

   NVFP4 build of Aggressive

   Quantized Aggressive to NVFP4A16 in compressed-tensors so the SM70 path accepts it at capability 70, with the same MTP graft. Same openness as W4A16, gsm8k 0.89, 61.4 tok/s on V100. Found that a plain full cudagraph capture corrupts the SM70 NVFP4 kernel; full_and_piecewise with greedy MTP is required.

8. 2026-08-15

   Tokenizer fix across all repos

   A user reported image inputs failing with an HTTP 400. The calibration save had written an active truncation block and a drifted pre-tokenizer regex plus phantom audio/TTS tokens into the tokenizer. Restored tokenizer.json and tokenizer_config.json verbatim from the base model on all 15 affected repos; vocab and merges were already identical.

1. 1

   Pick a tier and format

   Aggressive is the most open tier and the flagship. Cyber opens offensive-security prompts specifically. On V100, W4A16 AWQ is the safe default; the NVFP4 build of Aggressive is a little faster for the same openness. All quants are sized for two 32 GB cards.

2. 2

   Download the weights

   About 19 GB for W4A16, 21 GB for NVFP4. The NVFP4 weights live under an mtp-variant-nvfp4/ subfolder on the repo, so point the server at that path.

   shellcopy

   hf download philbert440/Qwen3.6-27B-Uncensored-Aggressive-W4A16-AWQ --local-dir ./q36-aggressive-w4a16

   Aggressive W4A16 AWQ (Hugging Face)Aggressive NVFP4

3. 3

   Install the engine

   Install the current upstream 1Cat-vLLM release (1.5.1). These checkpoints need no engine fork; the SM70 paths for FLASH_ATTN_V100, W4A16 TurboMind, and NVFP4 are all in the stock wheel. On Ampere or newer the W4A16 AWQ builds also run on stock vLLM.

   1Cat-vLLM releases

4. 4

   Serve W4A16 on 2× V100

   This is the model-card serve command: FLASH_ATTN_V100 attention, TurboMind W4A16, tensor-parallel 2, fp8_e5m2 KV, and MTP speculative decoding with four draft tokens. The full_and_piecewise cudagraph mode is required on the SM70 path.

   shellcopy

   export VLLM_SM70_FLASH_ATTN_V100=1 VLLM_SM70_QUANT_BACKEND=turbomind
   python -m vllm.entrypoints.openai.api_server \
     --model philbert440/Qwen3.6-27B-Uncensored-Aggressive-W4A16-AWQ --trust-remote-code --dtype half \
     --attention-backend FLASH_ATTN_V100 --tensor-parallel-size 2 \
     --kv-cache-dtype fp8_e5m2 --max-num-seqs 6 \
     --speculative-config '{"method":"mtp","num_speculative_tokens":4,"attention_backend":"FLASH_ATTN_V100","draft_sample_method":"greedy"}' \
     --compilation-config '{"cudagraph_mode":"full_and_piecewise","cudagraph_capture_sizes":[1,2,4,8]}'

   Model card: serve

5. 5

   Serve NVFP4 (and a note)

   NVFP4 adds the VLLM_SM70_NVFP4_TURBOMIND=1 flag and points --model at the mtp-variant-nvfp4/ subfolder. There is one MTP layer, so num_speculative_tokens of 2 is the sweet spot; higher reuses the same layer and acceptance drops. Keep full_and_piecewise and greedy MTP or the SM70 NVFP4 path corrupts.

   shellcopy

   export VLLM_SM70_FLASH_ATTN_V100=1 VLLM_SM70_QUANT_BACKEND=turbomind VLLM_SM70_NVFP4_TURBOMIND=1
   python -m vllm.entrypoints.openai.api_server \
     --model philbert440/Qwen3.6-27B-Uncensored-Aggressive-NVFP4/mtp-variant-nvfp4 --trust-remote-code --dtype half \
     --attention-backend FLASH_ATTN_V100 --tensor-parallel-size 2 --kv-cache-dtype fp8_e5m2 --max-num-seqs 6 \
     --speculative-config '{"method":"mtp","num_speculative_tokens":2,"attention_backend":"FLASH_ATTN_V100","draft_sample_method":"greedy"}' \
     --compilation-config '{"cudagraph_mode":"full_and_piecewise","cudagraph_capture_sizes":[1,2,4,8]}'

Runs on upstream 1Cat-vLLM, no fork. Serve with the SM70 FLASH_ATTN_V100 backend and the TurboMind quant path.

MTP speculative decoding and an fp8_e5m2 KV cache are part of the served config.

NVFP4 on V100 needs the full_and_piecewise cudagraph mode with greedy MTP; a plain full cudagraph capture corrupts the SM70 NVFP4 path (fixed as the default in 1Cat-vLLM #174).

• A broad eval gave the optimizer no gradient at gentle KL for the cyber profile: at low KL the broad refusal count barely moved. Scoping the bad-prompt set to offensive-security prompts gave the optimizer something to descend, which is why Cyber is a targeted direction and Aggressive is broad.
• 512×2048 calibration overflows an 80 GB card during AWQ smoothing because the FFN is 17408 wide, so 256×1024 was the working ceiling. That is enough samples at this model size, but it is a hard constraint, not a choice.
• transformers drops the MTP head on load, so every quant needs the 15-tensor bf16 graft re-added afterward or speculative decoding has nothing to draft with.
• The calibration run silently corrupted the tokenizer, breaking large image inputs with an HTTP 400 that looked like a serving bug. Copying the two tokenizer files verbatim from the base model fixed it across all repos.
• The Medium and ThinkingCap tiers were side experiments in where to sit on the openness/reasoning curve. Medium is a gentle abliteration at openness 0.25. ThinkingCap builds on bottlecapai's terse reasoning finetune of the same base, and its Medium tier is the sharpest in the set (gsm8k 0.94, openness 0.167) at the cost of being the least open.
• On the composite ranking across all tiers and formats, Aggressive W4A16 AWQ came out on top (score 1.267): it is the most open without a reasoning collapse. The NVFP4 build matches its openness and runs a little faster on V100.
• Earlier multi-direction and staged abliteration experiments (MPOA, staged jailbreak passes) did not beat Heretic's single KL-optimized direction and were dropped.