Immersion Tank First Fill — Contamination From Skipped DI Flush

Read your tank manufacturer's commissioning procedure end-to-end before ordering fluid or scheduling fill day. Most published procedures explicitly call for DI flush, vacuum dehydration, and inline filter on fill. Print the procedure and hang it on the shop wall as the gating checklist. Operators who skip these steps almost always didn't read the doc - not because they decided to deviate. This single planning step costs nothing and is the highest-leverage move in the entire commissioning window.

Order consumables two weeks before fill day: 200 L laboratory-grade DI water (~$200-$400 CAD), 5-10 L of dielectric flush solvent (typically the operating fluid decanted from a separate clean drum), 5-um pre-filter cartridge for the gross flush, 1-um filter cartridge plus housing for the operating fill, three glass sample bottles (Boston round amber, single-use), and the lab panel order paperwork. Long-lead items are the certified lab bottles and lab schedule; order ahead.

Block the shop on fill day. No machining, sanding, sawing, welding, drywall work, or grinding in the same bay or any bay sharing HVAC for the 24 hours preceding fill and 24 hours after. Particulate generated upwind will find an open tank. Schedule fill day for the slowest operations slot, not a rebuild day. A clean shop is the cheapest contamination-control investment in the entire build.

Inspect the empty tank under work light. Wipe every interior surface with a lint-free cloth dampened in fresh dielectric coolant - never water, never IPA on aluminum-bodied tanks, never paper towel. If the wipe comes off the cloth visibly tinted, you've confirmed why a DI flush is mandatory. Reject any tank shipped with VCI bags or VCI paper inserts that haven't been removed and the surfaces wiped clean.

Verify every gasket, lid seal, and cable-gland o-ring is OEM-spec material, fresh from the bag, installed dry. Viton/FKM is the safe default for hydrocarbon dielectrics; silicone is the failure mode (silicone migration). Reject any gasket with extra RTV 'to seal it better' - that RTV is the contamination axis that has no remediation path. Tighten cable-glands to spec torque - under-tight leaks air, over-tight crushes the gasket asymmetrically and creates a worse leak path.

Pre-flush the tank with DI water + dielectric solvent. Pump 100-200 L of laboratory-grade DI water into the tank along with 5-10 L of OEM-approved flush solvent. Run circulation pump 60 minutes at low speed. The DI water lifts polar contaminants (machining-oil bases, VCI, water-soluble dust); the dielectric solvent lifts non-polar contaminants (waxes, hydrocarbon oil, packing residue). Drain through a 5-um pre-filter to a clean disposal drum. Inspect the drained flush - visible tint or particulate confirms the procedure caught what it was meant to catch.

Drain completely, air-dry, then vacuum-dehydrate the empty tank. Wet-vac and absorbent rags clear the bulk water; vacuum pump connected to the sealed tank for 4-8 hours pulls residual water from gasket polymers and seam capillaries. Target a 24-hour vacuum hold at the tank-rated working vacuum without measurable rise. This step alone keeps fill-day Karl Fischer water content below 20 ppm instead of 100-200 ppm. Skipping it pre-injects water content into the operating fluid the day it lands in the tank.

Fill through a 1-um inline filter cartridge mounted in a housing on the supply line between drum and tank inlet. Every drop of operating fluid passes through it. Pump rate slow - under 10 L/min on a residential or light-commercial fill - to give the filter time to actually catch particulate. Watch the filter pressure differential; a sharp climb on a fresh cartridge means you have a dirty drum and the filter is doing exactly what it's supposed to do. This step alone is the highest-leverage $50-$200 CAD spend in the entire commissioning budget.

Capture a day-zero baseline sample within 30 minutes of fill completion, before any miner is energized. Mid-depth, centre of tank. 250 mL Boston round amber glass bottle, single-use. Label with date, tank ID, fluid brand and lot number, ambient conditions. Ship to Insight Services, Polaris Labs, ALS Canada, Wear Check, or Bureau Veritas for the ASTM D877 + ASTM D6304 + ISO 4406 panel. ~$80-$160 CAD, 3-7 business days. This sample is your warranty evidence - without it, every later degradation claim is contested by the manufacturer.

Run a 24-hour cold-loop circulation before energizing the first miner. Pump on, miners off and still on the bench or in the dry-bay. The loop equalizes temperature, distributes fresh fill into stagnant zones (corners, drain ports, sensor wells), and gives the inline filter a final pass at any post-fill particulate. At hour 24, pull a second sample - particle count should already be at or better than ISO 4406 18/16/13. This is the final go/no-go gate before energizing silicon.

REMEDIATION (tank already filled with procedure skipped): add a 1-um inline filter to a running tank. Plumb a filter housing on the return side of the pump (between heat exchanger and tank inlet). Replace cartridge at hour 24, hour 72, day 7, day 14. Pull spent cartridges and photograph each for the service record. Particulate-only contamination clears in 7-10 days at typical immersion pump rates. Cost: ~$80-$200 CAD per cartridge, ~$300-$700 CAD for housing if not already installed.

REMEDIATION: vacuum-dehydrate the running loop if Karl Fischer water content exceeds 50 ppm. Rent a Pall HVP-100, Hydac FAM-25, or Des-Case unit ($300-$800 CAD per week of rental plus operator time). Connect to the running loop in bypass - fluid pulled out, dehydrated, returned. Run 24-72 hours per the unit's procedure. Pull a post-cycle sample. Successful cycle drops 200 ppm water content to under 20 ppm and recovers most of the breakdown loss. Does not remove silicone, oil, or oxidation byproducts - those failure modes still demand fluid replacement.

REMEDIATION: drain, lab-test, and decide replacement vs recovery for a multi-axis-contaminated loop. If lab returns ISO 4406 21/19/16 AND Karl Fischer >100 ppm AND D877 <25 kV, you have multiple contamination axes open and the cheapest path is full drain plus fresh-fluid order plus proper commissioning procedure on refill. Math: 200 L tank refill at $8-$22 CAD per litre = $1,600-$4,400 CAD; one cooked hashboard at $400-$900 CAD plus labour matches it. A multi-axis-contaminated loop typically cooks 1-3 boards before drain - pay the fluid bill, save the silicon.

Inspect every miner pulled during a remediation drain. Loupe or stereo microscope on every hashboard. Looking for early-stage carbon tracking (faint brown branching across solder mask, not yet black), pad discolouration adjacent to high-voltage rails, solder-mask blistering. Boards with early brown tracking can occasionally be cleaned with isopropyl alcohol and a fine fibre brush IF the tracking has not yet carbonized into the FR4. Black tracking is permanent; brown is sometimes recoverable. Document the inspection per-board - this is the warranty-desk evidence.

Refill following the full commissioning procedure as if it were a new tank, with one addition: document the prior-state contamination in the service record. Day-zero baseline on the refill becomes the new warranty floor. Run the post-refill 24-hour cold loop. Resubmerge miners in stages - first miner solo for 24 hours with a 12-hour D877 sample, second cohort once first sample lands clean, and so on. This catches reseal failures and residual contamination before propagation.

Stop DIY and ship to D-Central when: multiple hashboards across multiple miners in the same first-90-day tank show carbon tracking (cascade event); PSU teardown reveals secondary-side rectifier or MOSFET damage; lab returns multi-axis contamination (water + silicone + particulate simultaneously); you do not own a programmable DC load, portable D877 tester, or vacuum dehydration unit; two contamination events in the same loop within 12 months; or the operation is in warranty-claim territory and needs a forensic-grade service record.

D-Central first-fill-failure remediation bench process: full miner teardown, per-board carbon-tracking inspection under stereo microscope, PMIC and voltage-domain electrical test under programmable DC load with active rail monitoring, per-chip SHA-256d validation on the test fixture, full PSU bench-test under nameplate load with output-ripple capture, gasket and seal replacement on every chassis, 24-hour pre-deployment burn-in at hashing load. Every miner leaves the bench with a service record showing exactly what was tested and replaced - the document warranty desks demand.

Ship contaminated-loop hardware correctly. Drain each miner 30 minutes minimum hanging over a drip tray, then 30 minutes on absorbent pads, then bag in a heavy-duty contractor bag (residual fluid film weeps for hours). Cap any open coolant fittings. Double-box with 5+ cm of foam every side. Label outside: 'IMMERSION-EXPOSED - DRAINED - FRAGILE' and 'POSSIBLE RESIDUAL DIELECTRIC FLUID - HANDLE WITH GLOVES.' Include a service note: tank fluid age and brand, days-since-fill, observed failure timeline, lab panel results, procedure-compliance answers (DI flush yes/no, vacuum dehydrate yes/no, inline filter yes/no, day-zero baseline yes/no).