Humidity Ratio for HVAC Latent Load: Multi-Input Calculation with Altitude Correction and Wet-Bulb Approximation Limits
← Back to Blog
Psychrometrics May 3, 2026 17 min read

Humidity Ratio for HVAC Latent Load: Multi-Input Calculation with Altitude Correction and Wet-Bulb Approximation Limits

Why RH Understates Latent Load and Why Humidity Ratio Corrects It

Latent cooling load on a coil equals ṁ_dry-air × ΔW × h_fg, and the only correct moisture variable in that equation is humidity ratio (W in gr/lb or kg/kg), not relative humidity. This distinction is absent from most comfort-control discussions, but in coil sizing it governs everything. Engineers using ΔRH instead of ΔW for cooling coil latent load systematically underestimate 15–30% when entering-air RH exceeds 60%, because W = 0.621945 × P_v / (P_atm − P_v) depends on actual vapor pressure non-linearly per ASHRAE Fundamentals 2021 Chapter 1 Equation 22. The RH-based shortcut holds only when entering and leaving dry-bulb temperatures are nearly identical — a condition that contradicts the fundamental purpose of a cooling coil.

The consequences extend beyond sizing margin. Per ASHRAE Standard 62.1-2022 Section 6.2.2 ventilation rate procedure, outdoor-air dehumidification capacity shortfall propagates through the cooling coil to space relative-humidity excess. ASHRAE Standard 160-2021 Section 5.1 places a 60% RH ceiling for mold growth risk; once humidity ratio control fails, sensible-only coil operation cannot recover space conditions without a reheat penalty. This calculator derives W from dry-bulb plus one of four moisture inputs — RH, wet-bulb, dew point, or direct entry — per ASHRAE Fundamentals 2021 Chapter 1 Section 1.6 methodology. The Magnus saturation equation per Alduchov & Eskridge (1996) provides ±0.4% accuracy from −40°C to 50°C across the full HVAC operating range.

Magnus Equation, Four Moisture-Input Modes, and Derived Properties

Step 1: Saturation Vapor Pressure (Magnus approximation):

Metric: P_sat = 0.61078 × exp(17.625 × T / (243.04 + T)) [kPa]
Imperial: P_sat = 0.08855 × exp(17.625 × Tc / (243.04 + Tc)) [psi], where Tc = (T_°F − 32) / 1.8

Magnus accuracy ±0.4% from −40°C to 50°C per Alduchov & Eskridge (1996). ASHRAE Fundamentals 2021 Chapter 1 Equations 5–6 (Hyland-Wexler) provide ±0.05% from −100°C to 200°C for applications outside the HVAC range.

Step 2: Humidity Ratio — four input modes

Mode A (RH input):
P_v = (RH/100) × P_sat
Metric: W = 621.945 × P_v / (P_atm − P_v) [g/kg]
Imperial: W = 4350 × P_v / (P_atm − P_v) [gr/lb]

The constant 621.945 = 1000 × M_w / M_a = 1000 × (18.015 / 28.966) per ASHRAE Fundamentals 2021 Chapter 1 Equation 22; 4350 = 7000 × (18.015 / 28.966) for gr/lb units.

Mode B (Wet-bulb input):
Metric: W = 1000 × [(2501 − 2.326 × T_wb) × W_sat(T_wb)/1000 − 1.006 × (T_db − T_wb)] / (2501 + 1.86 × T_db − 4.186 × T_wb) [g/kg]
Imperial: W = 7000 × [(1093 − 0.556 × T_wb) × W_sat(T_wb)/7000 − 0.240 × (T_db − T_wb)] / (1093 + 0.444 × T_db − T_wb) [gr/lb]

Adiabatic saturation energy balance per ASHRAE Fundamentals 2021 Chapter 1 Equation 33. The first bracket in the numerator is the latent heat of vaporization taken at the wet-bulb temperature, the second term is the sensible heat the air surrenders to drive that evaporation, and the denominator is the enthalpy of vapor at T_db less the enthalpy of make-up water at T_wb. W appears linearly, so this direction is solved in closed form; only the inverse problem — recovering T_wb from a known W — needs Newton-Raphson.

The field shortcut W ≈ W_sat(T_wb) − C × (T_db − T_wb) linearizes the same balance, with C ≈ 0.407 g/kg per °C or 1.58 gr/lb per °F. C is a function of both temperatures rather than a fixed number, and the two unit-system values must stand in the ratio 7 / 1.8 = 3.889 to describe the same physics — a pair that does not is a reliable sign that one of them is wrong.

Mode C (Dew point input):
W = W_sat(T_dp)
Direct equality by definition: dew point is the exact temperature at which air saturates at the current moisture content per ASHRAE Fundamentals 2021 Chapter 1 Section 1.6.

Mode D (Direct W entry):
Used when W is already specified in design documents — ASHRAE Fundamentals 2021 Chapter 14 climate data tables, refrigeration system performance ratings per AHRI Standard 540-2020.

Variable definitions and typical HVAC ranges:
T_db: dry-bulb temperature, −10 to +50°C (14 to 122°F)
T_wb: wet-bulb temperature, ≤ T_db; typical HVAC −5 to +35°C
T_dp: dew point temperature, ≤ T_db; typical HVAC −20 to +25°C
RH: relative humidity, 0–100%
W: humidity ratio, 0–30 g/kg (0–210 gr/lb) for HVAC; industrial drying reaches 100+ g/kg
P_v: partial vapor pressure, 0–12 kPa for HVAC
P_atm: standard atmospheric pressure 101.325 kPa (14.696 psi); altitude correction per ASHRAE Fundamentals 2021 Chapter 1 Equation 3

Derived calculator outputs:
Dew point via inverse Magnus: α = ln(P_v / 0.61078); T_dp = 243.04 × α / (17.625 − α)
Saturation humidity ratio W_sat at T_db (Mode A formula with RH = 100%)
Degree of saturation μ = W / W_sat × 100
Relative humidity when not a direct input: RH = P_v / P_sat × 100

Field Measurement: Which Instrument Matches Which Input Mode

Mode A (RH input): Capacitive RH sensors are the standard BMS sensor type. Vaisala HUMICAP HMP110 provides ±2% RH with drift ±1%/year; Honeywell HIH-4030 provides ±3.5% RH with drift ±1.2%/year. Calibration interval per ASHRAE Standard 111-2008: 12 months. Reference standard per ASTM E337-15: chilled-mirror hygrometer (±0.2°C dew point). Capacitive sensor hysteresis is 1–3% RH between rising and falling cycles per Vaisala technical documentation — relevant only during rapid environmental transitions such as commissioning measurements in a space with fluctuating conditions.

Mode B (Wet-bulb input): Sling psychrometer is the legacy reference instrument. Wick maintenance is critical: distilled water only, cotton wick replaced quarterly per ASHRAE Standard 41.6-2014 (RA2021). Aspiration airflow must exceed 4.5 m/s (900 fpm) for accurate wet-bulb depression. Per HVAC-Talk forum discussion (thread 305312), a sling psychrometer reads systematically high with a contaminated wick or aspiration below 500 fpm; it never reads false low. In-duct wet-bulb probes (e.g., Shortridge wet-sock probe in a 1/4-inch drilled hole) account for duct heat infiltration in return air, providing the actual equipment-entering wet-bulb. Sling measurements taken in an attic environment underestimate equipment-entering wet-bulb by 1–3°F per HVAC-Talk thread 1764071. Aspirated Assmann-type psychrometers are preferred for stationary measurements and for cooling tower commissioning per CTI ATC-105-2019 Section 5.3.

Mode C (Dew point input): Chilled-mirror hygrometers (RH Systems 933, GE Optica, Vaisala DM70) provide ±0.1–0.2°C dew point accuracy per ASTM E337-15. Cost range $3,000–15,000 limits their use to calibration labs or commissioning-grade applications. BMS systems computing dew point from RH + dry-bulb inherit RH sensor error: ±2% RH translates to ±0.5–0.8°C dew point uncertainty per error propagation analysis in ASHRAE Standard 41.6 commentary.

Mode D (Direct W entry): ASHRAE Fundamentals 2021 Chapter 14 climate data tables list outdoor humidity ratio at 99.6%, 99%, and 1%/0.4% design conditions directly. Refrigeration applications per AHRI Standard 540-2020 specify W in equipment performance ratings.

Latent Load Calculation for Outdoor Air Ventilation in Houston (Climate Zone 2A)

Project: 100-person commercial office in Houston, TX. Outdoor air ventilation requirement: 500 L/s (1,060 CFM) per ASHRAE Standard 62.1-2022 Table 6.2.2.1 (0.06 cfm/ft² + 5 cfm/person for office space category).

Design data per ASHRAE Fundamentals 2021 Chapter 14 (Houston Bush Intercontinental Airport, Site 722430):
Outdoor 0.4% summer design: T_db = 35°C / T_wb_coincident = 25.6°C (95°F / 78°F)
Indoor design per ASHRAE Standard 55-2023 Section 5.2.4: T_db = 24°C / RH = 50%

Step 1: Indoor humidity ratio (Mode A — RH input)

P_sat_indoor = 0.61078 × exp(17.625 × 24 / 267.04) = 0.61078 × exp(1.584) = 2.978 kPa
P_v_indoor = 0.50 × 2.978 = 1.489 kPa
W_indoor = 621.945 × 1.489 / (101.325 − 1.489) = 9.28 g/kg dry air
T_dp_indoor: α = ln(1.489 / 0.61078) = 0.891; T_dp = 243.04 × 0.891 / 16.734 = 12.9°C (55.3°F)

Imperial cross-check (indoor T_db = 75.2°F):
P_sat = 0.08855 × exp(1.584) = 0.4316 psi
P_v = 0.50 × 0.4316 = 0.2158 psi
W = 4350 × 0.2158 / (14.696 − 0.2158) = 64.8 gr/lb (≈ 65 gr/lb)

Step 2: Outdoor humidity ratio (Mode B — wet-bulb input)

W_sat at T_wb = 25.6°C:
P_sat_25.6 = 0.61078 × exp(17.625 × 25.6 / 268.64) = 0.61078 × exp(1.679) = 3.276 kPa
W_sat(25.6) = 621.945 × 3.276 / (101.325 − 3.276) = 20.78 g/kg

Numerator = (2501 − 2.326 × 25.6) × 0.020779 − 1.006 × (35 − 25.6) = 2441.45 × 0.020779 − 9.456 = 50.73 − 9.46 = 41.274
Denominator = 2501 + 1.86 × 35 − 4.186 × 25.6 = 2501 + 65.10 − 107.16 = 2458.94
W_outdoor = 1000 × 41.274 / 2458.94 = 16.79 g/kg (117.5 gr/lb)

Cross-check: this state carries a vapor pressure of 2.663 kPa against P_sat(35°C) = 5.616 kPa, so RH = 47.4% and dew point = 22.2°C. Entering 35°C at 47.4% RH through Mode A returns the same 16.79 g/kg, which confirms the wet-bulb and RH inputs describe one point on the chart. For final design take the outdoor humidity ratio from the ASHRAE Fundamentals 2021 Chapter 14 tables for the station and design percentile actually in use — the DB/MCWB and the DP/MCDB conditions sit at different moisture contents and are not interchangeable.

Step 3: Latent load per ASHRAE Handbook HVAC Systems 2024 Chapter 23 Section 23.4

ρ_air at 35°C ≈ 1.13 kg/m³ per ASHRAE Fundamentals 2021 Chapter 1 Equation 28
ṁ_dry-air = 0.500 m³/s × 1.13 kg/m³ = 0.565 kg/s

ΔW = 16.79 − 9.28 = 7.51 g/kg = 0.00751 kg/kg
Q_latent = 0.565 × 0.00751 × 2501 = 10.61 kW (36,200 BTU/hr)

Imperial cross-check (ASHRAE 0.68 latent constant):
ΔW = 117.5 − 64.8 = 52.7 gr/lb
Q_latent = 0.68 × 1,060 × 52.7 = 38,000 BTU/hr ≈ 11.13 kW

The 4.9% spread between the two lines is a density bookkeeping difference, not a psychrometric one: the 0.68 constant carries standard air density 0.075 lb/ft³ (1.2 kg/m³), while the metric line uses 1.13 kg/m³ evaluated at the 35°C outdoor condition. Either is defensible for sizing provided the same density basis carries through the sensible calculation.

Q_sensible = 0.565 × 1.006 × (35 − 24) = 6.25 kW
Q_total_OA = 10.6 + 6.3 = 16.9 kW (57,600 BTU/hr, 4.8 tons)

Engineering decision on this result:

Three coil system configurations apply. Option A: mixed-air cooling coil (outdoor air blended with return air before the coil). Risk is that if the system operates at 100% outdoor air, mixed-air SHR assumptions do not apply. Option B: Dedicated Outdoor Air System (DOAS) per ASHRAE Standard 62.1-2022 Section 6.2.2, handling outdoor air independently. DOAS coil requires deep-coil low SHR design (SHR 0.5–0.7) per ASHRAE Handbook HVAC Systems 2024 Chapter 4, with supply temperature 12–13°C to match indoor dew point. Option C: enthalpy recovery ventilator (ERV) per AHRI Standard 1060-2018, latent effectiveness 50–65% for enthalpy-wheel units; reduces coil Q_latent from 10.61 kW to 10.61 × (1 − 0.55) = 4.8 kW. Per ASHRAE Standard 90.1-2022 Section 6.5.6.1, energy recovery is required at 1,060 CFM against a 95°F outdoor design in Houston. Selected option: Option C (ERV) with coil Q_latent = 4.8 kW.

Where the Simplified Wet-Bulb Approximation Exceeds 2% Error and When to Switch to an Iterative Solver

The field shortcut W ≈ W_sat(T_wb) − C × (T_db − T_wb) linearizes the same balance, and it fails in one consistent direction: it always reads high. The coefficient is not the culprit. Evaluated properly as C = 1.006 / (2501 + 1.86 × T_db − 4.186 × T_wb), it holds between 0.402 and 0.409 g/kg per °C across the entire HVAC range — stable to about 1%. The error comes from the shortcut's second assumption. The full balance does not carry W_sat(T_wb) at full weight; it scales that term by (2501 − 2.326 × T_wb) / (2501 + 1.86 × T_db − 4.186 × T_wb), a factor that sits below unity and falls further as the depression widens.

Shortcut error against the full balance, taking C = 0.407 g/kg per °C:

15°C db / 12°C wb (RH 71%): +0.2%
24°C db / 17°C wb (RH 50%): +0.7%
35°C db / 25.6°C wb (RH 47%): +1.0%
40°C db / 25°C wb (RH 30%): +1.7%
45°C db / 20°C wb (RH 7%): +3.6%
70°C db / 35°C wb (RH 11%): +4.1%

The pattern tracks the depression rather than the temperature: a 3°C depression at 15°C costs 0.2%, while a 25°C depression at 45°C costs 3.6%. Mode B in this calculator evaluates the full balance per ASHRAE Fundamentals 2021 Chapter 1 Equation 33, so none of that error reaches the result. What remains are the assumptions inside Equation 33 itself — ideal-gas behaviour, constant specific heats, and the above-freezing form of the latent heat term, which stops applying once the wick ices below 0°C.

Practical guidance by application:
HVAC range (T_db −10 to +40°C, RH 30–95%): Mode B is the direct route; a shortcut result hand-checked against it in this range should be expected to sit roughly 1% high.
ASHRAE Climate Zone 1A–2A summer outdoor design (T_db 32–38°C, RH 50–70%): the psychrometrics are solid here, so the remaining uncertainty is the design percentile, not the equation — cross-check the outdoor W against the ASHRAE Fundamentals 2021 Chapter 14 tabulated values for the station in use.
Industrial dryer applications (T_db 60–150°C): the constant-specific-heat assumption behind Equation 33 degrades; use the Hyland-Wexler formulation per ASHRAE Fundamentals 2021 Chapter 1 Equations 5–6, or validated software (ASHRAE LibHuAirProp, EES, REFPROP).
Cryogenic applications (T_db below −50°C): use IAPWS-IF97 industrial steam tables.

In field work the wet-bulb reading carries far more leverage on W than the choice of equation does. At 35°C dry-bulb, a 0.5°C error in wet-bulb shifts W by 0.67 g/kg at a 15°C depression and by 1.25 g/kg near saturation — in relative terms up to 8% in dry air, roughly twice the worst-case shortcut error above. Instrument discipline therefore governs the answer: confirm aspiration velocity and wick condition per ASHRAE Standard 41.6-2014 (RA2021) before attributing a discrepancy to the psychrometric route. Where a calibrated capacitive RH sensor or a chilled-mirror dew point reading is available, Mode A or Mode C removes that leverage entirely.

Mode selection guide: use Mode A when BMS capacitive RH sensor data is available; Mode B for field commissioning with an aspirated psychrometer (verify airflow velocity per ASHRAE Standard 41.6-2014); Mode B for cooling tower analysis where wet-bulb is the primary measured variable per CTI ATC-105-2019; Mode C when specifying compressed air dryers per ISO 8573-3:2002, where dew point is the primary performance specification.

Altitude-Induced Humidity Ratio Shift: 22% Increase at Denver, 32% at Mexico City

Applying sea-level psychrometric formulas at elevation introduces a systematic error that grows with altitude. Since W = 0.621945 × P_v / (P_atm − P_v), any reduction in P_atm increases W for identical temperature and relative humidity. Altitude-pressure correlation per ASHRAE Fundamentals 2021 Chapter 1 Equation 3:

P_atm(Z) = P₀ × (1 − 6.8754×10⁻⁶ × Z)^5.2559, where Z is elevation in feet and P₀ = 101.325 kPa.

Calculated values for HVAC design reference cities:
Sea level: 101.325 kPa (14.696 psi)
Atlanta (1,010 ft): 97.7 kPa (−3.6%)
Denver (5,280 ft): 83.4 kPa (−17.7%)
Mexico City (7,350 ft): 77.2 kPa (−23.9%)
Bogota (8,660 ft): 73.4 kPa (−27.6%)

Humidity ratio shift at T_db = 24°C / RH = 50%, where P_v = 1.489 kPa is fixed by temperature and RH alone and only the denominator changes:
Sea level: W = 9.28 g/kg
Denver: W = 11.30 g/kg (+21.8%)
Mexico City: W = 12.24 g/kg (+32.0%)
Bogota: W = 12.88 g/kg (+38.9%)

Eng-Tips forum thread 225425 (HVAC design at altitude): engineer designing a Salt Lake City system (elevation ~4,250 ft, P_atm ≈ 87 kPa) asked how to correct sensible and latent load equations. Per ASHRAE Fundamentals 2021 Chapter 1 commentary, both equations scale by the ratio (P_local / P_sea-level):

Sensible: Q_s = 1.1 × CFM × ΔT × (P_local / P_sea-level) ≈ 0.94 × CFM × ΔT at Salt Lake City
Latent: Q_l = 4,840 × CFM × ΔW × (P_local / P_sea-level) ≈ 4,160 × CFM × ΔW at Salt Lake City

Using the sea-level constant 1.1 or 4,840 without altitude correction overestimates load by 21% at Denver elevation, since P_local / P_sea-level = 0.823 there and the uncorrected constant is 1/0.823 too large. An AHU selected at sea-level airflow ratings delivers 18% less mass airflow at Denver, leaving the space under-cooled. Elevation guidance: below 1,500 ft sea-level formulas are acceptable (error < 6%); 1,500–3,000 ft requires an explicit (P_local / P_sea-level) correction; above 3,000 ft altitude correction is mandatory; above 7,000 ft Magnus equation accuracy degrades and the Hyland-Wexler formulation is required. For elevation pressure calculation, see the Air Density Calculator applying ASHRAE Fundamentals 2021 Chapter 1 Equation 3.

Unit Confusion: gr/lb vs lb/lb vs g/kg, and Where 7000× Errors Arise in Latent Calculations

ASHRAE psychrometric formulas use three W unit conventions simultaneously, with no consistent labeling across source documents:

The ASHRAE enthalpy formula h = 0.240 × T_°F + W × (1061 + 0.444 × T_°F) uses W in lb/lb (water mass per pound of dry air).
The ASHRAE latent load formula Q (BTU/hr) = 0.68 × CFM × ΔW uses ΔW in gr/lb (grains per pound of dry air).
Metric formulas and SI psychrometric charts use W in kg/kg or g/kg.

Conversion factors: 1 lb = 7,000 grains. Therefore W (gr/lb) = W (lb/lb) × 7,000, and W (g/kg) / 7 ≈ W (gr/lb). Exact metric-to-imperial: W (g/kg) × 0.99988 / 0.142857 = W (gr/lb).

Common error sources per Eng-Tips forum thread 415398 (chilled-water calculations discussion):
Engineer reads W = 0.0093 lb/lb from a psychrometric chart, plugs it into Q = 0.68 × CFM × ΔW expecting gr/lb — result is 7,000× too small.
Engineer reads W = 65 gr/lb from a digital psychrometer, plugs it into the ASHRAE enthalpy formula expecting lb/lb — result is 7,000× too large.
Mixing constants: 0.68 (gr/lb basis) and 4,840 (lb/lb basis) for latent load are both correct, but require matching ΔW units.

Verification check: in HVAC applications, humidity ratio falls in 0.005–0.020 lb/lb or 35–140 gr/lb. Any intermediate result outside these ranges indicates likely unit confusion.

Humidity ratio W = m_v / m_a (per pound of dry air) per ASHRAE Fundamentals 2021 Chapter 1 Equation 21 should not be confused with specific humidity q = m_v / (m_a + m_v) = W / (1 + W), which the meteorological community often uses. At W below 0.020, the difference between W and q is less than 2%; at industrial drying W above 0.040, the difference exceeds 4%. For HVAC applications, default to humidity ratio unless a specific process standard requires specific humidity.

Humidity Ratio Calculator

Humidity ratio calculation with automatic conversion between gr/lb and g/kg, support for all four moisture-input modes (RH, wet-bulb, dew point, direct W entry), and derived properties (dew point, vapor pressure, degree of saturation, W_sat, RH) is available in the Humidity Ratio Calculator.

FAQ

Why does the standard latent load formula Q = 0.68 × CFM × ΔW use the constant 0.68? Where does that number come from?

The constant 0.68 derives from imperial unit reconciliation: 0.68 = 60 min/hr × 0.075 lb/ft³ × 1076 BTU/lb ÷ 7000 gr/lb. The components are: 60 min/hr converts CFM to ft³/hr; 0.075 lb/ft³ is standard air density at sea-level 70°F; 1076 BTU/lb is approximate latent heat of vaporization at typical partial vapor pressure; 7000 gr/lb converts grains to pounds for dimensional consistency when ΔW is expressed in gr/lb. The formula applies at standard atmospheric pressure 14.696 psi and standard air density 0.075 lb/ft³. Altitude correction multiplies 0.68 by (P_local / P_sea-level) per ASHRAE Fundamentals 2021 Chapter 1 commentary.

Mixing 80% return air at 80°F / 48% RH with 20% outdoor air at 103°F / 100% RH: is W_mixed = 0.8 × 75 gr/lb + 0.2 × 325 gr/lb = 125 gr/lb correct, or do I need a mass-flow-weighted average?

Linear mass-flow-weighted averaging is correct when mixing fractions represent dry-air mass flow rates, not volumetric rates. Per ASHRAE Fundamentals 2021 Chapter 1 Section 1.8 (Mixing of Two Streams), W_mixed = (ṁ_a1 × W_1 + ṁ_a2 × W_2) / (ṁ_a1 + ṁ_a2), where ṁ_a is mass flow of dry air. When mixing fractions are stated as CFM ratios, a density correction applies: ṁ = CFM × ρ_air. In this example, outdoor air at 103°F has density 0.071 lb/ft³ versus return air at 80°F at 0.074 lb/ft³ — a density ratio of 0.96 — so the volumetric average underestimates W_mixed by approximately 1–2% per Eng-Tips forum thread 160848 analysis.

Sling psychrometer reads 90% RH while a Fluke 971 reads 73% RH in the same room. Which instrument should I trust?

A sling psychrometer reads systematically high when aspiration velocity is below 4.5 m/s (900 fpm), the wick is contaminated, or non-distilled water is used, per ASHRAE Standard 41.6-2014 (RA2021) and HVAC-Talk forum thread 305312 consensus. The Fluke 971 capacitive sensor provides ±3% RH per the manufacturer's specification but is subject to calibration drift of ±1–2% per year per ASHRAE Standard 111-2008. The reference standard for resolving disagreements is a chilled-mirror hygrometer (RH Systems 933, GE Optica) with ±0.2°C dew point accuracy per ASTM E337-15. A properly aspirated sling psychrometer with a clean cotton wick is accurate within ±2% RH; capacitive sensors are more reliable in mid-range RH (35–75%); both degrade at extremes.

Sizing a DOAS coil from 100°F / 63°F WB to 55°F / 55°F gives Q_total = 92,907 BTU/hr, but Q_sensible = 142,358 BTU/hr and Q_latent = −54,612 BTU/hr. Is the negative latent an error?

The negative figure is real arithmetic on an impossible pair of states, and it points at the leaving condition rather than the entering one. Air at 100°F dry-bulb with a 63°F wet-bulb is a 37°F depression — a desert condition at roughly 9% RH, dew point 32°F, W = 26.6 gr/lb. Air specified as leaving at 55°F saturated carries W = 64.1 gr/lb, so the stated process demands ΔW = −37.5 gr/lb. A cooling coil cannot add moisture, so the leaving state as written cannot occur. Resolution: coil leaving W must equal min(W_entering, W_sat at the coil surface temperature). With entering W at 26.6 gr/lb and a surface temperature well above the 32°F entering dew point, no condensation occurs at all — the coil runs fully sensible and the leaving state is 55°F at 26.6 gr/lb, close to 42% RH, not saturation. Q_latent is zero and the full 142,358 BTU/hr is sensible. Per Eng-Tips forum thread 486893, specifying a saturated leaving condition by habit is the common origin of this result; check the entering dew point against the coil surface temperature before assuming any dehumidification takes place.

Sling psychrometer in the attic reads 3°F lower wet-bulb than an in-duct probe at the same equipment. Which represents the coil-entering condition?

The in-duct probe represents actual coil-entering wet-bulb per ASHRAE Standard 41.6-2014 (RA2021) Section 5 measurement protocol. The sling measurement in the attic captures attic ambient air before duct heat infiltration changes the air state. Per HVAC-Talk thread 1764071, attic heat infiltration raises return air dry-bulb by 3–5°F and wet-bulb by 1–3°F before the air reaches equipment. For commissioning measurements, use an in-duct probe (Shortridge wet-sock probe in a drilled hole) for representative coil-entering readings. ASHRAE Standard 90.1-2022 Section 6.4.4.1 requires R-6 minimum return duct insulation to limit this heat gain.

Related Calculations

The complete moist-air state from dry-bulb and RH — six properties including enthalpy, specific volume, and vapor pressure — is calculated in the Psychrometric Calculator, which applies the same Magnus equation framework per ASHRAE Fundamentals 2021 Chapter 1. Cooling coil total capacity from entering and leaving enthalpy states is handled in the Enthalpy Calculator per ASHRAE Handbook HVAC Systems 2024 Chapter 23 Section 23.4. Wet-bulb temperature determination for cooling tower and evaporative cooling design uses the Wet Bulb Temperature Calculator per CTI ATC-105-2019 acceptance test methodology. Condensation risk assessment on chilled water piping and supply diffusers relies on dew point via the Dew Point Temperature Calculator per ASHRAE Standard 160-2021 Section 5. Dehumidification system latent capacity sizing uses the Latent Heat Load Calculator per ASHRAE Handbook HVAC Systems 2024 Chapter 23. Cooling coil sensible heat ratio for coil row depth selection is covered by the Sensible Heat Ratio Calculator. Airflow mass-rate conversion for DOAS and ERV applications uses the Specific Volume Air Calculator.