Tail-call infoz reporting in PrepareInsertLargeImpl to avoid stack frame. In `PrepareInsertLargeImpl`, computing `SlotAddress` before `SetCtrl` and keeping the returned slot pointer and `has_infoz` live across `SetCtrl` and the conditional call to `ReportInsertMissToInfozAndComputeProbeLength` forces Clang to spill a callee-saved register and allocate a stack frame (`pushq %rbp; movq %rsp, %rbp; pushq %rbx; pushq %rax` and matching epilogue) on the hot fast path of `PrepareInsertLarge`, even when `common.has_infoz()` is false. Change `ReportInsertMissToInfozAndComputeProbeLength` to `ReportInsertMissToInfozAndGetSlot(common, policy, hash, target)`, computing and returning `SlotAddress` inside the cold helper, and check `common.has_infoz()` after `SetCtrl` so `PrepareInsertLargeImpl` can tail-call `ReportInsertMissToInfozAndGetSlot` when `common.has_infoz()` is true and sink the `SlotAddress` arithmetic into the non-infoz fast path. Passing `hash` as the third argument (`%rdx`) and `target` as the fourth argument (`%rcx`) matches `PrepareInsertLarge` and `PrepareInsertLargeSlow`, avoiding any register shuffling on entry or before the tail calls. This eliminates the stack frame and callee-saved register spill on `PrepareInsertLarge`. PiperOrigin-RevId: 999623392
diff --git a/absl/container/internal/raw_hash_set.cc b/absl/container/internal/raw_hash_set.cc index 3465a90..6b0dbba 100644 --- a/absl/container/internal/raw_hash_set.cc +++ b/absl/container/internal/raw_hash_set.cc
@@ -1103,18 +1103,23 @@ probe_length); } -// Outlines cold infoz recording with computation of probe_length so callers can -// skip tracking of probe_length. `common` is first and `hash` is third to match -// argument order in callers. -ABSL_ATTRIBUTE_COLD ABSL_ATTRIBUTE_NOINLINE void -ReportInsertMissToInfozAndComputeProbeLength(CommonFields& common, - size_t target, size_t hash) { +// Outlines cold infoz recording with computation of probe_length and slot +// address so callers can skip tracking probe_length and tail-call without +// keeping the slot address live across a call (which would cost a callee-saved +// register and a stack frame on every insert). +// (Returning a value passed in by the caller doesn't work: the compiler infers +// that and keeps the caller's copy alive across the call instead.) +ABSL_ATTRIBUTE_COLD ABSL_ATTRIBUTE_NOINLINE void* +ReportInsertMissToInfozAndGetSlot(CommonFields& common, + const PolicyFunctions& __restrict policy, + size_t hash, size_t target) { const size_t cap = common.capacity(); auto seq = probe(ProbeCapacity{cap}, hash); while (((target - seq.offset()) & cap) >= Group::kWidth) { seq.next(); } common.infoz().RecordInsertMiss(hash, seq.index()); + return SlotAddress(common.slot_array(cap), target, policy.slot_size); } ABSL_ATTRIBUTE_COLD ABSL_ATTRIBUTE_NOINLINE void ReportGrowthToInfoz( @@ -2454,12 +2459,11 @@ // Compute before `SetCtrl` to avoid reloading `control_`. void* res = SlotAddress(common.slot_array(cap), target_group_offset, policy.slot_size); - const bool hash_infoz = common.has_infoz(); SetCtrl(common, target_group_offset, H2(hash), policy.slot_size); // Call NOINLINE function to move infoz instructions out of line. - if (hash_infoz) { - ReportInsertMissToInfozAndComputeProbeLength(common, target_group_offset, - hash); + if (common.has_infoz()) { + return ReportInsertMissToInfozAndGetSlot(common, policy, hash, + target_group_offset); } return res; }