feat(runtime): add semantics command queue api (#12198) bd587cb83f * wip: semantic system * feat: semantic state (#11888) * wip: semantic state * docs: semantic steps * test: initial state tests * chore: live region sample and set on inferred text component * fix: component list propegation * fix: visiblity changes exclude semantics and list examples * chore: update keys * refactor: remove semantic cursor and query artboard directly * refactor(flutter): semantic painter mixin * chore: remove redundant semanticsChanged call * chore: clangy clank * docs: cleanup * refactor: atomic node id * refactor: remove node id type alias * feat: focus and semantic handshake * refactor: opt in semantics for flutter * chore: cleanup * fix: tmp step around failing editor tests * chore: clang clank * refactor: cleanup public API * feat: add focusable as a state * chore: TEMP REMOVE TESTS RESTORE LATER * feat: semantic inference from subtree * refactor: expanded definition and state changes * feat: add support for passthrough bit flags (#11992) * add support for passthrough bit flags (cherry picked from commit de4a5504a448df8f051fadae2abf154ea1fad9fd) * chore: post cherry pick rerun code generator * feat: expose more passthrough properties * feat: add all passthrough properties as editor options and labels * chore: update keys after merge * chore: resolve merge issues and other post merge updates * chore: updated example with new core props * chore: remove expanded from state * feat: phase2 semantic roles and states * chore: clang clank * feat: add semantic actions * refactor: remove numericX * refactor: remove heading and progressIndicator from Roles * fix: missing potential semantic configurations * feat: add semantic traits for accessibility tristate support (#12148) * feat: add semantic traits * fix: semantic action dispatch for listeners targeting nested shapes * fix: core generator call onPropertyChanged to update editor state * refactor: editor UI for trait selection * chore: updated tests and examples * chore: clang clank * chore: updated tests * fix: pass correct from/to mask values in bitmask passthrough notifications * feat: add semantic artboard boundary (#12180) * feat: add semantic artboard boundary nodes for targeted collapse and bounds dirtying * fix: copy vector before iterating * feat: make semantics runtime opt-in * refactor: move around the entry point for semantics * fix: flutter check attach, correct comments, expanded test * fix(flutter): radio button isButton true (untested) * chore: additional null checks * chore: format def file * chore: uint32 for traits and states diff * feat: add semantics command queue api * fix(semantics): order nodes by visual position instead of static graph order (#12200) * fix(semantics): order nodes by visual position instead of static graph order * fix: use isBoundaryNode() instead of semanticData() for container bounds check * perf(semantics): specialize updatedGeometry as POD bounds struct (#12205) * refactor: dedicated SemanticBoundsUpdate struct * perf(semantics): tighten FFI diff readers and lock WASM layouts * chore: update code comment * chore(semantics): test and sample cleanup * chore(flutter): other cleanup and analyzer fixes * fix(semantics): emit diff arrays in deterministic tree pre-order * fix(semantics): scope semantic node id allocation per-manager * fix(semantics): static_assert FFI struct sizes on 64-bit native * test(semantic): SemanticTreeModel.applyDiff and the incremental refresh() paths * docs(semantics): document Mixed-wins precedence for Checked state * refactor(semantics): derive trait/state bit constants from JSON * refactor(semantics): tighten SemanticNode encapsulation * fix(semantics): (refactor) route fireSemanticAction through the manager index * refactor(semantics): bind semantic listeners strictly to target's SD * test: update .riv files to have listener on SD * perf(semantics): drop std::function closure in label derivation * perf(semantics): fuse post-order bounds walk with reorder check * chore(semantics): simplify and tighten the /simplify review findings Aggregated review cleanups from three parallel audits of this session's work. No behaviour changes outside the applyDiff no-op guard (which tightens defense-in-depth without changing semantics for correctly-formed diffs). Reuse - SemanticManager::reconcileBoundsForSubtree: replace field-by-field bounds compare with AABB::operator!=. - Delete unused C++ effectiveChecked/effectiveMixed helpers from semantic_state.hpp; rewrite the Mixed-precedence docstring as plain prose. Dart helpers stay (used by the Flutter overlay). Efficiency - SemanticTreeModel.applyDiff: thread a `changed` flag through all six mutation loops so no-op diffs (identical payload) no longer bump version or notify listeners. New unit test pins the behaviour across updatedSemantic, updatedGeometry, and childrenUpdated. - SemanticManager::refresh() incremental path: when containerBoundsChangedIds is empty (common case), skip building the boundsDirtyUnion hashset and check m_dirtyBoundsNodes directly. - deriveLabelVisit: introduce LabelDerivationScratch struct owned at the top-level call and clear()-ed between interactive nodes, so per-node set and string allocations are reused across the walk. Comment hygiene - Strip narration / session / what-describing comments across semantic_manager.cpp, semantic_data.cpp, state_machine_instance.cpp, rive_semantics_overlay.dart, and rive_binding.cpp. - Trim the speculative 32-bit Dart FFI sizing note and the redundant inline "see precedence note below" breadcrumb on SemanticState. * chore(semantics): apply /simplify review findings across runtime and FFI Per-frame wins: - Return nullptr from stateMachineSemanticsTakeDiff when the diff is empty so idle frames skip the FFI struct allocation (Dart/WASM already handle null). - Skip markNodeDirty(Bounds) in updateWorldBounds when the computed AABB matches the cached value. - Short-circuit applyInferredSemanticsIfNeeded when the inferred role+label are unchanged. - Guard every *Changed() handler (role/label/stateFlags/traitFlags/value/hint/headingLevel) so same-value writes don't dirty the node. - Drop ComponentDirt::Paint from the visibility re-sync; opacity does not affect tree membership (per shouldExcludeFromSemanticTree). - toSemanticNodeFFI emits nullptr for empty value/hint instead of a 1-byte "\0" buffer each. Reuse: - SemanticDirt now uses RIVE_MAKE_ENUM_BITSET; hand-rolled |, &, and hasSemanticDirt removed. - Replace hand-rolled "find first child of T" loops with ContainerComponent::firstChild<T>() (semantic_data.cpp, semantic_manager.cpp, focus_data.cpp, semantic_provider.cpp). - Replace two hasChildBounds two-branch merges with AABB::expand(). - Dart _mapToWidgetRect uses AABB.transform() (four-corner transform) — also a correctness fix under rotation/skew. - Dart _listEquals -> foundation.listEquas. Cleanup: - Delete dead m_dirtyStructureNodes field (written, never read). - Fix SemanticsChildrenUpdateFFI.childCount from @Int32 to @Uint32 to match the C++ uint32_t. - Trim the inference registry: drop redundant null-checks on table entries, drop per-rule type re-check inside the infer fn, drop the POD struct constructor. Registry shape preserved for upcoming rules. - Remove a commented-out dropdown_list demo line. * chore: clang clank * chore: address copilot review comments * chore(semantics): update core definitions after merge * chore: update to new enum util functions after merge * chore: updated .riv files after merge new prop keys * chore: clang clank * chore: remove accessibility_tools package * test(semantic): expanded c++ test suite * test: simpsons list filtering test * test(semantic): fixed list scroll focus * chore: clang clank * fix: clang * chore: test cleanup * chore: cleanup before merge * wip: tmp integration guides * fix(semantics): tear down old painter when RiveSemanticsWidget swaps painter/artboard * fix(semantics): gate Text inference registry on WITH_RIVE_TEXT * docs(semantics): clarify that semanticBounds walks the full subtree * feat(rive_native): expose frame/tree debugger * perf(rive_native): hoist artboard→widget transform onto a single SemanticsNode * refactor(flutter): rename public API to 'drainSemanticsDiff' and 'focusSemanticNode' * refactor!(semantics): rename 'takeDiff' to 'drainDiff' * chore(editor): use uistrings for semantics * feat: Add drainSemanticsDiff operation to command queue (#12247) * fix: Use new drainDiff function * chore(flutter): rename ffi bindings to match C++ names exactly * feat(editor): show warning on role change * chore(semantics): change warning icon * chore: cleanup * chore: readd lost files after merge * chore: clang formating * docs: update guides on latest changes * fix(command-queue): consistent semantic error handling + harden drain * fix(command-queue): lock per-state-machine mutex for semantic commands * perf(command-queue): move semantics diff to global listener when sole recipient * fix(command-queue): map semantics bounds with the real transform, not bounds-equality) * chore: per-state-machine lock now covers only what needs the instance and comment cleanup * chore: more comment cleanup * chore: move docs to semantics subfolder Co-authored-by: David Skuza <david@rive.app> Co-authored-by: Erik <erik@rive.app> Co-authored-by: Gordon <pggordonhayes@gmail.com> Co-authored-by: Hernan Torrisi <hernantorrisi@gmail.com>
diff --git a/.rive_head b/.rive_head index 71e5f7a..1819944 100644 --- a/.rive_head +++ b/.rive_head
@@ -1 +1 @@ -88115b09496d99f10462da6e4ececcbd87988090 +bd587cb83fb48eb8ab67c9bf23eb9ecf184f7125
diff --git a/include/rive/command_queue.hpp b/include/rive/command_queue.hpp index 4a3ebb7..8208fac 100644 --- a/include/rive/command_queue.hpp +++ b/include/rive/command_queue.hpp
@@ -8,6 +8,8 @@ #include "rive/refcnt.hpp" #include "rive/math/vec2d.hpp" #include "rive/viewmodel/runtime/viewmodel_runtime.hpp" +#include "rive/animation/semantic_listener_group.hpp" +#include "rive/semantic/semantic_snapshot.hpp" #include <condition_variable> #include <cstdint> @@ -334,6 +336,16 @@ virtual void onStateMachineSettled(const StateMachineHandle, uint64_t requestId) {} + + // Delivered when an incremental semantic diff is available for this + // state machine. Emitted only when drainSemanticsDiff produces a + // non-empty diff. Bounds inside the diff are reported in view space + // using the fit/alignment/scale/view-bounds arguments provided to the + // drain command. + virtual void onSemanticsDiffReceived(const StateMachineHandle, + uint64_t requestId, + SemanticsDiff diff) + {} }; CommandQueue(); @@ -563,6 +575,22 @@ float timeToAdvance, uint64_t requestId = 0); + // Enable the semantic subsystem on the given state machine. Must be + // called before diffs are delivered. Safe to call multiple times. + void enableSemantics(StateMachineHandle, uint64_t requestId = 0); + + // Drain the current semantic diff for the given state machine. + // The response is delivered via + // StateMachineListener::onSemanticsDiffReceived when the diff is non-empty. + // Output bounds are mapped to view space using the provided fit/alignment/ + // scale/view-bounds parameters. + void drainSemanticsDiff(StateMachineHandle, + Fit fit, + Alignment alignment, + float scaleFactor, + Vec2D viewBounds, + uint64_t requestId = 0); + // Pointer events struct PointerEvent { @@ -584,6 +612,21 @@ void deleteStateMachine(StateMachineHandle, uint64_t requestId = 0); + // Fire a semantic action (tap / increase / decrease) on the given node. + // Fire-and-forget; reports a stateMachineError if the handle is unknown or + // semantics isn't enabled. An unknown node id is a silent no-op. + void fireSemanticAction(StateMachineHandle, + uint32_t semanticNodeId, + SemanticActionType actionType, + uint64_t requestId = 0); + + // Request focus on the given semantic node. Fire-and-forget; reports a + // stateMachineError if the handle is unknown or semantics isn't enabled. + // An unknown or non-focusable node id is a silent no-op. + void requestSemanticFocus(StateMachineHandle, + uint32_t semanticNodeId, + uint64_t requestId = 0); + RenderImageHandle decodeImage(std::vector<uint8_t> imageEncodedBytes, RenderImageListener* listener = nullptr, uint64_t requestId = 0); @@ -865,6 +908,10 @@ instantiateStateMachine, deleteStateMachine, advanceStateMachine, + enableSemantics, + drainSemanticsDiff, + fireSemanticAction, + requestSemanticFocus, bindViewModelInstance, runOnce, draw, @@ -925,6 +972,7 @@ viewModelDeleted, stateMachineDeleted, stateMachineSettled, + semanticsDiffReceived, fileAssetsListed, fileError, artboardError, @@ -964,6 +1012,7 @@ std::mutex m_messageMutex; PODStream m_messageStream; ObjectStream<std::string> m_messageNames; + ObjectStream<SemanticsDiff> m_messageSemanticsDiffs; // Listeners FileListener* m_globalFileListener = nullptr;
diff --git a/include/rive/semantic/semantic_snapshot.hpp b/include/rive/semantic/semantic_snapshot.hpp index c90534e..0db95ca 100644 --- a/include/rive/semantic/semantic_snapshot.hpp +++ b/include/rive/semantic/semantic_snapshot.hpp
@@ -1,6 +1,8 @@ #ifndef _RIVE_SEMANTIC_SNAPSHOT_HPP_ #define _RIVE_SEMANTIC_SNAPSHOT_HPP_ +#include "rive/math/aabb.hpp" + #include <cstdint> #include <string> #include <vector> @@ -28,6 +30,16 @@ int32_t parentId = -1; // Sibling position under parent (or root index if parent is -1). uint32_t siblingIndex = 0; + + AABB bounds() const { return AABB(minX, minY, maxX, maxY); } + + void setBounds(const AABB& value) + { + minX = value.minX; + minY = value.minY; + maxX = value.maxX; + maxY = value.maxY; + } }; // Parent children list update for robust reorder/reparent sync. @@ -49,6 +61,16 @@ float minY = 0.0f; float maxX = 0.0f; float maxY = 0.0f; + + AABB bounds() const { return AABB(minX, minY, maxX, maxY); } + + void setBounds(const AABB& value) + { + minX = value.minX; + minY = value.minY; + maxX = value.maxX; + maxY = value.maxY; + } }; // Incremental delta payload emitted by SemanticManager::refresh().
diff --git a/src/command_queue.cpp b/src/command_queue.cpp index cbf1bac..54f4220 100644 --- a/src/command_queue.cpp +++ b/src/command_queue.cpp
@@ -669,6 +669,57 @@ m_commandStream << requestId; } +void CommandQueue::enableSemantics(StateMachineHandle stateMachineHandle, + uint64_t requestId) +{ + AutoLockAndNotify lock(m_commandMutex, m_commandConditionVariable); + m_commandStream << Command::enableSemantics; + m_commandStream << stateMachineHandle; + m_commandStream << requestId; +} + +void CommandQueue::drainSemanticsDiff(StateMachineHandle stateMachineHandle, + Fit fit, + Alignment alignment, + float scaleFactor, + Vec2D viewBounds, + uint64_t requestId) +{ + AutoLockAndNotify lock(m_commandMutex, m_commandConditionVariable); + m_commandStream << Command::drainSemanticsDiff; + m_commandStream << stateMachineHandle; + m_commandStream << requestId; + m_commandStream << fit; + m_commandStream << alignment.x(); + m_commandStream << alignment.y(); + m_commandStream << scaleFactor; + m_commandStream << viewBounds; +} + +void CommandQueue::fireSemanticAction(StateMachineHandle stateMachineHandle, + uint32_t semanticNodeId, + SemanticActionType actionType, + uint64_t requestId) +{ + AutoLockAndNotify lock(m_commandMutex, m_commandConditionVariable); + m_commandStream << Command::fireSemanticAction; + m_commandStream << stateMachineHandle; + m_commandStream << requestId; + m_commandStream << semanticNodeId; + m_commandStream << actionType; +} + +void CommandQueue::requestSemanticFocus(StateMachineHandle stateMachineHandle, + uint32_t semanticNodeId, + uint64_t requestId) +{ + AutoLockAndNotify lock(m_commandMutex, m_commandConditionVariable); + m_commandStream << Command::requestSemanticFocus; + m_commandStream << stateMachineHandle; + m_commandStream << requestId; + m_commandStream << semanticNodeId; +} + RenderImageHandle CommandQueue::decodeImage( std::vector<uint8_t> imageEncodedBytes, RenderImageListener* listener, @@ -1858,7 +1909,45 @@ } break; } - + case Message::semanticsDiffReceived: + { + StateMachineHandle handle; + uint64_t requestId; + m_messageStream >> handle; + m_messageStream >> requestId; + SemanticsDiff diff; + m_messageSemanticsDiffs >> diff; + lock.unlock(); + auto itr = m_stateMachineListeners.find(handle); + const bool hasPerHandleListener = + itr != m_stateMachineListeners.end(); + if (m_globalStateMachineListener) + { + // Copy to the global listener only when a per-handle + // listener also needs the diff; otherwise move. + if (hasPerHandleListener) + { + m_globalStateMachineListener->onSemanticsDiffReceived( + handle, + requestId, + diff); + } + else + { + m_globalStateMachineListener->onSemanticsDiffReceived( + handle, + requestId, + std::move(diff)); + } + } + if (hasPerHandleListener) + { + itr->second->onSemanticsDiffReceived(handle, + requestId, + std::move(diff)); + } + break; + } case Message::fileError: { FileHandle handle;
diff --git a/src/command_server.cpp b/src/command_server.cpp index 7ee5189..c272c51 100644 --- a/src/command_server.cpp +++ b/src/command_server.cpp
@@ -10,6 +10,7 @@ #include "rive/assets/image_asset.hpp" #include "rive/assets/script_asset.hpp" #include "rive/file.hpp" +#include "rive/semantic/semantic_manager.hpp" #include "rive/viewmodel/runtime/viewmodel_runtime.hpp" #include "rive/lua/rive_lua_libs.hpp" @@ -468,6 +469,38 @@ return inverse * pointerEvent.position; } +// Maps a semantics bounds payload from artboard space to view space. +// Templated so the same transform path can be reused for both +// SemanticsDiffNode and SemanticsBoundsUpdate via bounds()/setBounds(). +template <typename T> +static void mapSemanticsBounds(T& value, const Mat2D& transform) +{ + value.setBounds(transform.mapBoundingBox(value.bounds())); +} + +// Applies artboard->view bounds mapping to all geometry-bearing collections +// in a semantic diff before it is delivered to listeners. +static void mapSemanticsDiffToViewSpace(SemanticsDiff& diff, + const Mat2D& transform) +{ + for (auto& node : diff.added) + { + mapSemanticsBounds(node, transform); + } + for (auto& node : diff.moved) + { + mapSemanticsBounds(node, transform); + } + for (auto& node : diff.updatedSemantic) + { + mapSemanticsBounds(node, transform); + } + for (auto& bounds : diff.updatedGeometry) + { + mapSemanticsBounds(bounds, transform); + } +} + void CommandServer::checkPropertySubscriptions() { for (auto& subscription : m_propertySubscriptions) @@ -1869,6 +1902,217 @@ break; } + case CommandQueue::Command::enableSemantics: + { + StateMachineHandle handle; + uint64_t requestId; + commandStream >> handle; + commandStream >> requestId; + lock.unlock(); + if (auto wrapper = getStateMachineWrapper(handle)) + { + std::unique_lock<std::mutex> stateMachineLock( + wrapper->m_mutex); + wrapper->instance->enableSemantics(); + } + else + { + ErrorReporter<StateMachineHandle>( + this, + handle, + requestId, + CommandQueue::Message::stateMachineError) + << "State machine " << handle + << " not found for enableSemantics."; + } + break; + } + + case CommandQueue::Command::drainSemanticsDiff: + { + StateMachineHandle handle; + uint64_t requestId; + Fit fit; + float alignmentX; + float alignmentY; + float scaleFactor; + Vec2D viewBounds; + commandStream >> handle; + commandStream >> requestId; + commandStream >> fit; + commandStream >> alignmentX; + commandStream >> alignmentY; + commandStream >> scaleFactor; + commandStream >> viewBounds; + lock.unlock(); + if (auto wrapper = getStateMachineWrapper(handle)) + { + std::unique_lock<std::mutex> stateMachineLock( + wrapper->m_mutex); + auto* manager = wrapper->instance->semanticManager(); + if (manager == nullptr) + { + stateMachineLock.unlock(); + ErrorReporter<StateMachineHandle>( + this, + handle, + requestId, + CommandQueue::Message::stateMachineError) + << "Semantics not enabled on state machine " + << handle + << "; call enableSemantics before " + "drainSemanticsDiff."; + } + else + { + SemanticsDiff diff = manager->drainDiff(); + if (!diff.empty()) + { + // Compute the transform while locked (it reads the + // artboard); identity means no mapping. + Mat2D transform; + if (auto* artboard = wrapper->instance->artboard()) + { + AABB artboardBounds = artboard->bounds(); + AABB surfaceBounds = + AABB(Vec2D{0.0f, 0.0f}, viewBounds); + if (surfaceBounds.width() != 0.0f && + surfaceBounds.height() != 0.0f) + { + Alignment alignment(alignmentX, alignmentY); + transform = + rive::computeAlignment(fit, + alignment, + surfaceBounds, + artboardBounds, + scaleFactor); + } + } + + // Map the owned diff outside the lock. + stateMachineLock.unlock(); + if (transform != Mat2D()) + { + mapSemanticsDiffToViewSpace(diff, transform); + } + + std::unique_lock<std::mutex> messageLock( + m_commandQueue->m_messageMutex); + messageStream + << CommandQueue::Message::semanticsDiffReceived; + messageStream << handle; + messageStream << requestId; + m_commandQueue->m_messageSemanticsDiffs + << std::move(diff); + } + } + } + else + { + ErrorReporter<StateMachineHandle>( + this, + handle, + requestId, + CommandQueue::Message::stateMachineError) + << "State machine " << handle + << " not found for drainSemanticsDiff."; + } + break; + } + + case CommandQueue::Command::fireSemanticAction: + { + StateMachineHandle handle; + uint64_t requestId; + uint32_t semanticNodeId; + SemanticActionType actionType; + commandStream >> handle; + commandStream >> requestId; + commandStream >> semanticNodeId; + commandStream >> actionType; + lock.unlock(); + if (auto wrapper = getStateMachineWrapper(handle)) + { + std::unique_lock<std::mutex> stateMachineLock( + wrapper->m_mutex); + if (wrapper->instance->semanticManager() == nullptr) + { + stateMachineLock.unlock(); + ErrorReporter<StateMachineHandle>( + this, + handle, + requestId, + CommandQueue::Message::stateMachineError) + << "Semantics not enabled on state machine " + << handle + << "; call enableSemantics before " + "fireSemanticAction."; + } + else + { + wrapper->instance->fireSemanticAction(semanticNodeId, + actionType); + } + } + else + { + ErrorReporter<StateMachineHandle>( + this, + handle, + requestId, + CommandQueue::Message::stateMachineError) + << "State machine " << handle + << " not found for fireSemanticAction."; + } + break; + } + + case CommandQueue::Command::requestSemanticFocus: + { + StateMachineHandle handle; + uint64_t requestId; + uint32_t semanticNodeId; + commandStream >> handle; + commandStream >> requestId; + commandStream >> semanticNodeId; + lock.unlock(); + if (auto wrapper = getStateMachineWrapper(handle)) + { + std::unique_lock<std::mutex> stateMachineLock( + wrapper->m_mutex); + auto* manager = wrapper->instance->semanticManager(); + if (manager == nullptr) + { + stateMachineLock.unlock(); + ErrorReporter<StateMachineHandle>( + this, + handle, + requestId, + CommandQueue::Message::stateMachineError) + << "Semantics not enabled on state machine " + << handle + << "; call enableSemantics before " + "requestSemanticFocus."; + } + else + { + // A missing/non-focusable node is a silent no-op. + manager->requestFocus(semanticNodeId); + } + } + else + { + ErrorReporter<StateMachineHandle>( + this, + handle, + requestId, + CommandQueue::Message::stateMachineError) + << "State machine " << handle + << " not found for requestSemanticFocus."; + } + break; + } + case CommandQueue::Command::runOnce: { CommandServerCallback callback;
diff --git a/tests/unit_tests/runtime/command_queue_test.cpp b/tests/unit_tests/runtime/command_queue_test.cpp index 6efb0d9..b5e4e15 100644 --- a/tests/unit_tests/runtime/command_queue_test.cpp +++ b/tests/unit_tests/runtime/command_queue_test.cpp
@@ -7,6 +7,8 @@ #include "rive/command_queue.hpp" #include "rive/command_server.hpp" #include "rive/file.hpp" +#include "rive/semantic/semantic_role.hpp" +#include "rive/semantic/semantic_state.hpp" #include "common/render_context_null.hpp" #include <fstream> @@ -5524,3 +5526,545 @@ commandQueue->disconnect(); serverThread.join(); } + +// ============================================================ +// Semantics +// ============================================================ + +namespace +{ +// Flatten a diff's added+moved+updatedSemantic nodes into a single list, +// useful for locating a specific node by role in test assertions. +struct SemanticTestModel +{ + std::unordered_map<uint32_t, SemanticsDiffNode> nodes; + + void apply(const SemanticsDiff& diff) + { + for (auto id : diff.removed) + nodes.erase(id); + for (auto& n : diff.added) + nodes[n.id] = n; + for (auto& n : diff.moved) + nodes[n.id] = n; + for (auto& n : diff.updatedSemantic) + { + auto it = nodes.find(n.id); + if (it != nodes.end()) + { + // updatedSemantic carries full content, preserve existing + // geometry if present. + float minX = it->second.minX, minY = it->second.minY; + float maxX = it->second.maxX, maxY = it->second.maxY; + it->second = n; + it->second.minX = minX; + it->second.minY = minY; + it->second.maxX = maxX; + it->second.maxY = maxY; + } + else + { + nodes[n.id] = n; + } + } + for (auto& n : diff.updatedGeometry) + { + auto it = nodes.find(n.id); + if (it != nodes.end()) + { + it->second.minX = n.minX; + it->second.minY = n.minY; + it->second.maxX = n.maxX; + it->second.maxY = n.maxY; + } + } + } +}; + +class SemanticTestListener : public CommandQueue::StateMachineListener +{ +public: + void onStateMachineSettled(const StateMachineHandle, uint64_t) override + { + m_settleCount++; + } + + void onStateMachineError(const StateMachineHandle, + uint64_t requestId, + std::string error) override + { + m_errorCount++; + m_lastErrorRequestId = requestId; + m_lastError = std::move(error); + } + + void onSemanticsDiffReceived(const StateMachineHandle handle, + uint64_t requestId, + SemanticsDiff diff) override + { + CHECK(handle == m_handle); + m_diffCount++; + m_lastDiffRequestId = requestId; + m_model.apply(diff); + m_lastDiff = std::move(diff); + } + + StateMachineHandle m_handle = RIVE_NULL_HANDLE; + int m_settleCount = 0; + int m_diffCount = 0; + int m_errorCount = 0; + uint64_t m_lastDiffRequestId = 0; + uint64_t m_lastErrorRequestId = 0; + std::string m_lastError; + SemanticsDiff m_lastDiff; + SemanticTestModel m_model; +}; + +// Helper that spins up server, loads semantic/simpsons.riv, creates artboard + +// default view model instance + state machine, and returns the handles. +// Caller owns the commandQueue and server lifetime. +struct SemanticFixture +{ + rcp<CommandQueue> commandQueue; + std::unique_ptr<gpu::RenderContext> nullContext; + std::unique_ptr<CommandServer> server; + FileHandle fileHandle = RIVE_NULL_HANDLE; + ArtboardHandle artboardHandle = RIVE_NULL_HANDLE; + ViewModelInstanceHandle viewModelHandle = RIVE_NULL_HANDLE; + StateMachineHandle stateMachineHandle = RIVE_NULL_HANDLE; + + SemanticFixture(SemanticTestListener* listener = nullptr) + { + commandQueue = make_rcp<CommandQueue>(); + nullContext = RenderContextNULL::MakeContext(); + server = + std::make_unique<CommandServer>(commandQueue, nullContext.get()); + + std::ifstream stream("assets/semantic/simpsons.riv", std::ios::binary); + fileHandle = commandQueue->loadFile( + std::vector<uint8_t>(std::istreambuf_iterator<char>(stream), {})); + artboardHandle = commandQueue->instantiateDefaultArtboard(fileHandle); + viewModelHandle = + commandQueue->instantiateDefaultViewModelInstance(fileHandle, + artboardHandle); + stateMachineHandle = + commandQueue->instantiateDefaultStateMachine(artboardHandle, + listener); + commandQueue->bindViewModelInstance(stateMachineHandle, + viewModelHandle); + if (listener) + listener->m_handle = stateMachineHandle; + } + + // Advance N frames to let the state machine settle into a steady state, + // mirroring the warmup that semantic_simpsons_test.dart does before + // reading its initial tree. + void warmup(int frames = 10, float dt = 0.1f) + { + for (int i = 0; i < frames; ++i) + { + commandQueue->advanceStateMachine(stateMachineHandle, dt); + } + } + + void drain(uint64_t requestId = 0, + Fit fit = Fit::contain, + Alignment alignment = Alignment::center, + float scaleFactor = 1.0f, + Vec2D viewBounds = Vec2D(500.0f, 500.0f)) + { + commandQueue->drainSemanticsDiff(stateMachineHandle, + fit, + alignment, + scaleFactor, + viewBounds, + requestId); + } + + void pump() + { + server->processCommands(); + commandQueue->processMessages(); + } +}; +} // namespace + +TEST_CASE("Semantics advance does not auto-deliver diff", "[CommandQueue]") +{ + SemanticTestListener listener; + SemanticFixture fx(&listener); + + fx.commandQueue->enableSemantics(fx.stateMachineHandle); + fx.warmup(); + fx.pump(); + + CHECK(listener.m_diffCount == 0); + CHECK(listener.m_errorCount == 0); +} + +TEST_CASE("Semantics enable + initial diff on drainSemanticsDiff", + "[CommandQueue]") +{ + SemanticTestListener listener; + SemanticFixture fx(&listener); + + fx.commandQueue->enableSemantics(fx.stateMachineHandle); + fx.warmup(); + fx.drain(); + fx.pump(); + + CHECK(listener.m_diffCount >= 1); + CHECK(listener.m_errorCount == 0); + CHECK(!listener.m_model.nodes.empty()); + + // The fixture should include a tabList (simpsons.riv layout). + bool foundTabList = false; + for (auto& kv : listener.m_model.nodes) + { + if (kv.second.role == static_cast<uint32_t>(SemanticRole::tabList)) + { + foundTabList = true; + break; + } + } + CHECK(foundTabList); +} + +TEST_CASE("Semantics no diff when not enabled", "[CommandQueue]") +{ + SemanticTestListener listener; + SemanticFixture fx(&listener); + + fx.warmup(); + fx.pump(); + + CHECK(listener.m_diffCount == 0); + CHECK(listener.m_errorCount == 0); +} + +TEST_CASE("Semantics drainSemanticsDiff errors when not enabled", + "[CommandQueue]") +{ + SemanticTestListener listener; + SemanticFixture fx(&listener); + + const uint64_t reqId = 0x1234; + fx.drain(reqId); + fx.pump(); + + CHECK(listener.m_diffCount == 0); + CHECK(listener.m_errorCount == 1); + CHECK(listener.m_lastErrorRequestId == reqId); +} + +TEST_CASE("Semantics drainSemanticsDiff only emits for non-empty diff", + "[CommandQueue]") +{ + SemanticTestListener listener; + SemanticFixture fx(&listener); + + fx.commandQueue->enableSemantics(fx.stateMachineHandle); + fx.warmup(); + const uint64_t reqId = 0xABCD; + fx.drain(reqId); + fx.pump(); + + CHECK(listener.m_diffCount == 1); + CHECK(listener.m_lastDiffRequestId == reqId); + CHECK(listener.m_errorCount == 0); + + // No changes since the previous drain: no callback expected. + fx.drain(0xBCDE); + fx.pump(); + + CHECK(listener.m_diffCount == 1); + CHECK(listener.m_lastDiffRequestId == reqId); + CHECK(listener.m_errorCount == 0); +} + +TEST_CASE("Semantics fireSemanticAction tap changes selected tab", + "[CommandQueue]") +{ + SemanticTestListener listener; + SemanticFixture fx(&listener); + + fx.commandQueue->enableSemantics(fx.stateMachineHandle); + fx.warmup(); + fx.drain(); + fx.pump(); + + // Find tabs and the one that's currently selected. + uint32_t selectedTabId = 0; + uint32_t otherTabId = 0; + for (auto& kv : listener.m_model.nodes) + { + const auto& node = kv.second; + if (node.role != static_cast<uint32_t>(SemanticRole::tab)) + continue; + if (hasSemanticState(node.stateFlags, SemanticState::Selected)) + selectedTabId = node.id; + else if (otherTabId == 0) + otherTabId = node.id; + } + REQUIRE(selectedTabId != 0); + REQUIRE(otherTabId != 0); + + // Fire tap on the other tab, then advance to let the state machine + // process the deferred action and update the data-bound selection. + fx.commandQueue->fireSemanticAction(fx.stateMachineHandle, + otherTabId, + SemanticActionType::tap); + fx.warmup(); + fx.drain(); + fx.pump(); + + auto itOther = listener.m_model.nodes.find(otherTabId); + auto itInitial = listener.m_model.nodes.find(selectedTabId); + REQUIRE(itOther != listener.m_model.nodes.end()); + REQUIRE(itInitial != listener.m_model.nodes.end()); + CHECK( + hasSemanticState(itOther->second.stateFlags, SemanticState::Selected)); + CHECK(!hasSemanticState(itInitial->second.stateFlags, + SemanticState::Selected)); + CHECK(listener.m_errorCount == 0); +} + +TEST_CASE("Semantics commands on invalid state machine handle", + "[CommandQueue]") +{ + SemanticTestListener bogusListener; + SemanticFixture fx; // no main listener — we only watch the bogus one. + + // A state machine handle whose instantiation fails (name not found). The + // listener is bound to the handle, so per-handle state-machine errors route + // to it; the instantiation failure goes to the artboard channel instead. + auto bogusHandle = fx.commandQueue->instantiateStateMachineNamed( + fx.artboardHandle, + "this state machine does not exist", + &bogusListener); + bogusListener.m_handle = bogusHandle; + + fx.commandQueue->enableSemantics(bogusHandle, 0xE1); + fx.commandQueue->drainSemanticsDiff(bogusHandle, + Fit::contain, + Alignment::center, + 1.0f, + Vec2D(500.0f, 500.0f), + 0xE2); + fx.commandQueue->fireSemanticAction(bogusHandle, + 42, + SemanticActionType::tap, + 0xE3); + fx.commandQueue->requestSemanticFocus(bogusHandle, 42, 0xE4); + + fx.pump(); + + // One stateMachineError per semantic command; the instantiation failure is + // an artboardError, so it does not reach this listener. + CHECK(bogusListener.m_errorCount == 4); + CHECK(bogusListener.m_diffCount == 0); +} + +TEST_CASE("Semantics drainSemanticsDiff maps bounds into view space", + "[CommandQueue]") +{ + // This validates the view-space mapping math by running the same .riv + // through drainSemanticsDiff twice (small and large views), then comparing + // width/height ratios for the same semantic node id. + auto captureTabsById = [](Vec2D viewBounds) + -> std::unordered_map<uint32_t, SemanticsDiffNode> { + SemanticTestListener listener; + SemanticFixture fx(&listener); + fx.commandQueue->enableSemantics(fx.stateMachineHandle); + fx.warmup(); + fx.drain(0, Fit::contain, Alignment::center, 1.0f, viewBounds); + fx.pump(); + + REQUIRE(listener.m_diffCount >= 1); + + std::unordered_map<uint32_t, SemanticsDiffNode> tabsById; + for (auto& kv : listener.m_model.nodes) + { + const auto& node = kv.second; + if (node.role != static_cast<uint32_t>(SemanticRole::tab)) + { + continue; + } + AABB nodeBounds = node.bounds(); + if (nodeBounds.width() > 0.0f && nodeBounds.height() > 0.0f) + { + tabsById[node.id] = node; + } + } + return tabsById; + }; + + Vec2D smallViewBounds(200.0f, 200.0f); + Vec2D largeViewBounds(800.0f, 800.0f); + + auto smallTabs = captureTabsById(smallViewBounds); + auto largeTabs = captureTabsById(largeViewBounds); + + REQUIRE(!smallTabs.empty()); + REQUIRE(!largeTabs.empty()); + + bool foundSharedTabId = false; + uint32_t sharedTabId = 0; + for (const auto& kv : smallTabs) + { + if (largeTabs.find(kv.first) != largeTabs.end()) + { + foundSharedTabId = true; + sharedTabId = kv.first; + break; + } + } + REQUIRE(foundSharedTabId); + + const auto& smallTab = smallTabs.at(sharedTabId); + const auto& largeTab = largeTabs.at(sharedTabId); + + float smallWidth = smallTab.bounds().width(); + float smallHeight = smallTab.bounds().height(); + float largeWidth = largeTab.bounds().width(); + float largeHeight = largeTab.bounds().height(); + + float expectedScale = largeViewBounds.x / smallViewBounds.x; + + CHECK(largeWidth > smallWidth); + CHECK(largeHeight > smallHeight); + CHECK(largeWidth / smallWidth == Approx(expectedScale).epsilon(0.01)); + CHECK(largeHeight / smallHeight == Approx(expectedScale).epsilon(0.01)); +} + +TEST_CASE("Semantics requestSemanticFocus errors when not enabled", + "[CommandQueue]") +{ + SemanticTestListener listener; + SemanticFixture fx(&listener); + + // No enableSemantics: the command must surface a stateMachineError rather + // than silently no-op (consistent with drainSemanticsDiff). + const uint64_t reqId = 0x5151; + fx.commandQueue->requestSemanticFocus(fx.stateMachineHandle, 1, reqId); + fx.pump(); + + CHECK(listener.m_errorCount == 1); + CHECK(listener.m_lastErrorRequestId == reqId); + CHECK(listener.m_diffCount == 0); +} + +TEST_CASE("Semantics fireSemanticAction errors when not enabled", + "[CommandQueue]") +{ + SemanticTestListener listener; + SemanticFixture fx(&listener); + + // No enableSemantics: the command must surface a stateMachineError rather + // than silently no-op (consistent with drainSemanticsDiff). + const uint64_t reqId = 0x5252; + fx.commandQueue->fireSemanticAction(fx.stateMachineHandle, + 1, + SemanticActionType::tap, + reqId); + fx.pump(); + + CHECK(listener.m_errorCount == 1); + CHECK(listener.m_lastErrorRequestId == reqId); + CHECK(listener.m_diffCount == 0); +} + +TEST_CASE("Semantics requestSemanticFocus on a valid node routes without error", + "[CommandQueue]") +{ + SemanticTestListener listener; + SemanticFixture fx(&listener); + + fx.commandQueue->enableSemantics(fx.stateMachineHandle); + fx.warmup(); + fx.drain(); + fx.pump(); + + REQUIRE(!listener.m_model.nodes.empty()); + + // Route a focus request for a real node to an enabled state machine. + // (FocusNode-level focus behavior is covered by focus_test.cpp.) + const uint32_t targetId = listener.m_model.nodes.begin()->first; + fx.commandQueue->requestSemanticFocus(fx.stateMachineHandle, targetId); + fx.warmup(); + fx.drain(); + fx.pump(); + + // Routing to an enabled state machine must never surface an error, + // regardless of whether the node ultimately accepts focus. + CHECK(listener.m_errorCount == 0); +} + +TEST_CASE("Semantics drainSemanticsDiff honors scaleFactor when the view " + "matches the artboard", + "[CommandQueue]") +{ + // Fit::layout scales by scaleFactor, so bounds must be mapped into view + // space even when the view matches the artboard. + + // Discover the artboard bounds so the view can be set equal to them. + AABB artboardBounds(0.0f, 0.0f, 0.0f, 0.0f); + { + SemanticFixture probe; + probe.commandQueue->runOnce([&](CommandServer* server) { + auto* ab = server->getArtboardInstance(probe.artboardHandle); + REQUIRE(ab != nullptr); + artboardBounds = ab->bounds(); + }); + probe.pump(); + } + // surfaceBounds is AABB(origin, viewBounds), so view == artboard needs an + // origin-based artboard. + REQUIRE(artboardBounds.left() == Approx(0.0f)); + REQUIRE(artboardBounds.top() == Approx(0.0f)); + REQUIRE(artboardBounds.width() > 0.0f); + REQUIRE(artboardBounds.height() > 0.0f); + + const Vec2D viewBounds(artboardBounds.width(), artboardBounds.height()); + + auto boundsByIdAtScale = + [&](float scaleFactor) -> std::unordered_map<uint32_t, AABB> { + SemanticTestListener listener; + SemanticFixture fx(&listener); + fx.commandQueue->enableSemantics(fx.stateMachineHandle); + fx.warmup(); + fx.drain(0, Fit::layout, Alignment::center, scaleFactor, viewBounds); + fx.pump(); + std::unordered_map<uint32_t, AABB> out; + for (auto& kv : listener.m_model.nodes) + { + AABB b = kv.second.bounds(); + if (b.width() > 0.0f && b.height() > 0.0f) + { + out[kv.first] = b; + } + } + return out; + }; + + auto atScale1 = boundsByIdAtScale(1.0f); + auto atScale2 = boundsByIdAtScale(2.0f); + REQUIRE(!atScale1.empty()); + + // Doubling scaleFactor must double the mapped node sizes. + bool comparedAny = false; + for (const auto& kv : atScale1) + { + auto it = atScale2.find(kv.first); + if (it == atScale2.end()) + { + continue; + } + CHECK(it->second.width() / kv.second.width() == + Approx(2.0f).epsilon(0.02)); + CHECK(it->second.height() / kv.second.height() == + Approx(2.0f).epsilon(0.02)); + comparedAny = true; + } + REQUIRE(comparedAny); +}