Frame-Synchronous Hand Gesture Detection by Projected Winding Order

This is the paper in full, as announced on arXiv as arXiv:2609.13269. The code that produced every figure, and a demo that runs in the browser, are at Amey-Thakur/GESTURE-FX.

Abstract

Gesture recognition on video is normally posed as classification: assign a label to each frame, then act on the label. That formulation is adequate for control, where a command may be obeyed several frames late without a user noticing, and inadequate for synchronisation, where an output must be aligned to the frame on which the gesture physically occurred. We take the synchronisation problem for a specific and common movement, the rotation of an open hand about its own long axis, and show that it admits an exact solution requiring no classifier, no training data and no calibration.

Let s denote the normalised two-dimensional cross product of the two palm edges, taken at the wrist and the two outer knuckles, under the projection the camera already performs. We prove that s factorises as k(θ)cosθ with |k(θ)| > 0 everywhere, so that s vanishes if and only if the palm is edge-on and its sign tracks the face presented to the camera. Detecting the gesture therefore reduces to locating a zero crossing of a single scalar, which yields an instant rather than an interval. We further prove that the criterion is invariant to image mirroring, to hand scale and to handedness, and that these follow from the algebraic form rather than from any property of the estimator supplying the landmarks.

A second, two-handed interaction uses four fingertips as the corners of a window onto a restyled version of the same scene. We give the coverage predicate this requires, showing that the triangulation ordinarily used is unsound the moment the hands cross and that an even-odd test is both correct there and cheaper, and we derive the three operators that fill the window: an iterated edge-preserving filter whose range width must vary across iterations for a reason we quantify, a quantiser whose amplification of residual noise we bound, and a difference of Gaussians whose noise response we compute in closed form and use to fix its one free mixing parameter.

We embed both in a browser system in which landmark inference is rate limited well below the display rate and each recognised gesture carries the timestamp of its cause, so that presentation quality is independent of inference throughput. The system composites effects onto the recorded surface rather than the camera stream, so no editing stage is required. In its default configuration it runs entirely on the client, with no server, no key and no per-use cost; two optional paths depart from this, are disabled until enabled by the user, and are accounted for rather than elided.

Against a corpus with exact ground truth the criterion detects 95% of flips with no false positive in 240 near-miss sequences, and places each detection within 6.7 ms on average of the instant it occurred: a sixth of the interval at which the hand is observed. Reporting an event more finely than one samples is the practical consequence of treating a gesture as the zero of a continuous quantity rather than as a label attached to a frame, and it is unavailable to a classifier operating on the same stream. We state plainly what remains unverified. Keywords. Gesture Recognition · Hand Tracking · Projective Geometry · Real-Time Video · Human Computer Interaction · On-Device Inference

Introduction

A gesture recogniser that drives an interface is judged by whether it eventually fires. A gesture recogniser that drives a visual effect inside a recording is judged by when it fires, because the viewer sees the gesture and its consequence in the same footage and will attribute one to the other only if they coincide. An effect placed two hundred milliseconds after a hand movement does not read as caused by it; it reads as a coincidence.

This distinction is not usually drawn. The dominant formulation labels frames and acts on labels, which returns an interval during which a gesture was judged to be in progress. Recovering a single instant from such an interval is not well posed: the interval’s boundaries are artefacts of the classifier’s confidence profile, and that profile is at its worst exactly where the instant lies. During the fast middle of a hand rotation the hand is motion blurred and self-occluding, so a classifier is least certain precisely when certainty is required.

We show that for one important gesture the instant can be obtained directly, and exactly, from projective geometry. Contributions.

1. A criterion for detecting the rotation of an open hand about its long axis as the zero crossing of a single scalar computed from three landmarks, with a proof that the crossing coincides exactly with the edge-on configuration (Theorem 1).
2. Proofs that the criterion is invariant under image mirroring, under uniform scaling of the hand, and under exchange of handedness (Propositions 1–3), all following from its algebraic form rather than from the landmark estimator.
3. A coverage predicate for the two-hand window that remains correct when the quadrilateral self-intersects, with the exact area by which the triangulation it replaces overdraws in that case (Proposition 6, Corollary 2).
4. An analytic realisation of the three-representation cartoon decomposition, in which each of the three free parameters is fixed by a derived quantity rather than by inspection: the range schedule by the expected attenuation under sensor noise (Proposition 7), the band-selection neighbourhood by the quantiser’s slope (13), and the contour operator’s mixing fraction by its closed-form noise response (Proposition 8).
5. Sub-frame localisation of the event by interpolating the zero between the two samples that bracket it, with a proof that the error is second order in the sampling interval where reporting either sample is first order (Proposition 5), and a measurement showing the mean absolute error falling from 23.0 ms to 6.7 ms.
6. A generated corpus with exact ground truth, and the argument for generating rather than filming it, which is that the crossing frame is not observable in video to a precision finer than the quantity being measured (Section 6).
7. A rate-decoupled compositing architecture in which a trigger carries the timestamp of its cause, making the smoothness of the response independent of inference throughput (Section 4.6).
8. A bounded temporal buffer permitting retrospective compositing, which obtains an effect otherwise requiring generative synthesis from information the stream has already delivered (Section 4.7).
9. A statement of the condition under which a window tracked in one clip may be composited over a generatively restyled version of it, and of why that condition can be requested but not enforced (Proposition 10).
10. A complete client-side implementation, released under the MIT licence, with the components that were verified and those that were not reported separately (Sections 5–7).

Related Work

Hand pose estimation

Recovering hand keypoints from a single view was for a long time limited by the cost of annotation, since a hand is small, frequently self-occluded, and tedious to label. Simon et al. [1] broke the dependency by bootstrapping annotations across a multi-camera rig, using detections that succeed in one view to supervise the views in which they fail. The resulting detectors were integrated into the multi-person pose framework of Cao et al. [2], which established the part-affinity formulation for associating keypoints into skeletons.

Those systems target accuracy on server hardware. The mobile line of work instead fixes a latency budget and designs within it. Bazarevsky et al. [3] introduced a detector architecture tuned for mobile GPUs, extended to the body in Bazarevsky et al. [4] and to the hand in Zhang et al. [5], the last of which supplies the two-stage design we build on: a palm detector run intermittently, followed by a landmark regressor confined to the detected region, assembled in the pipeline framework of Lugaresi et al. [6].

We take such an estimator as given and contribute downstream of it. This matters for the scope of our results: the invariances in Section 4.3 are properties of the projection, not of any network, and therefore hold for any estimator whose output is a projection of the hand.

Classical hand analysis

Before learned estimators, hand gestures were recognised by pipelines assembled from the primitives collected in OpenCV [7]: skin-tone segmentation, contour extraction, convex hulls, and convexity defects to count extended fingers. These are sensitive to illumination, background and skin tone, which is precisely what learned landmark estimation removed.

It is worth being clear that the quantity we use is of the same kind as the ones those pipelines computed. A signed area over three points is a classical primitive, and had reliable landmark correspondences been available it could have been evaluated then. The contribution here is not the primitive but the observation that this particular one resolves the synchronisation problem exactly. The containment test of Section 4.9 belongs to the same family as the one a quadtree applies at every node, deciding which axis-aligned region a point falls in; Thakur et al. [8] builds that index for the interactive display of large point sets, where the cost of repeating the test is what bounds the frame rate. The problem here is the same in kind and smaller by orders of magnitude: four edges, once per fragment.

Gesture recognition from landmarks

Given landmarks, two families dominate. Learned classifiers map the landmark vector, or a window of them, to a gesture label; the real-time architecture of K{ö}p{ü}kl{ü} et al. [9] is representative, pairing a lightweight detector with a deeper classifier so that the expensive model runs only when a gesture is plausibly in progress. Rule-based recognisers instead threshold derived quantities such as joint angles and finger extension. The two are ends of a spectrum rather than alternatives: a neuro-fuzzy system keeps the rules and learns their parameters instead of fixing them by hand, which is the middle ground surveyed by Thakur et al.(2021{})Thakur, Dhiman, and Phansikar [10]. The learned end produces its per-frame label from a parameterised network trained by gradient descent, and the models and learning rules that make one, from the limitations of the perceptron to the rectified unit that answers them, are set out in Thakur and Konde [11].

Both families return a per-frame decision, and therefore an interval once aggregated over time. Our criterion is rule-based in implementation but is not a tuned rule: it is a closed-form consequence of the projection, and the thresholds around it gate evidence quality rather than define the gesture. That distinction is what allows a claim of exactness in Theorem 1 that a tuned rule could not support.

Locating a gesture in time

Two literatures address directly the question of where a gesture sits in a stream. Gesture spotting separates gestures from the movements between them in a continuous signal; the threshold model of Lee and Kim [12] is the canonical treatment, adding a garbage state against which candidate gestures compete so that a segmentation falls out of the recognition rather than being imposed before it. Temporal action localisation asks the same question of untrimmed video and answers it with proposals scored and refined by a network, in the multi-stage form of Shou et al. [13].

Both return a segment. That is the right output for their purposes, and it is one step short of what synchronisation needs: a segment has two boundaries and the instant we require is interior to it, so recovering the instant means positing a further rule over the segment. The boundaries are also where these methods are least certain, since they are defined by the decay of a score rather than by any event. The criterion here differs in kind rather than in accuracy. It does not locate a segment and then reduce it; the quantity it evaluates has a zero, the zero is the event, and there is nothing left to reduce.

Optical flow as a temporal cue

When the question is when something moved rather than what shape it took, dense optical flow is the usual instrument, most often in the polynomial-expansion formulation of Farneb{ä}ck [14]. It is the cue used for temporal localisation in the modular surveillance pipeline of Thakur and Talele [15], where the event to be located is a collision and flow magnitude peaks sharply at it.

Flow is poorly matched to the present problem. The gesture here is a rigid rotation passing through a particular orientation, and flow magnitude is elevated across the whole fast portion of that rotation rather than at its midpoint. Using it would reproduce the interval problem rather than resolve it. The distinction is between locating motion and locating a configuration, and only the latter admits the exact treatment of Section 4.3.

Filtering noisy interactive input

Landmark estimates jitter by a small but visible fraction of the frame even when the hand is stationary, and any geometry built from them inherits that jitter. A fixed low-pass filter trades jitter against lag and cannot serve both a stationary and a moving hand. Casiez et al. [16] resolve this with a filter whose cutoff rises with the estimated speed of the signal, so that a slow signal is smoothed heavily and a fast one is followed closely.

The corner filter in Section 4.8 is of this family. We depart from the original in one respect that matters for our architecture: because inference runs at a rate well below the display rate and that rate varies with load, we rescale the smoothing coefficient to the elapsed interval, so the filter’s behaviour in wall-clock time is invariant to the inference rate.

Appearance transformation of video

Restyling a subject is well studied as a learned image-to-image problem. The adversarial formulation it usually takes, a generator trained against a discriminator until its output is not separable from the target distribution, and the variants that formulation has since acquired, are surveyed by Thakur and Satish(2021{}) [17]. Two instances of it matter here. White-box cartoonisation [18] learns an explicitly decomposed representation, a surface carrying flat regions, a structure carrying segmented colour and a texture carrying contours, rather than an opaque mapping; it is that decomposition, and not its training, that Section 4.10 reproduces analytically. Open-domain sketch-to-photo synthesis [19] must produce photographic content for sketch classes absent from its training set, which is the sharpest case of the property at issue. Such methods synthesise content absent from the input, which is exactly what allows them to replace a subject and exactly what makes them unsuitable under a constraint of zero marginal cost and bounded latency: a generative model of useful quality is either hosted, and therefore metered, or too slow for interactive compositing.

We use an analytic shader instead, and are explicit in Section 7 that this restyles the subject present in the frame rather than replacing it. The distinction is stated because it is the one a reader is most likely to assume away.

Browser implementations

Several browser demonstrations composite effects over hand-tracked geometry, and the two-hand framing gesture in particular has an established following. In the survey we conducted before implementation, every such system terminated at a live preview: none encoded the composited output, and none stated a detection criterion that could be evaluated or reproduced. Where the interior of such a frame is restyled, video is routed to a hosted generative model, which reintroduces a server, a key and a per-use cost, and shifts the latency from frames to seconds or minutes.

Problem Formulation

Let V = (I₁, I₂, …) be a video stream with frame I_t observed at time t. Let Λ be a landmark estimator producing, for each observed frame, a set of hand landmark configurations in normalised image coordinates.

Definition 1 (Synchronisation problem). Given a gesture G occurring physically over the interval [t_a, t_b] with a distinguished instant t^⋆ ∈ [t_a, t_b], produce an estimate t̂^⋆ and a composited stream V’ in which a designated transformation is applied from t̂^⋆ onward, such that |t̂^⋆ - t^⋆| is below perceptual tolerance and V’ is encoded without a subsequent editing stage. Three constraints distinguish this from classification.

The instant is the deliverable. A classifier estimates the support [t_a, t_b]. Reducing that support to a point requires a rule that the classifier does not supply, and any such rule is sensitive to the confidence profile at the boundaries.

Evidence is worst where it is needed. t^⋆ lies in the interior of the movement, where angular velocity is highest, where motion blur is greatest, and where self-occlusion is most severe. Landmark confidence is therefore minimised at the instant to be recovered.

Inference and presentation compete. Let C_Λ be the per-frame cost of landmark estimation and C_R that of rendering. Naively evaluating both at the display rate f_d requires f_d(C_Λ + C_R), and since C_Λ ≫ C_R on commodity mobile hardware this caps the system at the inference rate. Section 4.6 removes the coupling.

Method

The projected palm winding

Let P₀, P₅, P₁₇ ∈ ℝ³ denote the wrist, the index knuckle and the little finger knuckle in a hand-fixed frame, with the y axis along the hand’s long axis and the z axis out of the palm. These three points span the palm plane and are, among the 21 landmarks, the three least affected by finger articulation.

Let R_θ be rotation about the long axis by θ and let π(x, y, z) = (x, y) be orthographic projection. Write the projected edges as

v₁(θ) = π(R_θ P₅) - π(R_θ P₀),    v₂(θ) = π(R_θ P₁₇) - π(R_θ P₀).

(1)

Definition 2 (Projected palm winding).

s(θ) = (v₁(θ) × v₂(θ))⁄ ‖ v₁(θ) ‖   ‖ v₂(θ) ‖,    a × b = a_x b_y - a_y b_x .

(2)

The numerator is twice the signed area of the projected triangle P₀ P₅ P₁₇; dividing by the edge lengths gives the sine of the angle between them, so s ∈ [-1, 1]. Its sign is the winding order of that triangle under projection.

The crossing theorem

Theorem 1 (Exact crossing). Write the unrotated projected edges as v₁(0) = (a, b) and v₂(0) = (c, d), and let κ = ad - bc. If κ ≠ 0 then for all θ

s(θ) = k(θ)cosθ,    k(θ) = κ⁄(‖ v₁(θ) ‖   ‖ v₂(θ) ‖),

(3)

with |k(θ)| > 0 for every θ. Consequently

sign s(θ) = sign(κ)sign(cosθ),    and    s(θ) = 0 ⇔ θ ≡ π⁄2 (mod π).

(4)

Proof. Rotation about the y axis maps (x, y, z) to (xcosθ + zsinθ, y, -xsinθ + zcosθ). The three landmarks lie in the palm plane z = 0, so after projection the x components are scaled by cosθ and the y components are unchanged: v₁(θ) = (acosθ,  b) and v₂(θ) = (ccosθ,  d). Hence

v₁(θ) × v₂(θ) = (acosθ)d - b(ccosθ) = (ad - bc)cosθ = κcosθ ,

(5)

which gives (3) on dividing by the norms. Both norms are bounded below by min(|b|, |d|) > 0, since b and d are the components along the axis of rotation and are unaffected by θ; the knuckles are displaced from the wrist along the hand, so neither vanishes. Therefore |k(θ)| > 0. Because k never changes sign, the sign of s is that of κcosθ, and s vanishes exactly where cosθ does. □

Corollary 2 (Instant recovery). A change of sign of s observed between two sampled frames localises θ = π/2 to the interval between them. The estimate t̂^⋆ is therefore accurate to the sampling interval of the estimator, independently of any confidence the estimator reports. This is the property the formulation was chosen for. The sampling interval, not the classifier’s certainty during the fastest part of the movement, bounds the error.

Figure 1. The projected palm triangle at three rotations, computed rather than drawn. Left: palm toward the camera, positive winding. Centre: edge-on, the projected triangle has no area and s = 0, which is the trigger. Right: back toward the camera, winding reversed. The printed values are those the implementation reads for the model hand used in the figure.

Figure 1 shows the configuration at θ ∈ {0, π/2, π}. Note that |s(0)| = 0.647 ≠ 1: the magnitude depends on the angle subtended at the wrist and hence on the individual hand. Only the sign and the zero are exact, and Theorem 1 claims only those. ### Invariance The following hold for the criterion itself, and therefore for any landmark estimator whose output is a projection of the hand. > **Proposition 3 (Mirror invariance).** Let M(x, y) = (-x, y) be reflection about the vertical axis, as applied to a > front-facing camera preview. Then s ∘ M = -s, and the zero set is > unchanged.

Proof. Reflection is linear, so the reflected edges are Mv₁(θ) and Mv₂(θ). Using v₁(θ) = (acosθ, b) and v₂(θ) = (ccosθ, d) from the proof of Theorem 1,

Mv₁(θ) × Mv₂(θ) = (-acosθ)d - b(-ccosθ) = -(ad - bc)cosθ = - v₁(θ) × v₂(θ),

(6)

while ‖ Mv ‖ = ‖ v ‖ since M is an isometry. Hence s ↦ -s for every θ. The zero set of -s equals that of s, so a crossing at a given instant remains a crossing at that instant. □ Mirroring is thus a global sign convention. A detector keyed on a change of sign is unaffected, whereas one keyed on an absolute sign, such as a palm-versus-back classifier, requires the convention to be tracked.

Proposition 4 (Scale invariance). For any λ > 0, s is unchanged under (v₁, v₂) ↦ (λ v₁, λ v₂).

Proof. The numerator is bilinear and so scales by λ²; the denominator is a product of two norms and so scales by λ². The quotient is homogeneous of degree zero. □ Hand size and distance from the lens act as such a λ under orthographic projection, so neither enters the criterion. No per-user calibration exists in the implementation.

Proposition 5 (Chirality). A left and a right hand differ in the sign of κ. The detection criterion is unaffected.

Proof. The two hands are related by reflection, which by the argument of Proposition 1 negates κ. By Theorem 1 the zero set of s is independent of signκ, and the detector is defined by a sign change of s, which is likewise independent of it. □

Proposition 6 (Degeneracy). κ = 0 if and only if the wrist and the two knuckles are collinear in the palm plane.

Proof. Immediate: κ is twice the signed area of the triangle they span. □ This configuration does not occur in a hand, so the hypothesis of Theorem 1 is satisfied in practice. It is nonetheless the correct failure mode to state, since a landmark estimator that collapses the three points would produce s ≡ 0 and no crossing.

From a crossing to a decision

Theorem 1 localises the instant but does not by itself distinguish a deliberate flip from any other rotation. The implementation gates the crossing with four conditions, each removing a failure observed during development:

1. |s| ≥ τ_arm sustained for 120 ms before the crossing, rejecting a hand that enters the field already rotating;
2. at least three fingers extended, rejecting a rotating fist and a conversational wrist turn;
3. the transit from |s| ≤ τ_cross to the opposite face completing within [20, 600] ms. The lower bound excludes a crossing that begins and ends inside one sample, which is what a single inverted landmark frame produces, and is far below the duration of a turn because it times the passage through edge-on rather than the turn: a hand that takes 300 ms to turn over is edge-on for some 40 ms of it, and a guard set at the duration of a turn rejects most real flips. The upper bound excludes a hand brought to edge-on and held there, not a hand turned over slowly, which passes through edge-on quickly whatever its overall pace and is the same gesture at a different speed;
4. the opposite face reaching |s| ≥ τ_confirm, rejecting a wobble toward edge-on that returns. Thresholds gate evidence quality; they do not define the gesture, which is defined by the crossing. The asymmetry of the design is deliberate: a missed gesture costs one repetition, whereas a spurious one corrupts a recording that cannot be repeated, so every default is biased toward the former.

Recovering the instant between samples

Theorem 1 places the gesture at the zero of s. A detector observes s only at the sampling instants, and the zero almost never coincides with one, so reporting the sample at which s was first observed inside a band about zero places the gesture early by about half the time the signal spends inside that band. At the rate this system tracks at, that error is tens of milliseconds and is visible: the effect begins before the hand has finished turning.

The zero is recoverable between samples. Let t_a be the last sampling instant carrying the sign presented before the rotation and t_b the first carrying the opposite sign. The reported instant is the linear interpolant,

t̂^⋆ = t_a + (t_b - t_a) (|s(t_a)|)⁄(|s(t_a)| + |s(t_b)|).

(7)

Proposition 7 (Second-order localisation). If the rotation rate is constant over [t_a, t_b], the error of (7) is O(Δ t²) in the sampling interval, whereas reporting either bracketing sample is O(Δ t).

Proof. By Theorem 1, s(t) = k(θ(t))cosθ(t) with k smooth and non-vanishing. Writing θ(t) = π/2 + ω(t - t^⋆) for a constant rate ω gives cosθ(t) = -sin(ω(t - t^⋆)), whose Taylor expansion about t^⋆ is -ω(t - t^⋆) + O((t-t^⋆)³). Hence s is linear in t about the zero up to a cubic remainder, and k contributes a smooth factor whose variation over an interval of length Δ t is O(Δ t) relative. Linear interpolation of a function that is linear to that order recovers its zero with an error of the order of the neglected terms, which is O(Δ t²). Reporting t_a or t_b instead incurs the full distance to the zero, which is O(Δ t). □ Section 6 measures this: the interpolant reduces the mean absolute localisation error from 23.0 ms to 6.7 ms and removes a 22.1 ms bias, at the cost of one division and two stored samples.

Rate-decoupled compositing

Let f_Λ be the rate at which the estimator is evaluated and f_d the display rate, with f_Λ ≪ f_d. Let a recognised gesture emit a trigger (t̂^⋆, e) carrying the estimated causal instant and an effect identifier, and let the effect be a function E(u) of normalised progress u ∈ [0, 1] over duration D. We render at f_d with

u(t) = (t - t₀)⁄(D),    t₀ = max(t̂^⋆, t_c - Δ),

(8)

where t_c is the confirmation time and Δ a bound on backdating.

Two consequences follow. Effect playback is sampled at f_d regardless of f_Λ, so reducing the inference rate degrades recognition latency and not smoothness. And because t₀ is anchored to the cause rather than to the detection, the effect occupies the correct position in the encoded output even though it was decided later. The clamp by Δ ensures the onset is not skipped when confirmation is slow, at the cost of a bounded misalignment; we use Δ = 120 ms.

The distinction is the one a distributed system draws between when an event happened and when a process learned of it. Ordering by a clock read at the observer orders by arrival, which is a property of the network rather than of the events; Lamport’s happened-before relation orders by cause instead, and the physical algorithms that narrow the gap sit alongside the logical and vector clocks that sidestep it in Thakur and Satish [20]. Anchoring the timeline on t̂^⋆ rather than on t_c is the same choice made for the same reason: the confirmation time is a property of the rate at which the hand is sampled, not of the gesture.

Retrospective compositing

Some desired effects require content absent from the current frame. Substituting the subject requires synthesis. Substituting the moment, however, requires only memory.

We retain a circular buffer of N frames at reduced resolution, captured at f_b ≪ f_d, holding N / f_b seconds of history. On a trigger the compositor selects

F^⋆ = argmin_i | τ_i - (t - D_r) |

(9)

for a requested delay D_r, where τ_i is the capture time of buffered frame i. Nearest-in-time rather than oldest is used so the effective delay remains meaningful while the buffer is filling and does not jump when it wraps. With N = 24, f_b = 8 Hz and quarter resolution the cost is approximately 5.5 MB of texture memory for three seconds of history. ### The two-hand frame A second interaction uses both hands. Four fingertips, two per hand, define a quadrilateral used as a window onto a restyled version of the same scene. Corners are held in anatomical order, so a corner corresponds to a fixed fingertip for the lifetime of the gesture and smoothing requires no correspondence search; crossing the hands yields a self-intersecting quadrilateral that recovers when they uncross, since the ordering is stateless.

Landmark noise is the dominant difficulty: fingertip estimates move by a noticeable fraction of the frame while the hands are still, and a window whose boundary shimmers is unusable regardless of its contents. We smooth each corner with a velocity-adaptive exponential filter in the manner of Casiez et al. [16], whose coefficient is rescaled to the elapsed interval,

α_{Δ t} = 1 - (1 - α₆₀)^{Δ t / Δ t₆₀},

(10)

so that the filter behaves identically at any inference rate, and we apply hysteresis to both gates, since a hand rotating toward the camera foreshortens and would otherwise cross a fixed threshold downward. ### Coverage of the window The window is a region a fragment shader must fill, so the quadrilateral has to become a predicate. The obvious construction, and the one we adopted first, splits it into the triangle fan F(Q) = T(c₀,c₁,c₂)∪ T(c₀,c₂,c₃) and tests membership of either. That is sound only while the quadrilateral is convex, and the anatomical ordering above guarantees that it is not always: crossing the hands exchanges one hand's two corners and yields a self-intersecting boundary. In that state the fan does not degrade gracefully. It covers most of the bounding rectangle, which is visible as the stylised region abruptly leaving the hands.

We use the even-odd rule instead: a point lies inside when a ray cast from it meets the closed boundary an odd number of times [21]. Write E(Q) for that region.

Proposition 8 (Coverage). Let Q be the closed polyline c₀c₁c₂c₃c₀ in general position.

1. If Q is simple and the diagonal c₀c₂ lies in its interior, then F(Q) = E(Q).

2. If the edges c₀c₁ and c₂c₃ meet at a point x interior to both, then

E(Q) = T(c₀,x,c₃) ∪ T(c₁,c₂,x) ⊆ F(Q),

and the inclusion is strict.

Proof. For (i), an interior diagonal partitions a simple quadrilateral into exactly the two triangles of the fan, and the even-odd region of a simple closed curve is its interior. For (ii), cutting Q at x decomposes the boundary into two closed simple loops, c₀ → x → c₃ → c₀ and x → c₁ → c₂ → x, meeting only at x and each traversed once. A ray from a point interior to either loop meets the boundary an odd number of times and a ray from a point exterior to both meets it an even number, which is the stated union. Containment in F(Q) is immediate because each lobe has its three vertices in one triangle of the fan, and both triangles are convex. Strictness holds because x is interior to c₀c₁, so the triangle T(c₀,c₁,c₂) of the fan contains a neighbourhood of points on the far side of c₀x from c₃; such points lie in no lobe, since each lobe is bounded by c₀x and xc₁ respectively. The corollary below measures the excess in one symmetric case. □

Corollary 9. For the crossed unit square c₀=(0,1), c₁=(1,0), c₂=(1,1), c₃=(0,0), the edges c₀c₁ and c₂c₃ meet at (1⁄2,1⁄2) and

|E(Q)| = 1⁄2,    |F(Q)| = 3⁄4 .

The fan therefore covers half as much area again as the window, and one third of what it draws lies outside it. The predicate costs four half-open vertical comparisons and at most four divisions. The fan it replaces costs a signed area for each of the five distinct edges of its two triangles, ten multiplications, so correctness here is obtained at a lower price than the failure it replaces.

Stylising the interior

The window shows the same scene in another medium. We compute the three representations of the white-box formulation [18] analytically rather than learning them, which removes the model but keeps the decomposition: a surface carrying flat interiors, a structure carrying quantised colour, and a texture carrying contours. **Surface.**An iterated bilateral filter [22], in the manner established for real-time abstraction by Winnem{ö}ller et al. [23],

I^(m+1)(x) = 1⁄(Z(x)) Σ_{y ∈ N(x)} I^(m)(y)  exp(-(‖y-x‖²)⁄(2σ_s²)) exp(-(‖I^(m)(y)-I^(m)(x)‖²)⁄(2σ_r^(m)2)),

(11)

with Z the sum of the weights. One pass suppresses noise; a short sequence collapses each region onto its own colour while leaving boundaries in place, and it is that flatness rather than any outline that makes the result read as drawn.

The range width σ_r^(m) is not held constant across the sequence, and the reason is quantitative rather than aesthetic.

Proposition 10 (Range attenuation under sensor noise). Let two pixels of one surface differ only by independent sensor noise, each of C channels distributed N(0,σ_n²). The expected range weight in (11) is

E[exp(-(‖Δ I‖²)⁄(2σ_r²))] = (1 + (2σ_n²)⁄(σ_r²))^-C/2.

Proof. Δ I has independent N(0,2σ_n²) components, so ‖Δ I‖²/(2σ_n²) ∼ χ²_C. Writing t=σ_n²/σ_r², the expectation is the moment generating function of χ²_C evaluated at -t, which is (1+2t)^-C/2. □ For C=3 the consequence is direct. A filter with σ_r=0.09 retains a weight of 0.74 between two samples of a surface carrying σ_n=0.03, and averages the noise away; against σ_n=0.06, which is an ordinary amount in a dim room, the weight falls to 0.39 and the filter defends each grain as though it were an edge. Widening to σ_r=0.24 restores 0.84. The schedule is therefore coarse to fine, σ_r = (0.24, 0.14, 0.09): the first pass averages, and the passes after it restore the boundaries the first softened. **Structure.**Tone is quantised in luminance with a controlled edge,

Q(L) = q + (Δ q)⁄2tanh(φ (L-q)⁄(Δ q)),    q = Δ q round(L/Δ q),

(12)

and the colour is rescaled by Q(L)/L, which flattens the shading and holds the hue. Rounding the three channels independently, which is what a posterisation does, moves a colour toward a vertex of the unit cube and turns skin green.

Differentiating (12),

Q'(L) = φ⁄2sech²(φ(L-q)⁄(Δ q)) ≤ φ⁄2,

(13)

with equality at a band centre. The quantiser is thus an amplifier of whatever variation reaches it, by a factor of φ/2 = 3.5 at the setting we use. Variation the surface has reduced below visibility is returned above it, and a large flat region whose tone sits near a band boundary flickers between two levels. We therefore select the band from a five-tap mean of the surface luminance with weights (2,1,1,1,1)/6, whose variance factor is Σ w_i² / (Σ w_i)² = 8/36 = 2/9; the input standard deviation falls by √2/9 = 0.471 and the effective amplification with it, to 1.65. Only the choice of band is made from a neighbourhood: the colour is still read per pixel, so no edge is softened. **Texture.**Line work is a difference of Gaussians [[24]](#ref-24),

D_σ = G_σ * L - G_kσ * L,    k = 1.6,

(14)

thresholded in the manner of the extended operator of Winnem{ö}ller et al. [[25]](#ref-25), though through a ramp rather than through that operator's hyperbolic tangent. A tangent approaches its limits without reaching them, so it returns a small nonzero coverage at zero difference and lays a faint wash of ink over every flat region in the frame; in a style that adds its line to a dark ground rather than mixing it in, the wash becomes the picture. A ramp is exactly zero above its bias. The two kernels are given equal weight, which is what makes that possible: both integrate to unity, so D_σ vanishes identically on any region of constant L, whatever that constant is. The weighted form G_σ - τ G_kσ with τ<1 instead leaves a residue (1-τ)L proportional to brightness, and no single threshold then draws the same line on a lit face and on a dark coat.

What the operator does respond to, besides contours, is noise.

Proposition 11 (Noise response). For per-pixel luminance noise of variance σ_L², independent across pixels,

Var D_σ = σ_L² ‖G_σ - G_kσ‖₂² = (σ_L²)⁄(4πσ²)(1 - 4⁄(1+k²) + 1⁄(k²)),

which for k = 1.6 equals 0.0212 σ_L²/σ².

Proof. Convolution with a fixed kernel scales an independent noise field’s variance by the squared L² norm of that kernel. Expanding, ‖G_σ - G_kσ‖₂² = ‖G_σ‖₂² - 2⟨ G_σ,G_kσ⟩ + ‖G_kσ‖₂². In two dimensions ‖G_a‖₂² = 1/(4π a²) and ⟨ G_a, G_b⟩ = 1/(2π(a²+b²)), since the integral of a product of Gaussians is a Gaussian of the combined width evaluated at the origin. Substituting a=σ, b=kσ gives the stated form, and k=1.6 gives 1 - 4/3.56 + 1/2.56 = 0.267, hence the coefficient 0.267/4π = 0.0212. □ The consequence settles a design choice. A camera frame with per-channel noise σ_n = 0.03 carries luminance noise σ_L = σ_n√0.2126²+0.7152²+0.0722² = 0.750 σ_n = 0.0225, so at σ = 1.4 pixels Proposition 8 gives a response of standard deviation 2.3×10^-3. The operator does not threshold at a point but over a ramp: ink begins at D_σ = -1.6×10^-3 and is laid in full at -4.8×10^-3, so the contrast at which a line is half drawn is ε = -3.2×10^-3. Grain therefore reaches two thirds of the contrast the operator asks of a contour, and exceeds outright the contrast at which it begins to draw, which is precisely the field of short marks we observed across flat walls. We read it instead from a mixture

L = (1-λ) L_surface + λ L_raw,    λ = 0.16,

(15)

where L_surface is the output of (11). The surface has already had the grain averaged out of it, so what reaches the operator is the raw share of the noise and its response scales with λ. Requiring the half-drawn contrast to stand at least eight standard deviations clear of that response bounds λ ≤ 0.17; we use 0.16, which places it at 8.6 and leaves the fine detail the half-resolution surface cannot carry. The value is fixed by the ratio the operator must achieve, not by inspection. ### Following the subject The gesture of Section 4.8 occupies both hands, so the device is propped rather than held and is not aimed accurately. The output frame is already a crop of a wider sensor, so the crop can follow a detected face [[3]](#ref-3) at no cost beyond the detection. Two properties are required of the update: that it not answer detection jitter, and that it not jump when it stops ignoring it. Writing p_t for the measured face centre and c_t for the framing,

c_t+1 = c_t + α (max(0, ‖e_t‖ - δ))⁄(‖e_t‖) e_t,    e_t = p_t - c_t .

(16)

Proposition 12 (Soft dead zone). The map (16) is continuous in e_t; its fixed set is the closed ball {c : ‖p-c‖ ≤ δ}; and for a stationary target the excess over the dead zone contracts geometrically,

‖e_t+1‖ - δ = (1-α)(‖e_t‖ - δ).

Proof. The coefficient max(0,‖e‖-δ)/‖e‖ tends to 0 as ‖e‖→δ from either side, so the displacement is continuous across the boundary. Inside the ball the coefficient is identically zero and every point is fixed; outside it the displacement has magnitude α(‖e‖-δ) > 0, so no point there is fixed. For ‖e_t‖ > δ the displacement is parallel to e_t with magnitude α(‖e_t‖-δ), so ‖e_t+1‖ = ‖e_t‖ - α(‖e_t‖-δ); subtracting δ gives the stated recurrence. □ A hard dead zone, which ignores motion below δ and follows it in full above, is discontinuous at the boundary and produces a visible jump each time the subject drifts across it. The form above never moves the framing by more than the excess, so a subject who crosses the frame slowly is followed the whole way, trailing by exactly δ. As in Section 4.8, α is rescaled to the elapsed interval, so the behaviour does not depend on the frame rate.

Delegated restyling

Section 4.10 restyles the subject that is present. Replacing it requires a generative model, and the constraint of zero marginal cost forbids hosting one. The remaining option is to let the user supply their own key, which moves the cost to them and keeps the deployment static. We implement that path and state exactly what it requires.

The model is asked to redraw the whole take, not the interior of the window. The window is then a reveal: the generated clip inside, the original outside. This is the only order that can work, because the boundary moves every frame and a model cannot be given a per-frame region to respect.

The correctness condition is a statement about the model’s output. Let R be the raw take and let the returned clip be

G(x,t) = S(R(W_t(x)), t)

(17)

for some appearance map S and per-frame spatial warp W_t. > **Proposition 13 (Alignment).** The composite that shows G inside the tracked window and R outside is > geometrically consistent if and only if W_t = id for every t.

Proof. The window is positioned from landmarks measured in R. A point x inside it displays G(x,t) = S(R(W_t(x)),t), whose content is that of R at W_t(x). Consistency requires the content shown at x to be the content of R at x, that is W_t(x) = x for all x in the window and all t; extending to the whole frame by continuity of the constraint across the boundary gives W_t=id. □ We cannot enforce Proposition 10; we can only request it. Every prompt therefore carries a fixed constraint forbidding the reframing a generative video model applies by default, and the style instruction is the only part the user edits. A residual warp is reported as a limitation rather than corrected, because correcting it means estimating W_t from the pair, which is a registration problem of the same order as the one avoided by asking the model not to introduce it.

Implementation

The system is a static web application. In its default configuration it contacts no server, holds no key and has no per-use cost; the one path that departs from this is Section 4.12, which is disabled until a user supplies their own credential. Hand landmarks come from a WebAssembly build of an on-device estimator [5], and face detection, when framing is enabled, from a second on-device model [3] built on the same runtime so the runtime is downloaded once. Compositing is WebGL 2 [26]; capture and encoding use the platform’s own interfaces [27][28].

Rendering is four passes rather than two. A first pass applies the persistent colour treatment together with the aspect crop, the mirror and the framing offset of Section 4.11. A second evaluates (11) at half resolution, ping-ponged between two targets so that no pass samples the texture it writes, and runs only while the window is open. A third composites the window, evaluating the style of Section 4.10 for the fragments inside it and no others. A fourth applies the transient effect. Separating the persistent treatment from the transient one is what allows any of the six filters to combine with any of the seven effects without a shader for each pairing.

The half resolution of the second pass is chosen for reach rather than for cost. A kernel of fixed tap count spans twice as much of the full frame there, which is what collapses skin and clothing into single regions instead of merely reducing their grain; the softness introduced at a boundary is removed again by (12), since quantising a ramp between two levels restores a step at the crossing.

Figure 2. The pipeline. Inference (cyan) is rate limited and feeds a timeline anchored to the causal instant; rendering runs at the display rate. The encoder reads the composited surface (magenta), so the effect is present in the output as pixels and no editing stage exists.

The decision with the largest consequence is that the encoder captures the composited surface rather than the camera stream (Figure 2). The alternative, recording the camera and reapplying effects afterwards, would require storing trigger times, decoding, compositing offline and re-encoding, and each stage is an opportunity for drift.

Figure 3. Verification of the temporal buffer. The synthetic subject renders the elapsed time at which each frame was drawn, and moves, so a recalled frame is distinguishable from a recoloured current one. The strip runs left to right: one live frame at 4.3 s, three recalled frames reading 2.5, 2.8 and 2.9 s, and one live frame after the effect at 5.6 s. The live clock at each recalled frame is not printed on it, but lies between those two bounds and advances with the strip, so each recalled frame is about the requested 2.2 s behind the moment it was shown.

Evaluation

We report what was measured and separate it from what was not, since the value of a systems claim lies in that distinction. **Frame budget.**At 60 Hz the budget is 16.7 ms. Landmark inference measured 6–11 ms per evaluation on desktop integrated graphics, texture upload 1–2 ms, and an effect shader 0.5–2 ms. Inference dominates, which is what motivates Section 4.6. Feature extraction and detector evaluation for two hands and five detectors measured below 0.1 ms combined, consistent with the criterion requiring one cross product per hand. **Window coverage.**The predicate of Section 4.9 was checked against the failure it replaces. With one hand’s corners exchanged, so that the boundary self-intersects, the even-odd predicate renders two lobes and leaves the region between them untouched; sampling on the axis through the crossing confirms the interior of each lobe is filled and the two points where the shape pinches are not. The fan predicate fills both, which is Corollary 2 at work: for the symmetric configuration it draws half as much area again as the window, a third of it outside. **Abstraction.**A flat field was given independent per-channel grain drawn uniformly on ± 15 of 255, a standard deviation of 8.7, and delivered to the renderer through the same capture path a camera uses. The mean absolute difference between horizontally adjacent output samples, measured well inside the window and clear of its border, was 4.5 of 255 without the window and 0.2 with it. The grain is drawn from a seeded stream, so the two figures are reproducible rather than merely reported. What the schedule of Section 4.10 leaves is therefore below the amplification of (13), which is what (15) and the five-tap band selection were sized to achieve. **Separation of the styles.**The seven media are intended to be nameable from a still. We rendered one frame through each and measured the mean absolute channel difference over the interior of the window for all twenty-one pairs. The widest, neon against ink, differed by 122.7 of 255; the closest, cartoon against paint, by 7.6, against a floor of 6 set before the measurement. Those two share a flattened surface and differ mainly in saturation and in the direction of their strokes, so they are the pair a reader should expect to find closest, and 7.6 is a thinner margin than the rest of the set enjoys. The measurement is reported, and its weakest pair named, because ``they look different’’ is otherwise an assertion by the author about their own work. **Behavioural coverage.**Four suites run against the shipped modules. Ninety assertions cover the render passes, the coverage predicate under crossing, the framing clamp, the noise reduction, the tracker’s hysteresis and dropout behaviour, the spoken command matcher, the credential store, the prompt constraint, the geometry track and the exchange of Section 4.12 against a stubbed transport. Forty-two further assertions drive the application through its own document with a synthetic camera and microphone, from the first-visit guide to a downloaded file, and confirm among other things that audio is present in the encoded output by decoding it rather than by trusting the container’s declaration.

That distinction earned its cost. It caught a defect no weaker check could see: the container was selected once at startup, before it was known whether a take would carry sound, and the preferred string named a video codec alone. A recorder given an explicit codec list encodes those codecs and no others, so an attached audio track was accepted and then discarded without error. The track was present in the stream, the file was well formed, its declared type named the container correctly, and every recording was mute. Only decoding the output distinguishes that state from a working one. The correction is to choose the container per take, once the track set is known, rather than once at startup.

A third suite of twenty-two puts synthetic landmark poses through the recording-cue detector at the rate the render loop actually calls it, which is where a detector that treats a throttled frame as an absent hand is caught. A fourth of ten stubs the speech interface and pushes transcripts through the shipped matcher. The four suites total one hundred and sixty-four assertions. **Temporal buffer.**The buffer was verified by rendering a subject that states its own draw time and moves between draws (Figure 3). With a requested delay of 2.2 s, the strip runs from a live frame at 4.3 s, through three recalled frames reading 2.5, 2.8 and 2.9 s, to a live frame at 5.6 s, with the subject visibly displaced between them. Each recalled frame therefore sits about 2.2 s behind the live clock at the moment it was shown, which advances across the strip. The buffer returns genuine earlier frames at approximately the requested delay, the residual being the 125 ms capture granularity at f_b = 8 Hz. **Two-hand frame.**The tracker was exercised with synthetic landmark configurations. It correctly returned no window with no hands, opened to full presence on a framing configuration, produced corners matching the constructed rectangle to two decimal places, held the window through a 133 ms tracking dropout, closed after the hands left, and rejected a configuration whose area fell below the opening gate. **End-to-end.**The second suite covers acquisition, recording, review, export, output presets, filters, effect rebinding, settings persistence, theme selection, focus containment and keyboard dismissal, and passes against both the development server and the deployed build. The distinction matters: the bundler moves stylesheet links in the built page, and an override that wins by source order in development can lose in the build. **Detector accuracy against known ground truth.**The quantity that matters for a frame-synchronous detector is the error between the instant the gesture occurred and the instant reported. That quantity cannot be obtained from recorded video: an annotator watching a hand rotate can identify the edge-on frame only to within the several frames over which the hand appears edge-on, which is the same order as the error to be measured. Any figure derived that way would report the annotator rather than the detector.

We therefore generate the corpus. A rigid twenty-one point hand is rotated about its long axis through π radians on a known schedule, projected orthographically, and perturbed by independent Gaussian landmark noise of standard deviation 0.004 frame widths. The instant θ = π/2 is then known exactly. Sequences are drawn from a seeded stream, so the figures below are reproducible rather than merely reported, and the evaluation runs the shipped detector modules rather than a reimplementation of them [29].

Four near-miss classes are included, each sharing part of a flip’s signature. A detector that fires on everything scores perfectly on the flip class alone, so the near misses are what make the positive result meaningful.

Class	Requirement	Fired	Correct
Palm flip, quick, 180–520 ms	must fire	54/60	90.0%
Palm flip, deliberate, 0.9–1.6 s	must fire	60/60	100.0%
Held edge-on, 0.8–1.4 s	must not fire	0/60	100.0%
Wobble to 54–79 degrees	must not fire	0/60	100.0%
Rotating closed hand	must not fire	0/60	100.0%
Entering frame mid-rotation	must not fire	0/60	100.0%
Total		95.0% detected, 0 false positives in 240 near misses

Table 1. Classification over 60 generated sequences per class, at f_Λ = 24 Hz with the shipped thresholds. The two flip classes differ only in how quickly the hand is turned.

	Reported at ta	Interpolated
	ms	frames	ms	frames
Mean absolute error	23.0	0.55	6.7	0.16
Median absolute error	22.2	0.53	4.0	0.10
Ninetieth percentile	41.7	1.00	18.1	0.43
Worst case	66.3	1.59	26.7	0.64
Mean signed error	-22.1	—	+0.8	—

Table 2. Localisation error on the 114 detected flips, reported at the bracketing sample and after the interpolation of Section 4.5. The sampling interval is 41.7 ms. Both columns are computed by the published evaluation.

Three things in Tables 1 and 2 are worth stating plainly.

The detector localises the gesture to a sixth of the interval at which it observes, and to a tenth of it in the median case, against the 0.55 frames that reporting the bracketing sample costs. That is the practical content of Proposition 5: a zero crossing carries information between samples that a per-frame label does not. A classifier operating on the same stream cannot report an instant finer than the frame it labels.

The mean signed error after interpolation is under one millisecond, so what remains is scatter rather than bias, and its source is the landmark noise entering through the two bracketing magnitudes.

The six missed flips are the fastest in the class. A hand turned over in 180 ms is edge-on for roughly 25 ms, which is shorter than the sampling interval, so no sample falls inside the band and no bracket exists. This is a limit of the observation rate rather than of the criterion, and only a higher tracking rate moves it. **An incidental finding.**A canvas capture stream emits frames only while the canvas is painted, and a browser may cease painting a surface it considers not visible while still reporting the document as visible. The recording then encodes nothing and the failure surfaces only at the end. We detect the condition with a timer rather than within the render loop, since the condition is the loop not running, and abandon the recording after 2.5 s with an explanation. We record this because it is a property of the platform that any comparable system will meet.

Limitations

Restyling is not substitution. The shader restyles the subject present in the frame. It cannot replace that subject, which would require synthesis of content the stream never carried. Systems that do so route video to a hosted generative model, reintroducing a server, a key and a per-use cost, and operate either offline or at a latency set by the service.

Orthography. Theorem 1 assumes orthographic projection, and the factorisation (3) does not hold verbatim under perspective. The qualitative conclusion does. Three points project to collinear image points under a pinhole camera exactly when they are coplanar with the centre of projection, so the projected triangle still degenerates once per half turn and s still changes sign there. The degeneracy occurs when the palm plane contains the centre of projection rather than at θ = π/2 precisely; the discrepancy is set by the angle the hand subtends from the optical axis and vanishes as the hand approaches it. The crossing therefore remains exact as an event, with a small bias in the associated angle, and only the magnitude thresholds, which are gates rather than definitions, are affected.

Rotation axis. The analysis assumes rotation about the hand’s long axis. A rotation with a component about the optical axis leaves s unchanged, so it is correctly ignored; a rotation about the remaining axis is not modelled and is excluded in practice by the finger-extension condition.

Unverified surfaces. No test was performed with a physical camera, with real hands, or on Apple platforms. The gesture thresholds are derived rather than fitted, but no threshold has met a human hand. Recording on iOS is the least certain component: the interfaces are supported and four documented defects are mitigated, but none of this was confirmed on a device.

The corpus is generated, not filmed. Table 1 measures the criterion and its gating; it does not measure the landmark estimator, which appears in the corpus as unbiased noise of constant variance and is in reality neither. A hand also deforms as it turns, and a rigid model does not. The figures are therefore a floor on the error attributable to the detector, not a prediction of field performance, and the gap between the two is the estimator’s contribution. We consider this the honest form of the measurement rather than a substitute for one: the alternative, annotating filmed rotations, would report the annotator’s uncertainty about the edge-on frame, which is of the same order as the quantity being measured. **The delegated path is verified as far as the free tier reaches.**Section 4.12 is implemented against a preview interface whose response shape has more than one documented form and whose model name has changed between revisions, so its transport was exercised against the live service rather than only against a stub. Four properties were confirmed. The endpoint accepts the request the implementation constructs, returning an interaction bearing the identifier and status field the client polls on, and reporting the terminal state that ends the loop. The call succeeds from a page rather than from a server: the service sends permissive cross-origin headers, so the browser-only architecture of Section 5 needs no proxy, and a proxy would have reintroduced the server the design exists to avoid. A take produced by the recorder, in the container of Section 5, is accepted and decoded: the service billed 539 video tokens for a four-second clip and returned a description naming frame content that was present only in the video, which establishes that frames survive capture, encoding and transport intact. Finally, the media type must be sent without its codec parameters, and the payload must be separated from the data URL header at the ;base64, marker rather than at the first comma, because a container that names two codecs contains a comma of its own; both were found by this exercise and both are defects the stub could not have exposed.

What remains open is the generation itself. The editing model carries a free-tier quota of exactly zero and answers with a quota refusal rather than a result, so the alignment of a real generation was not measured. Proposition 10 states the condition the composite requires; whether a given model satisfies it is an empirical question that a metered key would settle and this work does not, and the interface reports a mismatch rather than concealing it. **Two paths are not local, and both are opt-in.**Spoken command recognition uses the browser’s own speech interface, which in the two most common engines transmits audio to the vendor; the restyle transmits one recording to a hosted model. Both are disabled by default, both state what they do at the point of enabling, and both display an indicator while active. We record this here rather than only in the interface because a paper that claimed a wholly local system while shipping two exceptions would be making a false claim, and because the design question of how such an exception should be surfaced is not incidental to a privacy-preserving system. **A key held in a page is exposed to that page.**The delegated path requires a credential in the browser, which no client-side design can protect from the page that uses it. We reduce the surface by loading no third-party script at runtime, by keeping the credential out of storage unless asked, and by transmitting it in a header rather than a query string, but the residual risk is real and is stated to the user in those terms.

Conclusion

Posing hand-gesture detection as synchronisation rather than classification changes what counts as a solution. For rotation of an open hand about its long axis, the change admits an exact answer: the projected palm winding factorises as k(θ)cosθ with k non-vanishing, so the gesture is a zero crossing of one scalar and the crossing is the instant. The criterion needs no training data, no calibration and one cross product per hand, and is provably invariant to mirroring, to scale and to handedness.

Measured against a corpus with exact ground truth, the criterion detects 95% of flips with no false positive in 240 near-miss sequences, and places each one within 6.7 ms on average of the instant it occurred, which is a sixth of the interval at which the hand is observed. That a detector can report an event more finely than it samples is the practical consequence of treating the gesture as a zero of a continuous quantity rather than as a label attached to a frame, and it is the result we would ask a reader to take from this work.

The implementation, the derivation, the figure-generating code and the evaluation itself are released under the MIT licence, and the evaluation runs in a browser against the shipped modules, so every figure above can be reproduced by opening a page rather than by rebuilding a toolchain [29].

Availability

Source code, documentation and the scripts that generate every figure in this paper: https://github.com/Amey-Thakur/GESTURE-FX. A live deployment: https://amey-thakur.github.io/GESTURE-FX/. Released under the MIT licence.

---

Citation

Please cite this work as:

Thakur, Amey. "Frame-Synchronous Hand Gesture Detection by Projected Winding Order." arXiv:2609.13269 (Sep 2026). https://arxiv.org/abs/2609.13269.

Or use the BibTex citation:

@article{thakur2026winding,
  title         = "Frame-Synchronous Hand Gesture Detection by Projected Winding Order",
  author        = "Thakur, Amey",
  journal       = "arXiv preprint arXiv:2609.13269",
  year          = "2026",
  month         = "Sep",
  eprint        = "2609.13269",
  archivePrefix = "arXiv",
  url           = "https://arxiv.org/abs/2609.13269"
}

---

References