Gestures, joystick, keyboard and picking

Read taps, swipes, drags, pinches, a joystick and browser keys, and map screen points to the world.

Tap, swipe and long press

GameView reports onTap(x, y), onSwipe(direction) and onLongPress(x, y) in points from the view's top-left, which are CSS pixels on the web (don't multiply by devicePixelRatio). A long press fires after half a second held still, and no tap follows it.

Touches claimed by onOverlayTouch never reach these callbacks, and pausing the view cancels any touch in progress.

Tap to bounce, swipe to move, hold to shake

React and React Native · shared engine API

Tap to bounce, swipe to move, hold to shake
import { GameView, type SwipeDirection, type World } from "@nayan-ui/engine";


type Props = { world: World; move: (direction: SwipeDirection) => void };


export function Board({ world, move }: Props) {
  return (
    <GameView
      source={world}
      style={{ flex: 1 }}
      onTap={(x, y) => {
        const hit = world.pick(x, y);
        if (hit) world.animate(hit.entity, { scale: 1.2 }, { yoyo: true, repeat: 1 });
      }}
      onSwipe={move}
      onLongPress={(x, y) => {
        const hit = world.pick(x, y);
        if (hit) world.animate(hit.entity, { shake: 0.1 });
      }}
    />
  );
}

Drag and pinch

onDrag receives start, move and end; an end with cancelled means a pinch or the system took the touch, so put a held piece back. onPinch reports two-finger zoom and rotation, and starting one ends any drag.

Drag and pinch reference
Event fieldMeaning
DragEvent.phase"start" | "move" | "end".
DragEvent.x / yFinger position in the view, in points.
DragEvent.dx / dyDistance moved since the drag started.
DragEvent.cancelledOn end: the drag was interrupted.
PinchEvent.phase"start" | "move" | "end".
PinchEvent.scaleFinger spread relative to the start; 2 is twice as far apart.
PinchEvent.rotationRadians turned since the start, clockwise on screen.
PinchEvent.x / yMidpoint between the fingers, in points.

Drag pieces on a board and pinch to zoom

React and React Native · shared engine API

Drag pieces on a board and pinch to zoom
import { useRef } from "react";
import { GameView, type Camera, type Entity, type ScreenPlane, type Vec3, type World } from "@nayan-ui/engine";


const BOARD: ScreenPlane = { normal: [0, 0, 1] }; // an XY board facing the camera
const point: [number, number, number] = [0, 0, 0];


export function DragBoard({ world }: { world: World }) {
  const camera = useRef<Camera>({ eye: [0, 0, 20], target: [0, 0, 0], fov: 1, ortho: 6 }).current;
  const held = useRef<{ piece: Entity; from: Vec3 } | null>(null);
  const zoom = useRef(6);
  return <GameView source={world} camera={camera} style={{ flex: 1 }}
    onDrag={({ phase, x, y, cancelled }) => {
      const hit = phase === "start" ? world.pick(x, y) : null;
      if (hit) held.current = { piece: hit.entity, from: world.position(hit.entity) ?? [0, 0, 0] };
      const h = held.current;
      if (!h) return;
      if (phase !== "end") {
        if (world.screenToWorld(x, y, BOARD, point)) world.set(h.piece, { position: point });
        return;
      }
      if (cancelled) world.set(h.piece, { position: h.from });
      held.current = null;
    }}
    onPinch={({ phase, scale }) => {
      if (phase === "start") zoom.current = camera.ortho ?? 6;
      else camera.ortho = Math.min(12, Math.max(2, zoom.current / scale));
    }} />;
}

On-screen joystick

useJoystick() returns a stable { x, y } object that Joystick writes on touch, each axis in -1..1 with y up, without re-rendering React. Read it in onUpdate.

The stick is relative: wherever the finger lands in the pad is the centre. Render it beside GameView, not inside it, and position it with style; size is the pad diameter (default 150).

Steer a player with the stick

React and React Native · shared engine API

Steer a player with the stick
import { GameView, Joystick, useJoystick, type Entity, type World } from "@nayan-ui/engine";


export function Controls({ world, player }: { world: World; player: Entity }) {
  const stick = useJoystick();
  return (
    <>
      <GameView source={world} style={{ flex: 1 }} onUpdate={() => {
        world.set(player, { groundVelocity: [stick.x * 6, -stick.y * 6] }); // stick up = forward (-z)
      }} />
      <Joystick state={stick} size={140} />
    </>
  );
}

Pick, project and unproject

world.pick(x, y) tests the bounding boxes of what was last drawn and returns a PickHit { entity, point, distance } or null; it needs no physics, and pickable: false opts an entity out. screenToWorld(x, y, plane?) intersects a plane, by default the ground (y = 0); pass normal [0, 0, 1] for an XY board.

toScreen(position) returns view points, or null behind the camera, and all three return null until a frame has been drawn. Each takes a last out argument (a PickHit, [x, y] or [x, y, z]) so per-frame code allocates nothing.

Keep a name tag above an entity

React and React Native · shared engine API

Create the callback once (for example with useMemo) and pass it as onUpdate; place moves a View or DOM element you own.

Keep a name tag above an entity
import type { Entity, World } from "@nayan-ui/engine";


export function tagFollower(world: World, target: Entity, place: (x: number, y: number) => void, hide: () => void) {
  const at: [number, number, number] = [0, 0, 0];
  const screen: [number, number] = [0, 0];
  return () => {
    if (!world.worldPosition(target, at)) return hide();
    at[1] += 1.2; // above the head
    if (world.toScreen(at, screen)) place(screen[0], screen[1]);
    else hide(); // behind the camera, or nothing drawn yet
  };
}

Browser keyboard input

createKeyboardInput(surface, options) turns arrow keys, WASD, Space and Enter into pointer events on an element wrapping GameView, so onTap, onDrag and onSwipe work unchanged. It uses the DOM: call it in browser code only, never in React Native.

Enter calls GameView's onPrimaryAction (Start, Play Again) when set, otherwise it taps at options.primary() or the cursor. Keys typed into inputs and buttons are ignored, and window blur or a hidden tab releases held keys.

Browser keyboard input reference
Option / methodBehavior
mode: "pointer"Default. Arrows move a cursor; Space presses and holds there.
mode: "drag"Held arrows drag from the centre, for steering.
mode: "swipe"Each arrow is one 80-point swipe, for 2048-style games.
enabled()Return false to ignore keys and release held input, e.g. while paused.
primary()Point, in CSS pixels, that Enter taps without onPrimaryAction.
cancel() / dispose()Release a held press; remove listeners and the cursor on unmount.

KeyboardGame.tsx

React · browser

KeyboardGame.tsx
"use client";
import { useEffect, useRef } from "react";
import { createKeyboardInput, GameView, type SwipeDirection, type World } from "@nayan-ui/engine";


type Props = { world: World; paused: boolean; onSwipe: (direction: SwipeDirection) => void; onStart: () => void };


export function KeyboardGame({ world, paused, onSwipe, onStart }: Props) {
  const surface = useRef<HTMLDivElement>(null);
  const pausedRef = useRef(paused);
  useEffect(() => { pausedRef.current = paused; }, [paused]);
  useEffect(() => {
    if (!surface.current) return;
    const keyboard = createKeyboardInput(surface.current, { mode: "swipe", enabled: () => !pausedRef.current });
    return () => keyboard.dispose();
  }, []);
  return (
    <div ref={surface} tabIndex={0} style={{ position: "relative", height: "100%" }}>
      <GameView source={world} paused={paused} onSwipe={onSwipe} onPrimaryAction={onStart} style={{ height: "100%" }} />
    </div>
  );
}