We found something out while sketching on the board: our developer team has little artistic skill. Ask us to draw a rectangle and you get something... let's say "not so beautiful."
So we decided to make Drawdy watches what you draw with the pencil. If you sketch something that's clearly meant to be a rectangle, a circle, a diamond, or a triangle and then pause for a moment, we quietly swap your rough stroke for a clean version of the same shape — same size, same spot.
This post is about how that guess actually happens: when we decide to try, how we figure out which shape you drew, and how we make sure a wrong guess never costs you your drawing!
Better than a screenshot — try it! Sketch a rough shape below and hold still for a beat:
Knowing when to guess
The first question isn't what you drew — it's when to look. Recognizing on every pointer move would be wasteful and jumpy; waiting until you lift the pen would feel late. So we settle in the middle: we watch the pencil's stroke as it grows, and every time you move we reset a short timer. If you keep drawing, the timer keeps getting pushed back. The moment you hold still for about a third of a second, it fires.
private _scheduleInterpretation(): void {
this._clearTimer();
this._timer = setTimeout(() => {
this._timer = null;
this._tryInterpret();
}, PAUSE_MS);
}
The cheap guess first
When the timer fires, we run two recognizers in order, cheapest first. The cheap one is a plain geometric classifier — no templates, no machine learning, just measuring the stroke and asking a few pointed questions.
Before anything else, it checks whether the stroke is even closed. We compare the distance between where you started and where you ended against the size of the whole drawing (its bounding-box diagonal). If the ends are miles apart, this was probably a line or a squiggle, not a shape, and we bail out early:
distance(start, end) / diagonal ≤ 0.3
If it is roughly closed, the first real fork is "round or pointy?". Every point on a perfect ellipse satisfies one equation, and every point on a perfect diamond satisfies another — and both equations equal exactly 1 right on the outline:
ellipse: ((x − cx) / a)² + ((y − cy) / b)²
diamond: |x − cx| / a + |y − cy| / b
So we run every point of your stroke through both formulas and see which one lands closest to a flat 1 across the board. Whichever model your points hug more tightly wins:
for (const [x, y] of points) {
const dx = (x - cx) / a;
const dy = (y - cy) / b;
ell.push(dx * dx + dy * dy); // ellipse model
dia.push(Math.abs(dx) + Math.abs(dy)); // diamond model
}
return { ellipseStd: std(ell), diamondStd: std(dia) };
Why do this before counting corners? Because a hand-drawn circle is never smooth. It has little kinks that a corner-counter would happily mistake for the four sides of a polygon. Checking the overall roundness first stops a shaky circle from being read as a blocky shape. And it's still sharp enough to tell a genuinely bulged diamond from a real circle.
Both models measured against the same stroke — whichever one the points hug more tightly wins.
Hand-drawn edges are jittery, so first we simplify the stroke with the Douglas–Peucker algorithm. It throws away every point that sits close to a straight line and keeps only the ones that actually turn. A few hundred jittery samples collapse down to a handful of real vertices. Then we measure the angle at each surviving vertex and count only the sharp turns (more than 45°). Three sharp corners is a triangle. Four is either a rectangle or a diamond.
Telling those last two apart is the neat part, because a diamond is just a square turned 45°. The trick is where the corners sit relative to the bounding box. A rectangle's corners live in the corners of its box; a diamond's corners sit at the midpoints of the box's edges. We measure how close each corner is to an edge-midpoint — we call it "cornerness" — and if it's high enough, it's a diamond.
const isDiamond = cornerness > 0.25;
Falling back to templates
The geometric classifier is fast and it's right most of the time, but it's rigid — it only fires when it's confident. When it shrugs, we fall back to the classic $1 Unistroke Recognizer, a well-known gesture-matching algorithm that compares your stroke against a library of example strokes and finds the closest match.
this.points = Unistroke._resample(points, NumPoints);
this.points = RotateBy(this.points, 0);
this.points = ScaleTo(this.points, SquareSize);
this.points = TranslateTo(this.points, Origin);
this.vectors = Vectorize(this.points);
The templates it compares against aren't hand-recorded — we generate them in code. Rectangles, circles, diamonds, and triangles, at a range of proportions, drawn both clockwise and counter-clockwise. Each one is also offset to start from a different point along its outline. That last detail matters: two people drawing the "same" triangle might start at different corners, and we want both to match.
There's one thing $1 fundamentally can't do, and it's the reason the geometric classifier goes first. The version we use is rotation-invariant — it deliberately ignores orientation so a tilted sketch still matches. But that means it literally cannot tell a square from a diamond, because to it they're the same shape at different angles. The geometric pass, which does care about orientation, settles that question before $1 ever gets a look.
One design choice worth knowing about: the clean shape is built straight from your stroke's bounding box, so it always comes out upright. Sketch a tilted square and you get a level one back. We chose that on purpose — when someone scribbles a quick box mid-brainstorm, "straighten it for me" is almost always the favor they wanted. And if the stroke is tiny, or clearly open like a line or an arrow? We leave it alone. Not everything you draw is a shape, and the interpreter knows when it's not being asked.
That's the whole trick: a well-timed pause, a cheap measurement that catches the easy cases, and a proven template matcher for the awkward ones. Draw it rough, and if we're confident, you get it clean — and if we're wrong, your real stroke is one undo away.