--- # Athena Lab listing metadata. Not part of the blueprint; ignore when building. title: CNN Image Labeler summary: Local-first web tool to annotate any folder of images (polygons, boxes, points, whole-image tags) and export training datasets in YOLO and COCO formats. description: "CNN Image Labeler is a single-user, local-first image annotation application for building computer-vision training datasets. Point it at any folder of images, define your own classes in the UI, draw polygon/box/point annotations or apply whole-image tags, and export directly to YOLO segmentation, YOLO detection, or COCO JSON. Labels are stored in a portable sidecar directory beside the images with a crash-safe append-only journal." version: 1.0.0 category: Developer & Technical Tools platforms: [Web, Desktop] stack: [Python 3, Python standard library http.server, Pillow, tkinter, HTML5 Canvas, vanilla JavaScript, CSS] tested_with: [Claude Code] tags: [image annotation, dataset labelling, YOLO export, COCO export, bounding box, polygon segmentation, keypoints, whole-image classification, computer vision, local-first, crash-safe journal] --- # CNN Image Labeler — Application Blueprint This document is a complete, self-contained specification for rebuilding the **CNN Image Labeler** from scratch in any reasonable technology stack. It captures behaviour, data shapes, rules, and the reasoning behind non-obvious choices. The source code is treated as disposable; this blueprint is the durable artifact. A competent agent given only this document should produce a functionally equivalent application. --- ## 1. Purpose and Scope ### Core value (plain language) The application is a tool for **labelling images so that a machine-learning model can be trained on them**. A user opens a folder full of pictures, defines the set of "things" they care about (for example `car`, `roof`, `defect`), and then goes image by image drawing shapes around those things or tagging whole images. When done, they press one button and the app produces a ready-to-train dataset in the standard formats that computer-vision training tools expect (YOLO and COCO). The labels are saved right next to the images, so the folder is self-contained and can be moved or backed up freely. ### Domain terms (define these; the rebuilder has zero prior knowledge) - **Annotation**: a single labelled shape or tag on one image (e.g. one polygon outlining one object, or one whole-image tag). - **Class** (a.k.a. **classification**, **category**, **label**): a named kind of thing being labelled, e.g. `car`. Every annotation references exactly one class by name. - **Class registry**: the ordered list of classes for a project. It is the single source of truth used both by the drawing UI and by the exporters. - **Polygon**: a closed shape of ≥3 vertices used for segmentation masks. - **Bounding box** / **box** / **bbox**: an axis-aligned rectangle. The word "box" is the user-facing label; `bbox` is the internal type name. - **Point** / **keypoint**: a single clicked location. - **Tag** (a.k.a. **whole-image class**): a class applied to the entire image with no geometry — image-level classification. - **Project**: one opened image folder plus its class registry and labels. - **YOLO / COCO**: the two industry-standard dataset formats the app exports. ### What is IN scope - Open any local folder of images; no import/copy step, images stay in place. - Manage classes entirely from the UI (add, rename, recolour, rebind key, change default tool, delete). Classes are **data, not code**. - Four annotation types: polygon, bounding box, point, whole-image tag. - Durable, crash-safe label storage beside the images. - Export to YOLO segmentation, YOLO detection, and COCO JSON, as a downloadable zip. - Single user, single machine, no network beyond localhost. ### What is explicitly OUT of scope - **No authentication, accounts, multi-user, or collaboration.** One user, one machine, one open project at a time. - **No cloud storage or remote sync.** All data is on the local filesystem. - **No model training or inference inside the app.** It only produces datasets. (There is a deliberately-reserved extension point for model *pre-labelling* / suggestions, but v1 does not implement it — see §10.) - **No image editing** (crop, rotate, brightness). Images are read-only inputs. - **No recursive folder scanning.** Only images directly in the chosen folder are labelled; subfolders are not descended into for images. - **No undo history / version timeline** beyond a single-step undo and the raw append-only journal used for recovery. - **No per-annotation metadata** beyond `{type, cls, pts}` (no confidence, author, timestamp on individual shapes). - **No dataset-level train/val configuration UI.** The validation split is a fixed rule (see §8). --- ## 2. Users and Roles There is exactly **one role: the local user**. The application is a personal, single-user desktop-style tool delivered through a browser. - **The local user** can do everything: open folders, manage classes, annotate, navigate, and export. - There is **no anonymous vs authenticated distinction**, no admin, no read-only viewer. Every request is trusted. - **How the system decides the role**: it does not. There is no identity concept. The server binds to `127.0.0.1` (loopback only) and assumes the single operator of the machine is the user. Anyone who can reach the port has full control; this is acceptable *only* because it is loopback-bound (see §7). A rebuild MUST preserve the "loopback-only, no auth" security posture, or add real authentication if it exposes the server beyond localhost. Do not naively bind to `0.0.0.0` without adding auth — that would expose the user's filesystem browser and image contents to the local network. --- ## 3. Core Workflows The following are the complete user journeys, step by step. "Client" = browser UI; "Server" = local backend. ### 3.1 Launching and (re)attaching to a project 1. The user starts the server process (a command that launches the local HTTP server on a port, default `8210`) and opens `http://localhost:` in a browser. 2. On page load, the client immediately calls `GET /api/project` to see whether a project is already open server-side (e.g. after a page refresh while the server kept running). - **If a project is open**: the client receives `{root, project, images}`, fetches labels via `GET /api/labels`, renders the class chips, sets the active class to the first class (if any), positions to the first unlabelled image, and loads it. - **If no project is open**: the server responds with an error (HTTP 500 with `{error}`); the client catches it and just renders an empty class panel with the prompt to open a folder. This is the normal cold-start state, not an error the user sees. 3. State that must survive a browser refresh (which image, which class) is derived fresh from server state, not stored in the browser. The "current project" lives on the server as a single global; the browser is stateless across reloads. **Failure/edge**: If the server was restarted (so no project is open) but the browser still shows an old page, a refresh returns the cold-start state. The user re-opens the folder; because labels live on disk in the folder, nothing is lost. ### 3.2 Opening a folder of images 1. User clicks **Open folder…**. 2. Client calls `POST /api/pick_folder` (with the most recent folder as the starting directory, if any). 3. Server opens the **native OS folder-picker dialog** (see §8.7) and blocks until the user chooses or cancels. - **Chosen**: returns `{path: ""}`. Client proceeds to open it. - **Cancelled**: returns `{path: ""}`. Client does nothing (stays as-is). - **Native dialog unavailable** (no GUI/display): server returns `{error, fallback: true}` with HTTP 200. Client falls back to an **in-app folder browser** modal (see 3.2a). - **Network/other error** on the request: client also falls back to the in-app browser. 4. To open a chosen path, client calls `POST /api/open {path}`. 5. Server: - Resolves to an absolute path; errors if not a directory. - Loads or **creates** the project sidecar (`.labelproj/project.json`); a brand-new folder gets a default empty-class project. - Records the path in the recent-folders list (max 12, most-recent first, de-duplicated). - Sets this as the single open project; creates the label store. - Returns `{root, project, images}` where `images` is the sorted list of image filenames directly in the folder. 6. Client fetches labels (`GET /api/labels`), updates the header to show the path and image count, sets the active class to the first class if none is selected, positions to the **first image that has no label record**, and loads it. **Edge**: Opening a folder with zero images shows "(no images)" on the canvas area and the label panel functions but there is nothing to draw on. Opening a folder that already has a `.labelproj` restores its classes and labels exactly. #### 3.2a In-app folder browser (fallback only) A modal that lists the current directory's subfolders (each with a count of image files it contains), an "up one level" entry, a "Use this folder (N images)" button, and a Recent-folders list. Navigation is server-side via `GET /api/browse` (the server can read the local filesystem because it runs on the user's machine). The root view lists drive letters on Windows (`C:\`, `D:\`, …) or `/` on POSIX. Dotfiles and the `.labelproj` directory are hidden from the listing. Choosing "Use this folder" calls the same `POST /api/open` flow. ### 3.3 Managing classes Classes are the set of labels available in this project. The registry is an **ordered array**; the order is meaningful (it determines export class IDs). **Add a class**: 1. User types a name, optionally picks a "default tool" (`any`, `polygon`, `box`, `point`, `whole-image`), clicks **+ add** (or presses Enter in the name field). 2. Client calls `POST /api/classes/add {name, shape}`. 3. Server validates the name is non-empty and unique (case-sensitive exact match); rejects duplicates with an error the client alerts. Assigns: - `id`: smallest non-negative integer not already used (stable per class; independent of array order). - `key`: a keyboard shortcut — the first alphanumeric character of the lowercased name not already taken by another class; else the first free `a–z`; else empty string. - `color`: next colour from a fixed palette, cycled by current class count. - `shape`: the chosen default tool, defaulting to `any`. 4. Server appends to the registry, persists `project.json`, returns the whole updated project. Client re-renders chips. If this was the first class, it becomes the active class. **Edit a class** (click the ✎ on a chip): a modal with name, key (single char), colour (colour picker), and default tool. Save calls `POST /api/classes/update {id, name?, key?, color?, shape?}`; only provided, non-null fields change. Delete (with confirm) calls `POST /api/classes/delete {id}`. Both return the updated project. **Reorder classes** (`POST /api/classes/reorder {order:[ids]}`) exists on the server and reassigns array order to the given ID sequence, but the v1 UI does not expose a reorder control. A rebuild MAY add one; the endpoint's contract is: the new array order equals the given ID order (unknown IDs dropped, omitted IDs dropped). **Rules and edge cases**: - Add enforces name uniqueness; **update does NOT** re-check uniqueness, so a rename can create a duplicate name. This is a known gap; a rebuild should ideally enforce uniqueness on rename too, but must at minimum not crash. - Deleting a class removes it from the registry but **does not touch existing annotations**, which still carry the class *name* string. On export, any annotation whose class name is no longer in the registry is silently **skipped** (it has no class id). The edit modal states this to the user: "Existing annotations keep the label text but the class leaves the registry." - Renaming a class does **not** rewrite existing annotations' `cls` strings. Annotations keep the old name and will be skipped on export unless a class with the old name still exists. (A rebuild may choose to cascade renames; v1 does not. State whichever you choose.) ### 3.4 Annotating an image The active **tool** and active **class** are independent selections. The tools are `polygon`, `box` (bbox), `point`, `tag`, and `edit`. Only one tool is active. **Drawing a polygon**: 1. Select the polygon tool (or a class whose default tool is polygon; or press `space` to toggle between polygon and box). 2. Click on the image to place each vertex. An in-progress open polyline is shown. 3. Close the polygon by double-clicking or pressing `c`. A polygon needs ≥3 vertices to be created; closing with fewer discards the in-progress points. 4. On close, the polygon is added to the image's annotations with the active class and **autosaved** immediately. - Polygon clicks draw **over anything**, including on top of existing objects. **Drawing a box**: 1. Select the box tool. 2. Press and drag on the image; a dashed preview rectangle follows the cursor. 3. Release to create the box. The box is stored as two corners (top-left and bottom-right, normalised so corner 0 ≤ corner 1 on both axes). Autosaved. 4. A box smaller than a few display pixels on either side is discarded (treated as an accidental click, not a box). - A release **outside** the canvas still finalises the box (see §5, stuck-drag prevention). **Dropping a point**: 1. Select the point tool. 2. Click. A single point annotation is added at that location and autosaved. **Applying a whole-image tag**: 1. Select the **tag** tool. In tag mode the canvas is **inert** — clicks and drags on the image do nothing (you cannot move or draw objects). 2. Pick a class (click its chip or press its hotkey). This **toggles** that class as a whole-image tag: if not present it is added, if present it is removed. You stay in tag mode. 3. Active tags are shown as `#classname` pills near the image. Autosaved on each toggle. **Editing / moving / reclassifying** (the `edit` tool — the ONLY mode that manipulates existing geometry): 1. Select the **edit** tool. 2. Hover feedback: over a vertex the cursor is the move cursor (↔ four-way arrows); over an object's body the cursor is a hand; over empty space it is the default arrow. 3. **Move a vertex**: press on a vertex (polygon corner, box corner, or a point) and drag. A box corner drag reshapes the box from the opposite corner. Release autosaves. 4. **Move a whole object**: press inside an object's body and drag; all its vertices translate together. Release autosaves. 5. **Select without moving**: a press that does not drag selects the object under the cursor (shown with a dashed white outline). Clicking empty space deselects. 6. **Reclassify**: with an object selected, press a class hotkey or click a class chip; the selected object's `cls` changes to that class (recolours immediately, autosaves). You stay in edit mode. **Removing shapes**: - `z` undoes: if a polygon is in progress it removes the last placed vertex; otherwise it removes the most recently added annotation (array pop). Autosaves. - Press `Delete`/`Backspace` to enter a one-shot **delete mode** (a "DELETE — click a shape" banner shows). The next press on the canvas removes the topmost shape under the cursor (point within grab radius, or inside a polygon/box), then delete mode exits automatically. Autosaves. **Validation on drawing**: - Drawing is only possible when a project is open **and** an active class is set; if there are no classes, the canvas does nothing on interaction. - Coordinates are stored as integers in **image-pixel** space (rounded). ### 3.5 Navigating images and saving Saving is **automatic**; there is no explicit save button and no "mark done" step. - `→` (Right) = next image; `←` (Left) = previous image. Both **autosave the current image first**. - `Shift`+`→` / `Shift`+`←` = jump to the **nearest unlabelled** image forward / backward (skips images that already have a label record). Autosaves first. - `↑` = first image; `↓` = last image. Autosave first. - **Jump to** box + **go**: type a 1-based image number and jump to it (clamped to range). Autosave first. - **resume** button: jump to the first unlabelled image ("continue where you left off"). If all images are labelled, shows "all images labelled". Autosave first. **The empty-image rule (important)**: An image with **no annotations is never stored** as a record. Specifically, `autosave`: - If the current image has ≥1 annotation → save a record `{status:"labeled", annotations:[...]}`. - If it has 0 annotations **and a record previously existed** → **delete** the record (so it is not stored as "empty"). - If it has 0 annotations and no prior record → do nothing. Consequently, moving past an image without drawing anything leaves no trace, and the "labeled N" counter reflects only images that actually have annotations. There is no separate "empty/negative" marker in v1 (a deliberate simplification — see §10). **Completion feedback**: reaching the last image via "next" shows "DONE — all images seen". Jump-unlabelled with none remaining shows a status message. ### 3.6 Exporting a dataset 1. User picks a format (`YOLO segmentation`, `YOLO detection`, or `COCO JSON`) and clicks **export**. 2. Client calls `POST /api/export {format}`. 3. Server validates ≥1 class exists (else error "no classes defined"), builds the dataset into `.labelproj/export//` (wiping any previous export of that format first), zips it to `.labelproj/export/.zip`, and returns `{ok, format, stats:{images, annotations, classes}, dir, zip}`. 4. Client shows a success line with the counts, the export directory path, and a **download zip** link (`GET /download?zip=`). - Only images that have a saved record with ≥1 annotation are included. Images whose files are missing on disk, or whose dimensions cannot be read, are skipped. - See §8 for exact output formats. **Failure**: export errors (e.g. no classes) return `{error}`; the client shows the message inline in red. --- ## 4. Data Model All persistent data lives on the local filesystem. There is no database. ### 4.1 On-disk layout For an opened image folder `ROOT/`, all app data lives in a sidecar directory `ROOT/.labelproj/`: ``` ROOT/ img_001.png, img_002.jpg, ... # the user's images (never modified) .labelproj/ project.json # class registry + project meta labels.json # all labels for this folder labels_journal.ndjson # append-only write journal (recovery) labels.json.snapshot # rolling backup, every 10th save export/ yolo_seg/ yolo_seg.zip # last export of each format (regenerated) yolo_detect/ yolo_detect.zip coco/ coco.zip ``` App-level (not per-project) state lives with the application install, in a single file (`state.json`) holding recent folders. **Reasoning**: labels are stored **beside the images** so the folder is self-describing and portable — move or copy the folder and the labels travel with it. This is a deliberate choice over a central database (see §10). ### 4.2 Entity: Project (`project.json`) | Field | Type | Default | Notes | |---|---|---|---| | `name` | string | folder basename | Display name; used as COCO `info.description`. | | `created` | number (epoch seconds, float) | now | Creation timestamp. | | `image_exts` | array of string | the supported extensions | Informational; the scanner uses a fixed list. | | `default_shape` | string | `"polygon"` | Present but **unused by the v1 client** (the client's own default tool is polygon). Arbitrary; a rebuild may drop it. | | `classes` | array of Class | `[]` | The class registry, ordered. | ### 4.3 Entity: Class (element of `project.classes`) | Field | Type | Constraints | |---|---|---| | `id` | integer | Unique within the project; smallest free non-negative integer at creation. Stable; **not** the export id. | | `name` | string | Unique on add (case-sensitive). Referenced by annotations via this string. | | `key` | string | Single keyboard-shortcut character (may be empty). Auto-assigned, editable. Not guaranteed unique after manual edits. | | `color` | string | Hex colour `#rrggbb`. Auto-assigned from palette, editable. Used for chip swatch and shape stroke. | | `shape` | string | One of `any`, `polygon`, `bbox`, `point`, `tag`. The class's **default tool**, NOT a constraint (see §10). `any` = geometry-agnostic. | **Export id vs class id**: The export id of a class is its **index in the `classes` array** (0-based), NOT its `id` field. Array order therefore matters. The `id` field only identifies a class for CRUD operations. ### 4.4 Entity: Labels file (`labels.json`) A JSON object mapping **image filename** → **record**: ``` { "img_001.png": { "status": "labeled", "annotations": [ , ... ] } } ``` - Key = the image's filename (not a path, not an index). Chosen so labels are stable across folder moves and independent of scan order. - `status`: always `"labeled"` for stored records in v1 (empty images are not stored at all). The field is retained for forward-compatibility. - `annotations`: array of Annotation (see below). Never empty for a stored record (an emptied image's record is deleted). ### 4.5 Entity: Annotation (element of `annotations`) All geometry is in **image-pixel coordinates** (origin top-left, x right, y down), stored as integers. | `type` | `cls` | `pts` | Meaning | |---|---|---|---| | `polygon` | class name | `[[x,y], …]` (≥3 vertices) | Segmentation polygon. | | `bbox` | class name | `[[x0,y0],[x1,y1]]` | Axis-aligned box; corner 0 = top-left, corner 1 = bottom-right (normalised so x0≤x1, y0≤y1). | | `point` | class name | `[[x,y]]` | Single keypoint. | | `tag` | class name | *(absent)* | Whole-image class; no geometry. | **Reasoning for image-pixel coords** (not normalised 0–1): images in a folder are arbitrary, differing sizes. Storing raw pixels keeps the label independent of any display scaling and makes editing exact; normalisation is applied only at export time, per image, using that image's real dimensions. (An earlier sibling tool used fixed-size crops and could store normalised coords; this general tool cannot assume a fixed size.) ### 4.6 Entity: Journal (`labels_journal.ndjson`) Append-only, one JSON object per line, never rewritten: ``` {"ts": , "id": "", "rec": } ``` - Every save appends a line **before** `labels.json` is updated. - `rec: null` denotes a **delete** of that image's record. - Purpose: crash-safe recovery. `labels.json` can be reconstructed by replaying the journal (last line per `id` wins; `null` means removed). This exists because a raw overwrite once destroyed real labelling work in a predecessor tool. A rebuild MUST keep an equivalent durable-write mechanism. ### 4.7 Entity: App state (`state.json`) ``` { "recent": [ "", ... ] } # most-recent first, max 12 ``` De-duplicated; the just-opened folder is moved to the front. --- ## 5. Business Rules and Edge Cases State these precisely; several took real thought. 1. **One project open at a time.** The server holds a single global "open project". Opening another folder replaces it. There is no multi-project in-memory state. (Rationale: single-user local tool; keeps state trivial.) 2. **Empty images are never persisted.** See §3.5. The `autosave` rule is: ≥1 annotation → write `labeled` record; 0 annotations with a prior record → delete record; 0 annotations with no prior record → no-op. "Empty" always means "nothing here", never a stored row. Consequence: the progress counter counts only images with annotations; there is no negative/confirmed-empty concept. 3. **Autosave triggers.** Every mutation autosaves immediately: closing a polygon, finishing a box, dropping a point, toggling a tag, moving/reshaping in edit mode, reclassifying, undo (`z`), and delete-mode removal. Navigation also autosaves the current image before moving. There is no explicit save action. 4. **Durable write order (crash safety).** On save: (a) append `{ts,id,rec}` to the journal; (b) write `labels.json` atomically via temp-file-then-rename; (c) every 10th save (when the record count is a multiple of 10) copy `labels.json` to `labels.json.snapshot`. The atomic rename guarantees `labels.json` is never a partial file. Never overwrite `labels.json` in place. 5. **Concurrency.** The server is multi-threaded (a threaded HTTP server). All writes go through a single per-store lock, so saves are serialised. In practice there is one browser tab, so real concurrency is minimal, but the lock must exist to keep the journal and `labels.json` consistent under overlapping requests. `load` is not locked (readers may see a slightly stale but always internally-consistent file thanks to atomic replace). 6. **Class id vs export id.** Export ids are array indices (§4.3). A class's stable `id` is only for CRUD. Reordering classes changes export ids — this is intended, so the user can control the numeric class ids in the exported dataset by ordering. 7. **Deleted/renamed class → orphan annotations.** Annotations reference classes by name string. If the name is not in the current registry at export time, the annotation is silently skipped (no id). This is the intended failure mode; the UI warns on delete. 8. **Coordinate rounding.** All stored coordinates are integers (rounded on creation/move). Sub-pixel precision is not retained. Acceptable because labels are pixel-level. 9. **Box normalisation.** A box always stores its corners min→max. Reshaping a box by dragging a corner recomputes from the opposite (fixed) corner, so a box can be "flipped" through and stays normalised. Tiny boxes (< a few display px per side) are discarded on creation. 10. **Stuck-drag prevention.** Because a drag can end with the mouse released outside the canvas, the finalisation logic is bound at the window level as well as the canvas. Any in-progress move or box-drag is finalised on a window-level mouse-up. Without this, a release off-canvas would leave the tool wedged (an early bug). A "just dragged" flag also suppresses the click event that immediately follows a drag, so a drag never also registers as a draw. 11. **Tool vs class independence.** The tool (how you draw) and the class (what you label) are independent. Selecting a class whose `shape` is a concrete tool switches the tool to it as a convenience; `shape: any` leaves the tool unchanged. In **tag** mode, selecting a class toggles the whole-image tag and does NOT change the tool. In **edit** mode, selecting a class reclassifies the selected object and does NOT change the tool. 12. **Tag exclusivity.** In tag mode the canvas cannot move or draw anything; only class selection (toggling tags) is meaningful there. This prevents accidental object manipulation while tagging. 13. **Display scaling never upscales.** The canvas display scale is `min(MAXW/imageWidth, MAXH/imageHeight, 1)` with `MAXW≈1100`, `MAXH≈760`. Images larger than the viewport shrink to fit; smaller images render at 1:1 (never blown up). Canvas image smoothing is disabled and CSS `image-rendering: pixelated` is set, so pixels are crisp — important for precise labelling on small images. All mouse coordinates are converted from display space back to image space by dividing by the scale. 14. **Hit-test radii** (in display pixels, i.e. threshold compared to `distance × scale`): vertex grab for polygon/box corners < 8; point vertex < 9; point body < 12. Body hit for polygons/boxes uses point-in-polygon (ray casting). Hit-testing iterates top-most annotation first. 15. **Undo scope.** `z` pops the last in-progress polygon vertex if drawing, else pops the last annotation in the array (not necessarily the selected one). This is a simple single-step undo, not a full history. 16. **Path containment.** Image serving verifies the requested file resolves inside the open project root (prevents `..` path traversal). See §7. 17. **Non-recursive scan.** Only files directly in the folder with a supported extension are treated as images; sorted case-sensitively by filename. Subfolders are listed only in the fallback browser for navigation, not descended for labelling. 18. **Supported image extensions** (case-insensitive): `.png`, `.jpg`, `.jpeg`, `.webp`, `.bmp`, `.tif`, `.tiff`, `.gif`. 19. **Cache-busting.** Image requests append a random query string so a re-loaded image is never served from browser cache incorrectly. 20. **Validation split determinism.** The YOLO export sends every image whose enumeration index `k` satisfies `k % 7 == 0` to the `val` split (so the **first labelled image, k=0, is a val image**), the rest to `train`. This is a fixed, non-configurable rule in v1. Enumeration order = iteration order of the labels map. (A rebuild may make the ratio configurable; the default behaviour above must be reproducible if matching this build.) --- ## 6. Interface and Interaction Single-page app: a left **sidebar** (controls) and a right **main area** (image canvas). Dark theme. Describe behaviour, not exact pixels (colours below are the build's choices and are free to change, except where noted load-bearing). ### 6.1 Layout - **Sidebar** (fixed width ~300px, scrolls): title, folder bar, Classes, add-class row, Tool row, Progress, Export, Shortcuts. - **Main area**: a small top bar showing whole-image tag pills; the image **canvas**; below it the image dimensions (`W×Hpx`) and the image filename. (The filename sits **below** the image, under the dimensions — deliberately not at the top, where it looked out of place.) - A **modal** overlay used for both the fallback folder browser and the class-edit dialog. ### 6.2 Sidebar controls - **Folder bar**: current project path + image count (or "No folder open"), and an **Open folder…** button (label changes to "Choose in dialog…" while the native dialog is open, then reverts). - **Classes**: chips, one per class, each showing a colour swatch, the class name, its shortcut key, its default tool (shown only if not `any`; `bbox` displays as "box"), and a ✎ edit affordance. Clicking a chip selects/acts on the class (per current tool); clicking ✎ opens the edit modal. The active class chip is outlined. - **Add-class row**: a name input, a "+ add" button, and a "default tool (optional)" dropdown (`any`, `polygon`, `box`, `point`, `whole-image`). Enter in the name field adds. - **Tool row**: five buttons — `polygon`, `box`, `point`, `tag`, `edit`. The active tool is outlined. Clicking a tool sets it and clears any in-progress drawing and selection. - **Progress**: a fill bar, the text ` / labeled `, a "Jump to" number input + **go** button, a **resume** button, and a status line for transient messages (DONE, delete-mode banner, etc.). - **Export**: a format dropdown (`YOLO segmentation`, `YOLO detection`, `COCO JSON`) and an **export** button; below it a result line (progress, success with counts + download link + output path, or error). - **Shortcuts**: a static help list. ### 6.3 Canvas behaviour and cursors - Cursor by mode: drawing tools → crosshair; tag → default (inert); edit → move (over vertex), hand (over object body), default (empty); delete-mode → not-allowed. - In-progress polygon shows an open polyline with vertex dots; in-progress box shows a dashed rectangle. - Each annotation is drawn stroked in its class colour with a translucent fill (~22% alpha); vertices/corners get small handle dots; points render as a circle with a crosshair. - In edit mode the **selected** annotation gets a dashed white outline. - Whole-image tags are not drawn on the canvas; they appear as `#name` pills in the top bar. ### 6.4 View states - **No project**: class panel shows "Open a folder, then add classes."; canvas blank; drawing does nothing. - **Project, no classes**: chips show "No classes yet — add one below."; drawing does nothing (no active class). - **Loading an image**: the image loads asynchronously; the canvas resizes to the scaled image size then draws. - **Empty folder**: canvas area shows "(no images)". - **Exporting**: result line shows "exporting…", then success or error. - **Delete mode**: a red "DELETE — click a shape" banner in the status line until a shape is removed. ### 6.5 Keyboard shortcuts (complete) Key events are ignored while focus is in a text input or select. | Key | Action | |---|---| | `←` / `→` | previous / next image (autosaves) | | `Shift`+`←` / `Shift`+`→` | nearest unlabelled image, backward / forward | | `↑` / `↓` | first / last image | | per-class `key` | pick that class (draw mode: set active + maybe switch tool; tag mode: toggle whole-image tag; edit mode: reclassify selected object) | | `space` | toggle tool between polygon and box | | `c` | close the in-progress polygon | | `z` | undo (last polygon vertex, else last annotation) | | `Delete` / `Backspace` | enter one-shot delete mode | | `Enter` (in "Jump to" field) | jump to that image number | | `Enter` (in add-class name field) | add the class | | double-click on canvas | close the in-progress polygon | There is intentionally **no** Enter-to-save and **no** mark-empty key; saving is automatic and empty images are simply not stored. --- ## 7. Authentication, Authorisation and Security - **No authentication or authorisation.** Single-user local tool. Every request is fully trusted. - **Loopback binding is the security boundary.** The server binds to `127.0.0.1` only. This is what makes "no auth" acceptable: only processes on the same machine can reach it. A rebuild MUST NOT expose this server on a non-loopback interface without adding real authentication — doing so would let anyone on the network browse the user's filesystem (via the folder browser and image server) and read/modify labels. - **Filesystem access is intentional and broad.** The server can read any directory the user's process can (that is the whole point of the folder picker/browser). This is safe only under the loopback assumption. - **Path traversal protection**: the image endpoint resolves the requested filename against the open project root and verifies (via a common-path check) that the resolved file is inside the root before serving it. Requests for files outside the project, or non-files, return 404. A rebuild MUST keep this check — without it, `GET /img/..%2f..%2f` would read arbitrary files. - **Input validation**: `open` rejects non-directories; `add_class` rejects empty or duplicate names and invalid shapes; `export` rejects when no classes exist. JSON bodies are parsed defensively; missing required fields yield a 400 with a `missing field` message. - **The native folder dialog runs a subprocess.** It launches a short-lived Python process that shows the OS dialog and prints the chosen path. The command is fixed (no user string is interpolated into a shell); the only argument passed is the initial directory. A rebuild must not pass untrusted strings into a shell. - **No secrets, tokens, cookies, or sessions** exist anywhere. - **CORS**: not configured; the UI is same-origin. A rebuild should keep the API same-origin (no wildcard CORS) given the filesystem power of the API. --- ## 8. External Interfaces and Contracts ### 8.1 HTTP API (server, localhost) All JSON. Errors return `{ "error": "" }` with a 4xx/5xx status (validation/keys → 400; unexpected → 500). Success shapes below. **GET `/`**, **GET `/index.html`** → the HTML page. **GET `/app.js`**, **GET `/app.css`** → static assets. **GET `/api/browse?path=`** → directory listing for the fallback browser: ``` { "path": "", "parent": "", "dirs": [ { "name": "", "path": "", "images": }, ... ], "images": , "roots": } ``` Empty/missing `path` returns the drive roots (Windows) or `/` (POSIX). Dotfiles and `.labelproj` are excluded from `dirs`. **GET `/api/state`** → `{ "recent": [ "", ... ] }`. **GET `/api/project`** → `{ "root": "", "project": , "images": [ "", ... ] }`. If no project is open → error (500). Used for reattach-on-load. **GET `/api/labels`** → the entire `labels.json` object, or `{}` if none. **GET `/img/`** → the raw image bytes with an appropriate image content-type. 404 if outside the project or not a file. **GET `/download?zip=`** → the export zip bytes, with `Content-Disposition: attachment; filename=""`. 404 if not found. **POST `/api/pick_folder`** body `{ "initial"?: "" }` → `{ "path": "" }` on success (empty string = cancelled); or `{ "error": "...", "fallback": true }` (HTTP 200) if the native dialog is unavailable. **POST `/api/open`** body `{ "path": "" }` → `{ "root", "project", "images" }` (as GET `/api/project`). Creates the sidecar/project if absent. **POST `/api/save`** body `{ "id": "", "rec": }` → `{ "ok": true, "count": }`. `rec:null` deletes the record. **POST `/api/classes/add`** body `{ "name", "shape"?, "key"?, "color"? }` → the updated Project. 400 on duplicate/empty name or invalid shape. **POST `/api/classes/update`** body `{ "id", "name"?, "key"?, "color"?, "shape"? }` → updated Project. Only provided non-null fields change. **POST `/api/classes/delete`** body `{ "id" }` → updated Project. **POST `/api/classes/reorder`** body `{ "order": [, ...] }` → updated Project (array reordered to match; unknown/omitted ids dropped). **POST `/api/export`** body `{ "format": "yolo_seg"|"yolo_detect"|"coco" }` → `{ "ok": true, "format", "stats": { "images", "annotations", "classes" }, "dir": "", "zip": "" }`. 400 if no classes. ### 8.2 File format: YOLO segmentation export (`format: yolo_seg`) Directory `export/yolo_seg/` containing: - `images/train/` and `images/val/` — copies of the labelled images. - `labels/train/` and `labels/val/` — one `.txt` per image. - `data.yaml`. - `classification.csv` — only if any whole-image tags exist. Each label line: ` ...` where class_id is the class's array index and each coordinate is normalised to 0–1 by the image's width/height, formatted to **6 decimal places**. Polygons are emitted as their vertices; boxes are emitted as their 4 corners (TL, TR, BR, BL); points are omitted (no area). `data.yaml`: ``` path: . train: images/train val: images/val nc: names: ["class0", "class1", ...] ``` (names are JSON-quoted, in registry order.) `classification.csv`: header `image,tags`; one row per image that has tags, with `tags` = semicolon-joined class names. Split rule: image #k (0-based over labelled images) is `val` iff `k % 7 == 0`. ### 8.3 File format: YOLO detection export (`format: yolo_detect`) Same directory structure as 8.2. Each label line: ` `, all normalised 0–1 (6 decimals), where `cx,cy` is the box centre and `w,h` its size. **Boxes** use their own extent; **polygons** are reduced to their axis-aligned bounding box; **points are omitted**. Tags → same `classification.csv`. Same `data.yaml` and same split rule. ### 8.4 File format: COCO export (`format: coco`) Directory `export/coco/` containing `images/` (image copies) and `annotations.json`: ``` { "info": { "description": "", "date_created": "YYYY-MM-DD" }, "images": [ { "id": <1-based int>, "file_name": "", "width": , "height": , "tag_category_ids": [ , ... ] }, ... ], "annotations": [ , ... ], "categories": [ { "id": <0-based index>, "name": "" }, ... ] } ``` Annotation objects (all share `id` (1-based), `image_id`, `category_id`, `iscrowd:0`): - polygon → `segmentation: [[x1,y1,x2,y2,...]]` (flat), `bbox: [x,y,w,h]`, `area: w*h` (bbox area, not exact polygon area). - bbox → `bbox: [x,y,w,h]`, `area: w*h`. - point → `keypoints: [x,y,2]`, `num_keypoints: 1`, `bbox: [x,y,0,0]`, `area: 0`. Whole-image tags are attached to each image as `tag_category_ids` (a non-standard COCO extension field; standard consumers ignore it). **Contract note**: `category_id` and each image `id`/`file_name` are stable within one export but not across exports (ids are assigned by iteration order). External consumers should key on `file_name` and category `name`, not on numeric ids, if they need stability across re-exports. ### 8.5 File format: `labels.json`, `project.json`, journal, state Exact shapes in §4. These are the app's own persistence contracts; a rebuild that wants to read existing projects must match them. If a rebuild does not need to read existing data, it may choose its own on-disk shapes but should preserve the **semantics** (portable sidecar, journal-first durable writes, filename-keyed labels, image-pixel coords, array-order class ids). ### 8.6 Background jobs / scheduled tasks / webhooks None. Everything is synchronous request/response. There are no timers, queues, or external services. ### 8.7 Native OS folder dialog (integration) The **hard requirement** is: the user must be able to choose a real filesystem folder and have the server receive its absolute path. A browser cannot return a real path (the File System Access API yields only a sandboxed handle), so the server — running locally — opens an OS-native folder dialog and returns the chosen path. This build uses a GUI toolkit (tkinter) launched in a **short-lived subprocess** (not in the server thread, to avoid GUI-toolkit main-thread constraints inside the threaded server). If no GUI/display is available, the server signals `fallback: true` and the client uses the in-app server-side folder browser instead. A rebuild may use any native dialog mechanism; it must provide both the native path-return and the headless fallback. ### 8.8 Technology hard-requirements vs free choices - **Hard requirements** (must implement to be equivalent): YOLO seg/detect line formats and `data.yaml`; COCO JSON structure; the val split rule; image-pixel coordinate storage with per-image normalisation at export; the durable journal-first + atomic-write + snapshot storage; the portable `.labelproj` sidecar semantics; filename-keyed labels; class array-order → export id; loopback-only + path-traversal protection; the four annotation types and five tools with the exact interaction semantics in §3/§5; empty-image = no record; native folder path selection with headless fallback. - **Free choices**: the specific web framework or lack of one; the GUI toolkit for the dialog; CSS/theme; the port number; whether the UI is inline HTML/JS or a component framework; whether `.labelproj` is literally named that. --- ## 9. Technology Stack (INFORMATIONAL) This build used: - **Language/runtime**: Python 3 (server) + browser JavaScript (client). - **Server**: Python standard library `http.server.ThreadingHTTPServer` with a custom request handler; **no web framework**. Chosen for zero dependencies and to match the host repo's dependency-light style. - **Image dimensions**: Pillow (PIL) — used only at export time to read each image's width/height. - **Native folder dialog**: tkinter (`filedialog.askdirectory`) in a subprocess. - **Client**: a single static HTML page, one vanilla-JS file (no framework, no build step), one CSS file. Rendering uses the HTML5 `` 2D context. - **Packaging/zip**: standard library (`shutil.make_archive`). - **Storage**: plain JSON + NDJSON files on the local filesystem; no database. - **Default port**: 8210 (arbitrary; free to change). A rebuilder may retarget any of this (e.g. Node/Express + React, or Go + a native webview) provided the hard requirements in §8.8 are met. --- ## 10. Non-Obvious Decisions These are the judgment calls that a generic "build me an image labeller" prompt would not recover. Preserve the intent; the mechanism is negotiable. 1. **Labels live beside the images (`.labelproj` sidecar), not in a central database.** Rationale: a folder is then self-describing and portable — move or copy it and the labels travel; two folders never collide; there is nothing to migrate. Trade-off accepted: no cross-folder querying. This is deliberate. 2. **The class registry is the single source of truth, shared by the UI and the exporters.** An earlier design hard-coded classes in three places (UI chips, colours, and export id maps); adding a class meant editing code. Collapsing all of that to one editable data array is the core "classes are data, not code" decision. A rebuild must not re-scatter class definitions. 3. **Class `shape` is an optional default tool, NOT a per-class geometry constraint.** A class can be drawn with any tool; `shape` only pre-selects a tool when you pick that class (and `any` doesn't even do that). Export keys off each annotation's own `type`, never the class's shape, so one class may contain polygons, boxes, and points at once. This was an explicit correction of an earlier design that forced one geometry per class. 4. **Coordinates are stored in image pixels, normalised only at export.** Because folders contain arbitrarily-sized images, there is no fixed canvas to normalise against at label time; raw pixels keep edits exact and display-independent. 5. **Empty images are never stored, and there is no "mark empty/negative" action.** Saving is purely automatic on edit and on navigation; if you pass an image without drawing, nothing is written, and clearing an image's annotations deletes its record. This keeps "labelled N" honest and removes a whole class of "did I mark this?" friction. (Note: this means the app does not, in v1, capture confirmed-negative images as explicit training negatives — a conscious simplification. A rebuild that needs negatives should add an explicit, separate mechanism rather than reusing "empty".) 6. **Edit is its own mode; drawing tools never grab existing geometry.** The app went through a modeless "direct manipulation" phase (press an object to move it, press empty to draw) and it caused conflicts — you couldn't start a shape on top of an object, and tag/draw/move behaviours blurred. The resolution: a dedicated **edit** tool is the *only* place vertices/objects move or get reclassified; `polygon`/`box`/`point` purely draw (over anything); `tag` is inert on the canvas. This clean separation is intentional; do not re-merge them. (A consequence: box corner reshaping happens in edit mode, not in the box draw tool.) 7. **Reclassify lives in edit mode, not in "tag".** Users intuitively expect "change this object's class" to be a thing; the natural home is edit (select an object, then pick a class). "Tag" is strictly whole-image classification and was a common point of confusion — hence the explicit help text distinguishing them. 8. **Durable, journal-first writes with atomic replace and periodic snapshots.** This exists because a naive in-place overwrite once destroyed real labelling work in a predecessor. The journal is append-only and never rewritten, so `labels.json` is always reconstructible. This is non-negotiable for a tool where the labels represent hours of human effort. 9. **Native OS folder dialog via a subprocess, with a headless fallback.** The browser genuinely cannot hand back a real path; the local server can drive a native dialog. Running the GUI toolkit in a subprocess sidesteps its main-thread requirement inside the threaded HTTP server. The in-app browser exists only for headless environments. This two-path design is deliberate. 10. **No framework, inline static client.** Chosen for zero build step and zero dependencies, matching the host environment. This is a free choice, called out so a rebuilder doesn't assume the plain structure was an oversight. 11. **Validation split is a fixed `k % 7 == 0 → val` rule (first image is val).** Simple, deterministic, ~15% holdout, no UI. The exact "first image goes to val" behaviour is an artifact of the modulo rule; it is harmless and reproducible. A rebuild may make the ratio configurable but should note if it diverges from this default. 12. **Single open project as a server global.** No session or multi-project state because it is a single-user local tool. Simplicity over generality, on purpose. 13. **`bbox` internally, "box" in the UI.** The user-facing label was renamed to "box" for friendliness while the stored `type` stays `bbox` to avoid migrating data and export code. Display/label strings are cosmetic; the wire/stored value is `bbox`. 14. **Reserved but unbuilt: model pre-labelling.** The architecture intentionally leaves room for a future "predictions" source (a model proposes annotations the user accepts/edits), mirroring a predecessor tool's suggestion flow. v1 does not implement it; a rebuild need not, but should not preclude it. --- ## Appendix A — Rebuild acceptance checklist A rebuild is functionally equivalent if: - [ ] Opens any local image folder; creates/loads a portable sidecar beside the images; non-recursive scan of the supported extensions, sorted by filename. - [ ] Classes are fully UI-managed (add/edit/recolour/rebind/delete), stored as one ordered registry that drives both drawing and export; export ids = array order. - [ ] Four annotation types (polygon ≥3 pts, box as min/max corners, point, tag) stored in integer image-pixel coordinates keyed by image filename. - [ ] Five tools with the exact semantics: polygon/box/point draw (over anything), tag is inert + toggles whole-image class on class-pick, edit is the only mode that moves vertices/objects and reclassifies a selected object. - [ ] Autosave on every edit and on navigation; empty images never stored; emptied images' records deleted. - [ ] Durable storage: append-only journal first, atomic replace of `labels.json`, periodic snapshot; reconstructable from the journal. - [ ] Navigation: prev/next, nearest-unlabelled both directions, first/last, jump, resume-to-first-unlabelled — all autosaving. - [ ] Export to YOLO seg, YOLO detect, and COCO with the exact formats and the `k%7==0→val` split; download as zip; skips empty/missing/unreadable images. - [ ] Loopback-only server; path-traversal-safe image serving; no auth by design. - [ ] Native folder-path selection with a headless in-app fallback.