What the House Built for Me
The first note in this series described how the house is organized. This one is the inventory: what its members have built for me, grouped by what each part does for me. Everything here runs on my own machines and is in daily use. Each group says first what the service is and what it is for, then lists its features. The sixteen features with three figures under them are the ones where the work is in the pipeline, the model, the measurement and the guardrails rather than in the screen; “How it works” under each one opens the detail.
Core15 itemsTop
The house itself: the platform its members run on, and what keeps their work tested, paced, on schedule and under my control. Everything else on this page is built and runs on it.
- The platform
- Every member is a long-running session on a machine in my house, reached through Slack. The platform starts the sessions, carries messages between them and Slack, keeps each member’s memory, and restarts every member with a clean slate at four in the morning, after asking it to save what matters.
- Message delivery
- A message is typed into a member’s session and the screen is read back to confirm it landed; the answer is captured when the member finishes, never scraped off the screen. A separate check tells a session that is working from one that is stuck by how much processor time it is using.
- One control surface
- A single command shows what is running (the daemon, each member’s session, every background watcher) and controls it: start, stop, pause delivery, recycle one member. The same controls are in an interactive menu, so nothing important needs a typed command.
- Release gates
- Nothing deploys without a verified stamp, a change to a protected part of the platform needs my sign-off as well, and the running system cannot be edited in place, even by the engineer who wrote it. The stamp and the kept tests are described next.
- Nothing ships unproven
Proves every change before it ships.
- 25,000+ tests run before every release
- 142 kept tests, each proven to catch its bug
- 109 fixes they keep from coming back
How it works
On the code running today there are 935 test files and more than 25,000 tests; a change is stamped verified only when all of them pass, and a new test must also fail on the code from before the change, or it proves nothing. The production copy is owned by a separate account whose gatekeeper re-checks the stamp, and 142 kept tests, each shown to pass on its fix and fail on the code before it, keep 109 fixed bugs from coming back at every release.
- A workbench
- The real message pipeline runs against an emulated Slack workspace, with faults, restarts and hard kills injected on purpose, so a change to how messages are delivered or interrupted is proven in a scripted scenario before it reaches me.
- A governor for the week’s capacity
Paces the house’s work against a fixed weekly allowance.
- 4 pacing zones
- 13 past weeks replayed to test it
- 99.64% of each week's capacity put to use
How it works
The house runs on a fixed weekly allowance of model capacity with a five-hour window inside it; the governor projects where each window will land at its reset and sets one of four zones from the tighter, which decides the model background work uses, how many workers run and how fast the queue drains, and it can hold part of the week back for my own conversations. Whatever the week will not use goes to the backlog: replayed over 13 past weeks, the average week’s end went from 98.78 to 99.64 percent used.
- Self-written rules
- Each member keeps rules it wrote from its own mistakes and my corrections, with the date and the reason; they are part of its instructions every time it works. The first note in this series describes how.
- A self-improvement loop
- A model reads my recent messages for complaints about a member’s work and turns each into a candidate fix. Before every pass it is scored on a fixed set of my real lines, hand-labelled, and the pass is marked degraded when it catches fewer than 70 percent of the real complaints.
- The queue and its dispatcher
- Requests for the platform go on a shared board with their severity and urgency. When there is spare capacity, a dispatcher picks the next ready item and has it built on its own branch; the filer is told when the work is merged and again when it is live.
- Watchers
- Background watchers sample the machines (a process holding a core, a disk nearly full, a backup gone stale), and a reading becomes a finding only after it holds for several samples in a row. Others check whether a newer version of any model or engine the house runs is out, move the members onto a new version of their command-line tool between turns, and tell me when the house’s sign-in lapses; that sign-in can be renewed from my phone, in Slack.
- Config kept in step
- The members’ instructions live in version control and on the running machine at once; a watcher commits an edit made on the machine once it has held for an hour, and refuses to overwrite an edit made on both sides.
- Encrypted memory
- A member whose memory is its work can have it packed and encrypted before it is committed, so the history of the repository is the history of that memory, unreadable without the key.
- Follow-ups
- A member’s dated commitments live on a schedule, not in its memory: the house wakes the member when a follow-up is due, so a promise does not depend on that day being quiet.
- Background workers
- A member can hand a long job to a detached copy of itself and keep answering me; it is woken with the result.
Media6 itemsTop
My library of personal videos and public-domain films. It lives on an internal website the house built and runs for my own use, reachable only on my private network and installed on my phone like an app. The house prepares every film so I can watch it with good captions and go straight to the parts I want.
- The player
- A player that streams the library to my phone. Every film arrives with locally made captions, a track of sound captions, bookmarks and a rating of what each part of it is about, all described below; subtitles burned into the picture are found by on-device text recognition so they can be hidden under mine.
- Captions made at home
Subtitles and sound captions, made on my own machine.
- 9.4 s to caption a minute of film
- 303 kinds of sound it can caption
- 30 clips checked blind, by ear
How it works
An open-source speech model (Whisper large-v3) on the second machine’s GPU writes the subtitles, about 9.4 seconds per minute of film, after a separate voice-activity model has cut out the speech, which keeps a long film’s subtitles in sync. A second track captions sounds with the classifier built into the operating system (303 kinds of sound), only in gaps in the dialogue, and a blind check by ear on 30 clips is what tunes it.
- A bookmark agent that watches every film
Watches every film and marks the moments worth going back to.
- 187 frames read from a 93-minute film
- 30 s between frames on the first pass
- 5 s between frames on the close-up pass
How it works
The house cuts each film into contact sheets, a frame every 30 seconds, and a Claude model reads every sheet, lists the moments worth marking, then asks for a close-up sheet around each one, a frame every 5 seconds, and marks the first frame where the moment shows. The tool prints the measured time of every frame, so each mark lands on the second the moment begins: a 93-minute film becomes 187 frames in 8.4 seconds.
- Scene ratings, and a model of my taste
Grades what each part of a film is about, and predicts how much I will like it.
- 0.91 stars off my own rating, on average
- 1.07 stars off if it just guessed my average
- 85 of my rated films it was tested on
How it works
Every film is graded on a fixed set of dimensions (say tension, action, humour and dialogue), on five levels from none to extreme, as a whole and for every 30-second stretch, from the frames and the captions rather than from a plot summary. A ridge regression fitted to my star ratings turns the grades into a prediction of how much I will like a film; tested by hiding each rated film in turn, it misses my own rating by 0.91 stars on average over 85 films, against 1.07 if it just guessed my average.
- Remastering
- An open-source video upscaling model, ported to the second machine’s graphics chip, sharpens an old film frame by frame; the port took it from 7.5 seconds a frame to about 2.5.
- My own videos
- A video link I send is downloaded into my media library with its full details, private when I say so.
Venues6 itemsTop
A way for me to find places when I travel or go somewhere: what is around me, a trip planned around one area, how long the drive takes and where to leave the car. It is part of the same internal website.
- Near me, search and the map
- What is near me right now, search, and a map with my current position snapped to the road I am on.
- Bases
- The places I am staying are bases that every distance is measured from.
- Trip planning
- A planner finds groups of places where every pair is within a chosen distance, so a trip can be built around one area; a faster search took it from 19.4 seconds to 0.86 with the same answer.
- Research agents
- Agents propose new places; each proposal is checked against a real listing to prove it exists, matched against what the site already has, and added only after I review it.
- Driving times without a maps service
Tells me how long a drive takes without asking a maps service.
- 390,714 road junctions in its own map of the state
- 0.13 s to route a 30-mile drive, at most
- 33 / 45 test trips within 3 minutes of a commercial router
How it works
The house routes on its own road graph built from OpenStreetMap, 390,714 junctions for the whole state, with an A* search that answers a 30-mile route in at most 0.13 seconds of computing and calls nothing outside. Its speeds were fitted to a one-off sample of trips from a commercial router and scored only on trips the fit never saw: 33 of 45 land within 3 minutes of the commercial router’s time.
- A safe place to leave the car
Finds a safe place to leave the car near where I am going.
- 45,152 parking lots in its map
- 27 / 30 lots the map was missing, found in satellite images
- 2–4 spots it proposes for each visit
How it works
The finder starts from 45,152 parking lots from OpenStreetMap, each tied to the business within 60 metres whose lot it is, and where the map is thin a model reads satellite imagery: on 30 known spots the map had missed, it found 27. Every candidate gets a real walking route, then an agent looks at each from above, checks its opening hours for the time of the visit and proposes 2–4 spots; nothing is saved until I accept it.
Apps11 itemsTop
The rest of the internal website is apps, each built for one job in my life: my home’s energy, my drinks, the talk show I publish, and a dashboard of the house’s own capacity. New ideas start here as small experiments.
Energy2 items
- Use and forecast
- My home’s daily and hourly gas and electricity use with the weather beside it, a forecast of the days ahead, and the experiments described next.
- Energy, imported and tested
Reads my gas and power meters and tells me what a change I made actually did.
- 4 reads a day from the utility
- 7 + 7 days compared, before and after a change
- 81% of daily gas use explained by the weather
How it works
The utility offers no public API, so the house signs in to the usage service behind its website and reads both meters hourly and daily, four times a day, with about three years of history, reading the last week again because the utility revises days. An experiment is a marker at the moment I changed something: each clock hour of the 7 days before is compared with the 7 days after (the same weekdays), the weather’s share is taken out with a heating-and-cooling degree-day model of the house, and any other change inside either week is named instead of mixed in. With the big burner’s days set aside, the weather explains 81 percent of the day-to-day gas use.
Drinks2 items
- Recipes and an estimate
- My favourite drink recipes, and an estimate of my blood alcohol after each drink on the list, from the model described next.
- A drinks model fitted to my breathalyzer
Predicts my breath alcohol from what I drink, fitted to my own readings.
- 20 min for each drink to be absorbed
- 2 breathalyzers it learns from
- 0.005 its worst miss against my breathalyzer
How it works
Every drink is a dose absorbed over 20 minutes, more slowly after a meal, against one steady burn rate. The doses and the burn are fitted by weighted least squares to my own readings from 2 breathalyzers rather than a textbook formula, refit with every reading I log, and against my breathalyzer it misses by 0.005 at worst, inside the meter’s own error.
Podcast Publisher5 items
The LuciTales Show, my animated talk show: a script goes in, and a finished episode comes out, with its voices, its animation and its publishing.
- Conversations in several voices
- A script written like a screenplay is read one line per voice, each reading transcribed to check its words, then laid on a timeline with the gaps, overlaps, cut-offs and breaths of real conversation and mixed in one room. Each voice is checked against itself, described next.
- Voices that stay themselves
Keeps every synthesized voice sounding like itself, line after line.
- 2.5 semitones of pitch drift, and the line is redone
- 1.5× faster or slower, and the line is redone
- 5 tries at most for one line
How it works
After every line is read, a guard measures it against the same speaker’s other lines: pitch (more than 2.5 semitones off fails), pace (1.5× faster or slower fails) and timbre, the distance between speaker fingerprints (more than 0.10 past that speaker’s usual distance fails). A failed line is read again with a new seed, up to five times, a new reading never replaces a better one, and strong emotion in the script widens every limit.
- An animated talk show
- The conversation becomes a limited-animation cartoon on a set, with vertical cuts for a phone; the next item describes the mouths, the camera and the review. It has its own website, lucitales.com, which the house also built.
- A talk show from a script
Turns a script into an animated show, mouths on the words.
- 14 mouth shapes
- 24 frames a second
- 1 frame, the most any mouth runs late
How it works
A forced aligner times every letter, a pronouncing dictionary turns letters into sounds, and each frame gets one of 14 mouth shapes at 24 frames a second, one frame ahead of the sound; measured, the worst mouth is one frame late and every lip closure lands. The camera cuts by rules (a long line to a close-up, an interruption to whoever cuts in, no shot under 0.75 seconds), the gaps between speakers are drawn from the timing of real conversation, a fifth of them overlapping, and a review pass redraws each finished cut one frame a second and lists every defect, from a face cut by the edge to a caption over a mouth.
- Publishing
- A finished video goes to YouTube, Instagram or TikTok from one tool. Every upload is private unless I approved otherwise, the tool refuses to make anything public without a pointer to where I said so, and each video is checked against the platform’s limits before anything is sent.
The dashboard and new apps2 items
- The Claude dashboard
- The home page is a live dashboard of the house’s model capacity: how much of the week’s allowance is used, where it is projected to land, how fast it is being spent right now against the budget’s pace, and the current five-hour window. It refreshes every minute from the governor’s own figures.
- Experimental apps
- New ideas start as small apps inside the site, each kept apart from the rest, and a check fails any app that would block the server, so one broken app cannot take the site down. An app graduates when it earns it.
Studio16 itemsTop
Where the house makes things for me: songs, videos, voices, pictures and 3D prints. The heavy models run on a second machine in the house, so nothing is billed per song or per picture.
- Songs
- A member writes a style and lyrics and the house makes the song with an open-source music model on the second machine, then posts it in Slack to play inline. A take that goes silent before its end is caught by its level and made again with the next seed, up to three takes.
- Spatial mixes
- A song taken apart into its parts, each part placed and moved in 3D around the listener’s head for headphones, from 368 directions measured on a dummy head, with its own pitch, filters and echo; a two-minute song with twelve moving voices renders in about 15 seconds.
- Music videos
- A song becomes a video cut on its own beats: each cut lands on the last beat before its line starts, and every frame is drawn with sub-pixel moves so a slow pan does not shimmer.
- Lyric videos
- The words in moving 3D type, rendered in Blender, each line’s energy read from its section and its loudness. The timing is described next.
- Lyrics timed on the singer’s voice
Puts every sung word on screen as it is sung.
- 36 ms typical gap between a word and its sound
- 92% of words within a fifth of a second
- 194 words checked against the voice itself
How it works
The song is split into its parts and the words are force-aligned onto the isolated vocal one sung line at a time, so one bad line cannot drag the rest, and a word the aligner is unsure of keeps the transcription’s time. Checked against the vocal’s own onsets on 194 words that start with a hard consonant, the median word is 36 milliseconds off and 92 percent are within a fifth of a second.
- Light shows
- Glowing tubes that form pictures, morph on the beat and pulse with the music, with the song’s words in a band underneath; the tracing and the morphs are described next.
- Light shows drawn in neon tubes
Draws pictures in glowing tubes that change into each other on the beat.
- 64 glowing strands in every picture
- 24 points along each strand
- 5 levels of detail, from 16 to 96 strands
How it works
A picture is traced into lines (a threshold, centrelines, a graph of the junctions, short spurs pruned), drawn at 5 levels of detail, from 16 to 96 strands, and laid on a shared pool of 64 strands of 24 points each. One picture morphs into the next by pairing strands with the Hungarian method, and every point is keyed frame by frame on the song’s own beat clock.
- Road-trip videos
- A camera walks a real street on real map data with the buildings at their heights; it finds the moment the view opens up by casting rays through a 1-metre grid of the city and times that reveal to the song’s drop.
- The infinite zoom
- A chain of pictures, each painted around the one before it, flown through on a smooth banking path that cuts at the first depth where the next opening fills the screen, every picture upscaled four times.
- Voices made from a description
Makes a new voice from a few words describing it.
- 3 models working together on every line
- 0.63 how alike its own first and twentieth lines score (two different people: 0.21 at most)
- 45 s to read a minute of story
How it works
I describe a voice in words; one model reads a reference line in that voice, a second takes a speaker print from it, and a third reads every line of a story with that print and the line’s own direction (“hushed, with awe”), so the person stays the same while the acting follows the story. A separate speaker-recognition model scores the first and the twentieth line of one story 0.63 alike, against 0.06 to 0.21 between two different speakers, and a minute of narration takes 45 seconds to read.
- A voice cloned from a recording
Copies a voice from any recording, and warns me when the room will spoil it.
- 10–20 s of speech is all it needs
- −30 dB of room echo, where it warns me
- 0.67 match to the real person's voice (0.50 when acted)
How it works
From any recording the house keeps the best 10 to 20 seconds: 20-millisecond frames are sorted into speech and silence, long pauses are shortened, and the stretch with the most speech and the least clipping wins, taken from the driest part when the recording’s room changes. The room is measured in the pauses after words, because a clone carries its echo: close-mic podcasts read -41 and -47 dB, speeches in halls -23 to -27, and past -30 the tool warns. Cloned “close”, every line is read from the clip itself instead of acted, and it scores 0.67 against a held-out recording of the same speaker, against 0.50 acted.
- Story readings
- A story read aloud in a voice I choose or one made for it, each line moving on screen the way it reads. A line the voice says wrong is respelled for the voice while the screen keeps the line as written; the voices are the two items above.
- Spoken answers
- A member’s answer can be read aloud in a voice of its own, so no two members sound alike, in one of ten tones (warm, wry, urgent and so on) carried either by the voice’s own speaking style or by measured shifts in pace and pitch, with effects and a room mixed after it to one loudness. It is off until I turn it on for a channel, because some voices are synthesized by a hosted service.
- Images
- Members render images on the second machine, choosing the model that fits the job. The tool refuses a combination of model and adapter that would silently drop the adapter, before it wastes a render.
- 3D objects
- Props for the show’s set are made as 3D models and kept for reuse.
- 3D prints of real places
Turns a real place into a model I can print, buildings at their real heights.
- 6 printable parts, one per colour of filament
- 1 m laser scans the roofs are fitted to
- 0.4 mm the finest detail it keeps
How it works
The ground at its real elevation, the streets, the water and the buildings at their measured heights, with roofs fitted as flat, gable, hip or pyramid planes to 1-metre laser scans where a survey has them. The model is six watertight parts, each its own filament, every solid is checked the way the printer’s software will load it, and nothing narrower than 0.4 mm is kept.
Archive4 itemsTop
One archive of the notes and documents I want to keep for good, searchable, on my devices, and copied outside the house.
- The archive
- The notes and documents I want to keep for good live in one archive, in plain Markdown, synced to my devices and indexed for full-text search. One member, the keeper of the archives, is its only writer, so nothing in it ever conflicts. Every file a write touches is logged first, with the content it leaves and the content it replaced.
- Importing old notes
- My old notes came in from an older notes app, converted to Markdown, and the scanned papers among them were run through text recognition on my own machine, reading a document’s own text first and the picture only when there is none, so they can be searched.
- Backups
- The archive has three copies of its own: hourly to a storage server in the house, and nightly to two cloud drives, encrypted with file and folder names included. Every run keeps the files it changed or deleted as they were before, so a sync never loses anything.
- Mirrors
- A member’s working file can be mirrored into the archive one way: each time the member changes its file, the archive’s copy follows.
Working for me18 itemsTop
The members also run the everyday business of my life: claims, bills, vendors, email, the alerts that need me and the research behind a purchase, each job with a tool built for it.
- Insurance and damage claims
- When something of mine is damaged, a member runs the claim from start to finish: the estimates, the filing with the right company, every email on it labelled, the adjuster chased on a schedule, the payment tracked until it actually arrives, and the legal deadline for the claim kept on the calendar so it can never quietly lapse.
- Deadlines and bills
- Every due date that reaches my inbox, a statement, a renewal, a filing, becomes a dated follow-up: checked off when it is done, and raised with me once when it is not.
- Vendors and contractors
- Finding and vetting the people who work on the house, getting itemized quotes, comparing them line by line and negotiating, in messages drafted for me to send.
- Buying a car
- The research, the comparison of offers and the negotiation, worked through with me rather than for me.
- Email and calendar
- My inbox is sorted, labelled and summarized, and the replies it needs are drafted. Nothing is ever sent in my name: every message to someone outside the house is a draft I send myself.
- Only what needs me
- Every new text on my phone is read as it arrives and every email is sorted as it lands; a sender that matters wakes a member within a minute, at any hour, and everything else waits for one daily digest.
- Security alerts
- Every sign-in, new-device or code alert is checked against what I was actually doing at that minute, and I hear about it only when it was not me.
- Money alerts
- Card charges, payments and statements are read as they arrive; a payment I set up myself is recognized and set aside, and a changed amount or a quiet price increase on a subscription is raised.
- My company’s paperwork
- The state filings, the annual report and the registered agent are tracked, and each confirmation is checked against what was filed.
- Unsubscribing
- When I am done with a sender, a member unsubscribes me through the sender’s own link rather than just muting it.
- Notes on the people in my life
- A shared profile of each person I deal with, kept inside the house, so every draft fits the person it is going to.
- Spam, put to use
- The worst spam texts go to the talk show to be roasted, with every name and number stripped first.
- The house itself
- A daily look at the week’s weather for what it means at home, a freeze, a storm, high wind, and the utility bills brought into the energy view.
- A status board
- One live list of every open item and every job in flight, checked against its sources each day so it never shows something already done.
- Reading what the web tools cannot
- A tool reads a discussion thread from a site that refuses automated reading, through a chain of fallbacks, and says on its first line which one answered and whether what it has is incomplete.
- Checking before recommending
- Before a member quotes a product’s price or rating, a tool reads the listing as it stands that day, and says plainly when a store would not let it look.
- Voice memos
- A recording I share is transcribed locally, on my own machine, so nothing leaves the house.
- Websites
- The house builds and runs websites, including this one and the talk show’s, with a private preview for me to review before anything goes public.
Playground2 itemsTop
A place to try new members, define their personality and interact with them, and also to simulate meetings with people or social situations.
- Trying new members
- A new member is tried here first: I define its personality and talk with it to see how it comes across.
- Simulated meetings
- Members play the other people in a meeting or a social situation, so I can run through it with them.
None of these were planned as a product. Each one started as a problem I had or a job someone was doing by hand for the third time, and was built, tested and shipped the way the first note describes.