A Database With Good Lighting

Where this began

On July 9, 2026 at 3:37 in the morning, an empty folder, an OpenSea API key in a text file, and one prompt.

"Help plan out database collection and download of my NFT art vault. I will give you my opensea link. Lets create a database of the images (this will be hard many different file types, inspect ways to find original file, the address associated with like mints transfer, whatever etc." The opening message, verbatim, typos and all. The only thing in the folder was the key.

I loved art long before I loved crypto. I bought my first Bitcoin at eighteen in 2013, was around for the first NFTs in 2017 and walked away because I did not like the pictures, and it took until 2025 for that to change. When it changed, it changed hard: a game, a guild, then one artist's project on an AI art app, then a thousand works in a year. The picture that did it is the one below.

Here is the part nobody warns you about, on either side of the transaction. An NFT does not store the art. The token is a permanent line on a blockchain. The picture it points to is a guest on IPFS, or Arweave, or an artist's web host, or a marketplace's cache, and any of those can go dark when the rent stops. When it does, the token is still yours. It just points at nothing. The artist's work is what actually disappears.

This was not a thought experiment. When the pipeline first pulled the original file for every piece I owned, twelve were already gone, the hosts dead. For nine of them, a marketplace's cached thumbnail was the only copy left anywhere. I found that inside my own collection, the one I thought was safe.

What is in there matters, because this is not a spreadsheet of tokens. 1,934 stills, 1,194 motion pieces, 296 films, and 293 works of live code that run in the browser, by 839 artists on every continent (the four who chose anonymity are filed under Antarctica). A year of Justin Aversano's daily polaroid auctions. Work from 112 of the 500 flagship Art Blocks projects. And the artists' own words: 1,118 lines recovered from public conversation, each with a permalink, sitting beside the works they were about. The database exists to keep all of that intact after the marketplaces move on.

How this was built: eight weeks, sixty-three versions

Version one shipped in a day. Then the conversation kept going. Every version below passed the same gate before it went live: zero audit failures, zero leaks of the private wallet.

TL;DR

The data model

A Python package with one command per stage, each idempotent and resumable, all writing into one SQLite file. The hard part is not storage. It is answering "who made this?" without a column that says so.

The attribution chain

A token row has no artist column. That is the single most important fact about NFT data, and it explains half the wrong attributions you see on marketplaces. The token belongs to a contract; the contract maps to a collection; the collection has an owner address; the owner address is the artist. Except when it is not, which is often.

CREATE VIEW v_main AS
SELECT n.id, n.name AS title, n.contract_address, n.token_id, n.chain,
       COALESCE(n.collection_slug, c.opensea_slug) AS collection,   -- per-token beats per-contract
       col.owner_address                            AS artist_key,   -- the artist, usually
       n.minted_at, n.collected_at, n.collected_via, n.acquire_market
FROM nfts n
LEFT JOIN contracts   c   ON c.address = n.contract_address
LEFT JOIN collections col ON col.slug  = COALESCE(n.collection_slug, c.opensea_slug);
-- then: curation/overrides.json wins over every row above

• One contract, many artists. Art Blocks and fx(hash) host hundreds of projects under a handful of shared contracts. Each token's own metadata names its project, so the per-token collection field is what keeps a generative drop together and credited to the right person.
• Platform-owned collections. SuperRare-style contracts where the "owner" is the platform. Attribution falls to the minter, or to an "Artist" trait, or to parsing "by NAME" out of the collection title, depending on the venue. Each mode is a named override.
• Tezos. Attribution always follows the per-token creator recorded by the indexer, never the contract, because per-artist contracts there often have no owner at all. Learning that healed about 114 artist rooms in one fix.
• Humans. A curation layer of JSON overrides (merges, collaborations, platform accounts that are not artists, works to hide, captions that are the artist's recovered words) wins over everything above. It is the only place taste is allowed to override the chain.

The canonical run, in order

py -m vault sync                 # inventory the wallet (captures per-token collection)
py -m vault metadata --all       # each token's own metadata document
py -m vault artists --all        # collections + artist accounts
py -m vault provenance --all     # events from the marketplace index + the chain ledger
py -m vault enrich               # derive collected_at / collected_from / sales / classes
py -m vault origin               # mint + acquire venue   (MUST run after enrich)
py -m vault market               # receipt forensics: which marketplace really settled it
py -m vault editions             # true per-token edition size from the contract
py -m vault bids --platform auction   # reconstruct the bidding from calldata
py -m vault download --all       # originals, gateway rotation, sha-256, magic bytes
py -m vault site --base-url https://along7gallery.xyz
py -m vault factcheck            # re-verify chain facts into a ledger
py -m vault audit                # ~100 checks; 0 FAIL required to ship
py -m vault deploy               # rclone sync of site/ only, after the privacy grep

Order matters and the order was learned the hard way. origin labels a work "from the artist" only when the collector received it directly from the minter, and enrich is what fills in who the collector received it from. Run them backwards and a whole batch of gifts quietly become "unknown."

Getting the real files

The lazy way is to save whatever image the marketplace shows. That is wrong twice: it is a re-rendered copy, and it can differ from, or outlast, the artist's file.

for nft in wallet_nfts(address):
    meta = fetch(nft.token_uri)                       # the token's own record
    url  = meta.get("animation_url") or meta["image"] # the artist's declared source
    data = fetch_with_gateway_rotation(url)           # ipfs:// -> several gateways, politely
    ext  = sniff_type(data[:64])                      # trust the bytes, not the header
    save(f"{contract}/{token}.{ext}", data, sha256(data))

def sniff_type(head: bytes) -> str:
    if head[:3] == b"\xff\xd8\xff":                return "jpg"
    if head[:8] == b"\x89PNG\r\n\x1a\n":           return "png"
    if head[:6] in (b"GIF87a", b"GIF89a"):         return "gif"
    if head[:4] == b"RIFF" and head[8:12] == b"WEBP": return "webp"
    if head[4:8] == b"ftyp":                       # ISO-BMFF: video OR an AVIF still
        return "avif" if head[8:12] in (b"avif", b"heic") else "mp4"
    return puremagic_guess(head)                   # everything else, last

The 403 wall

The Tezos ingest was the first truly bulk download, about 2,100 files. At a quarter-second between requests, the public IPFS gateways throttled the IP after roughly 270 files: uniform 403s from three gateways and 429s from a fourth, on every row, including rows that had worked minutes earlier. A naive retry loop would have marked live art as permanently failed at the three-attempt cap.

Reading the chain twice

A marketplace can relist a piece, hide it, or lose its history. The chain cannot. So provenance is replayed from the events the ledger wrote, not the story a website tells today.

Every work's transfer history is read from two independent sources, the marketplace's event index and the chain explorer's account ledger, and merged. A mint is simply the first transfer, sent from the zero address. Purchases go one layer deeper: the transaction receipt and its logs name the marketplace contract that actually settled the sale, which is how a wall label can say where a piece was minted and where it was collected and mean both.

-- ERC-721 / FA2 one-of-ones: the last hop must land in a tracked wallet
SELECT n.id FROM nfts n
JOIN events e ON e.contract = n.contract_address AND e.token_id = n.token_id
WHERE e.ts = (SELECT MAX(ts) FROM events WHERE contract = e.contract AND token_id = e.token_id)
  AND lower(e.to_address) NOT IN (SELECT address FROM wallets);          -- these have LEFT

-- ERC-1155 editions: net balance across the tracked wallets must stay above zero
SELECT contract, token_id,
       SUM(CASE WHEN lower(to_address)   IN (SELECT address FROM wallets) THEN quantity ELSE 0 END)
     - SUM(CASE WHEN lower(from_address) IN (SELECT address FROM wallets) THEN quantity ELSE 0 END) AS held
FROM events GROUP BY contract, token_id HAVING held <= 0;                 -- these have LEFT

The truth machine

A museum should be able to prove what it puts on the wall. So a program checks every build before it ships, and a failed check blocks the release.

The audit started on day one with 21 checks and now runs 99, in numbered families. A sample of what they assert, each one born from something that actually went wrong:

def a3_venue_proofs(conn, check):
    proven = {r[0] for r in conn.execute(
        "SELECT lower(address) FROM address_book WHERE kind IN ('marketplace','platform')")}
    bad = [(r["title"], r["mint_venue"]) for r in conn.execute(
        "SELECT title, mint_venue, mint_venue_addr FROM v_main WHERE mint_venue IS NOT NULL")
        if (r["mint_venue_addr"] or "").lower() not in proven]
    check(not bad, "A3-venue-proofs",
          "every venue label traces to a proven contract address",
          f"{len(bad)} unproven venue labels: {bad[:5]}")

# ...99 of these; one FAIL and `py -m vault deploy` refuses to run

After a deploy, the checks run again against the live pages, because green on a laptop and green on the open internet are two different facts. The CDN caches the bare asset URL for hours and a plain fetch can hit an edge that has the fix when the origin does not; the only proof that counts is a cache-busted fetch of the versioned asset the live HTML actually references.

Bugs worth keeping

Each of these is now a gotcha in the project's working notes, so it never has to be relearned. The chip says what caught it.

Two hands on the keyboard

One human directing. Three Claude models across eight weeks. The AI did the engineering; the human made every call taste decides and did the seeing.

What the AI could do

The honest headline is speed at scale. A working archive, database, and public gallery existed by the end of the first day. Over the following weeks the same conversation learned to read three blockchains' APIs and one explorer's quirks, reconstruct auctions from raw calldata, transcode three hundred films for a television, fold duplicate artist identities across chains by handle and website and name, and render a 5,925-page site in about a hundred seconds. It built and audited a complete Spanish translation of the whole gallery, thirty thousand words, in one day. And eight weeks in, a fresh model found a bug that had been in the chain client since the first hour, because it read the code the way a reviewer would rather than the way an author does.

What only a human could do

None of the checks know what art is. The collector knew, from a thumbnail, that a "video" was a still. He knew which artist actually made a piece filed under a shared contract, which acquisition was a private deal the receipt forensics had mislabeled as a marketplace buy, which wallet must never appear on the site, and which of two identically named accounts was the real person. He read a room of 34 works and saw that 30 belonged to other people. He decided there would be no prices on the walls, that the artists' words belong next to the art, and that the whole thing should sound like a person and not a product. Every one of those decisions is now encoded in a curation file or a check, so the machine can hold the line he drew.

Privacy by design

Open and private only fight if you never draw the line. The collection is public art, so the art is public, all of it, and the facts about it are published as a CC0 dataset. But:

• Quiet wallets stay quiet. The pipeline knows which addresses are the collector's private ones, treats their activity as internal plumbing, suppresses them as minters, skips their bids, and a grep for the address across the built site must return zero hits before any deploy.
• No prices, anywhere. Sales history lives in the database because it is provenance. The walls show venues, dates, and counts. A check fails the build on any amount, including inside story prose, with a two-item allowlist for a memoir's public numbers.
• Only the built site leaves the machine. The database, the curation layer, the notes, the collector's own art files, and the API keys never ship. The dataset builder can only emit fields already public on the pages, so a leak would have to be visible on a wall first.
• Artist code cannot touch the gallery. 293 interactive HTML works and 28 SVGs ship on the apex origin, because hosting the real interactive art is the point. They run in a sandboxed iframe without same-origin, and the edge serves a sandbox CSP on the media path, so even direct navigation loads them in an opaque origin.

Limits, stated plainly

• Some bids are invisible by design. Foundation packs its auction state into events that need the verified ABI to decode; Ninfa settles via a generic signed-order call with off-chain bids. The site counts those as auctions won and does not invent bids for them.
• Tezos venue proofs are labels, not addresses. Acquisition venues on Tezos come from the indexer's sale records and a marketplace-contract map; the proof column stays empty and the audit exempts it on purpose.
• "Minter" is a blend. For most works it is the mint transaction's sender; for auction-house mints it is the first real owner. The fact-check treats a minter mismatch as a warning and only a block-timestamp disagreement as a failure.
• 127 works are cache copies. Their original hosts are dead. Nothing can bring the artist's file back; the label says so.
• Layer-2 editions may stay unknown. The free explorer tier serves Ethereum log history but blocks it on Base and Polygon, so some edition sizes there are honestly null.
• Curation debt is human debt. 17 artist bios are still stubs, 11 works are unattributed in a declared section, and one duplicate profile is waiting on a merge. The audit shows these as warnings, not failures, because they are decisions, not defects.
• The ledger goes stale between runs. The chain fact-checks carry a date; a check older than 30 days warns. The custody claim above is true as of the date on it, not forever.
• The pipeline is private. This page documents the method and the numbers; the production code, the curation layer, and the database stay on one machine. What ships publicly is the gallery and its CC0 dataset of facts.

Reproduce the idea

for nft in wallet_nfts(address):                      # 1. inventory: count before you save
    meta = fetch(nft.token_uri)                       # 2. the token's OWN metadata document
    url  = meta.get("animation_url") or meta["image"] # 3. the artist's declared source, never a preview
    data = fetch_with_gateway_rotation(url)           # 4. ipfs:// -> several gateways, politely
    ext  = sniff_type(data[:64])                      # 5. trust the bytes, not the header
    save(f"{contract}/{token}.{ext}", data, sha256(data))
    events = replay_transfers(contract, token)        # 6. provenance from the chain, not the site
audit(site)                                           # 7. nothing ships with a failing check

Some downloads will fail because the original host is already gone. That is not a bug; it is the entire reason to do this.

The fifteen

The banner at the top of this page, cell by cell, left to right and top to bottom. Each artist sent the file at full resolution; each tile links to the artist's room in the gallery.

Creation · Gül Yıldız deluge · ex_mortal Forget Me Not · Lorenipsum the grid · DeltaSauce mountain dew daze · Desultor March 21st · Justin Aversano Rise of the Immortals · 1dontknows Impermanent Embrace · Adamtastic Napkin Woman · josenarciso Rijksmuseum 13 · Rebecca Rose Beverly Hills · SigmaX Untitled · BoredJosei Coming Home To Finite · Mizuyokaii Coronal Revelations · intrepid I'll ask ChatGPT · vestica

All fifteen artworks remain the property and copyright of their artists. They appear here as part of the collection's record, not as anything for sale. Back to the top