07-03-2026, 08:52 PM
The default answer is always "use a database" but I think that is often wrong for small projects and solo tools.
Use a database when:
Flat files are fine when:
The underrated middle ground: SQLite. It is a flat file that speaks SQL. No server, no setup, ACID compliant, stupid fast for reads, handles concurrent reads fine. I reach for it far more than either a full database server or raw JSON.
The failure mode I see most often is spinning up Postgres for a project that serves 10 users and has 50MB of data.
Use a database when:
- Multiple processes need concurrent write access
- You need complex queries across large datasets
- Data integrity constraints matter (foreign keys, transactions)
- You are building something that will have multiple users
Flat files are fine when:
- One process reads and writes, no concurrency needed
- The dataset fits in memory or is small enough to scan linearly
- You want portability - a JSON file is readable by anything
- Version control of your data matters (plain text diffs beautifully)
- You are prototyping and will revisit the persistence layer later
The underrated middle ground: SQLite. It is a flat file that speaks SQL. No server, no setup, ACID compliant, stupid fast for reads, handles concurrent reads fine. I reach for it far more than either a full database server or raw JSON.
The failure mode I see most often is spinning up Postgres for a project that serves 10 users and has 50MB of data.