Neon’s serverless Postgres architecture is fundamentally different from PlanetScale’s MySQL, leading to distinct scaling behaviors and cost profiles.
Here’s a simplified view of a Neon transaction, showing how a compute node scales up, handles a query, and scales down, all within seconds.
{
"timestamp": "2023-10-27T10:30:05.123Z",
"event": "query",
"database": "mydb",
"user": "app_user",
"duration_ms": 15,
"compute_node_id": "cn-abc123xyz",
"storage_bytes_read": 10240,
"storage_bytes_written": 512
}
{
"timestamp": "2023-10-27T10:30:06.500Z",
"event": "compute_node_scale_up",
"database": "mydb",
"new_node_id": "cn-def456uvw",
"old_node_id": "cn-abc123xyz",
"reason": "load_threshold_exceeded",
"cpu_usage_percent": 85
}
{
"timestamp": "2023-10-27T10:30:08.900Z",
"event": "compute_node_scale_down",
"database": "mydb",
"node_id": "cn-def456uvw",
"reason": "idle_timeout",
"cpu_usage_percent": 10
}
PlanetScale, on the other hand, operates on a more traditional sharded MySQL architecture, where scaling involves adding or rebalancing shards, a process that typically requires more manual intervention and can have higher latency.
The Core Problem: State Management in Distributed Databases
Traditional databases keep their data and compute tightly coupled. When a database needs more power, you upgrade the whole server. When you need more storage, you add disks to that server. This works fine until you hit limits or need to scale independently.
Serverless databases like Neon decouple compute from storage. Your data lives in a distributed object store (like S3, but optimized for databases). When a query comes in, Neon spins up a temporary "compute node" that pulls the necessary data from storage, processes the query, and then spins down. This allows for near-instantaneous scaling of compute resources based on demand, and you only pay for the compute you actually use.
PlanetScale’s approach is different. It’s built on MySQL, but with a sophisticated sharding layer. Your data is split across multiple "primary" instances (shards). When you need more capacity, you can add more shards or increase the resources of existing ones. However, the compute and storage for each shard are still relatively coupled. Scaling often involves moving data between shards or provisioning larger instances, which can take time and might involve downtime or performance degradation.
How Neon’s Architecture Works
Neon uses a unique storage format called "rows" which are immutable, versioned data files. These files are stored in a distributed object store. A "branching" mechanism allows for efficient creation of new database states (like Git branches) without copying data.
When a query arrives:
- Compute Node Allocation: A request hits the Neon control plane. It identifies an available, scaled-down compute node or provisions a new one. This new node is essentially a fresh Postgres instance with a thin client that connects to the shared storage.
- Data Fetching: The compute node fetches the relevant data pages from the object store. Because of the optimized row format and caching, it only pulls what’s needed.
- Query Execution: The Postgres engine on the compute node executes the query.
- Write Operations: For writes, the compute node generates new "rows" files and uploads them to the object store. Crucially, it doesn’t directly modify existing data files.
- Scale Down: If the compute node becomes idle for a configured period (e.g., 5 minutes), it scales down, releasing its resources. You continue to pay for storage.
This architecture means that your database is always available, and compute can scale from zero to thousands of concurrent connections almost instantly. It also means that read performance is highly dependent on how well the data required by a query is cached in the compute node’s memory and local disk, and how efficiently it can be fetched from the object store.
How PlanetScale’s Architecture Works
PlanetScale is a managed MySQL service that emphasizes horizontal scaling through sharding.
- Sharding: Your data is partitioned across multiple MySQL primary instances. PlanetScale manages the routing of queries to the correct shard.
- Replication: Each primary instance has read replicas for scaling read operations.
- Scaling: To scale up, you can:
- Add more shards: This involves redistributing data, which can be a complex operation.
- Increase instance size: Upgrade the CPU, RAM, and disk of existing primary instances. This typically requires a brief restart or failover.
- Add read replicas: For read-heavy workloads.
- Connection Pooling: PlanetScale uses a robust connection pooler to manage connections efficiently.
PlanetScale’s strength lies in its familiarity for MySQL users and its robust tooling for schema changes and database management. However, scaling compute and storage together on each shard means that you might over-provision resources for one aspect to meet the needs of the other.
Key Differences and When to Choose Which
- Scaling Model: Neon scales compute independently and near-instantly. PlanetScale scales compute and storage more cohesively per shard, with scaling operations often requiring more planning.
- Cost: Neon’s serverless model means you pay for compute when used and storage separately. This can be extremely cost-effective for spiky or low-usage workloads. PlanetScale’s pricing is typically based on provisioned instance sizes and storage, which can be more predictable for consistent, high-throughput workloads.
- Database Engine: Neon is Postgres. PlanetScale is MySQL. This is often the primary deciding factor. If your application is already built for one, migrating can be a significant undertaking.
- Operational Overhead: Neon aims for minimal operational overhead, with automatic scaling. PlanetScale offers more control and features for managing traditional database operations, but with potentially higher operational engagement.
- Branching: Neon’s built-in branching for creating isolated, code-like database environments is a powerful feature for development and testing workflows, without data duplication. PlanetScale’s "branches" are more akin to Git branches for schema changes, not full database copies.
The most surprising aspect of Neon’s architecture is how it leverages Postgres’s extensibility and a custom storage format to achieve true serverless compute. It doesn’t just "scale down" a running Postgres instance; it spins up and down ephemeral instances that dynamically connect to a shared, distributed storage layer. This means that idle periods don’t incur compute costs, and the system can react to sudden traffic spikes by provisioning entirely new compute resources from a pool in seconds.
If you’re running a workload with highly variable traffic, where periods of intense activity are followed by lulls, Neon’s pay-for-what-you-use compute model can lead to significant cost savings compared to the provisioned capacity of PlanetScale. For example, a nightly batch job that runs for 30 minutes and then has no activity for 23.5 hours would incur minimal compute charges on Neon, whereas on PlanetScale, you’d be paying for the provisioned resources the entire time.
The next logical step is to explore how Neon’s branching feature can be integrated into CI/CD pipelines for automated testing.