I’m a developer at Snaplet where we provide anonymized production data and realistic seeds to our users.
I’m very interested by Neon for its cheap branching model, but I have questions.
How is the compatibility to Postgresql right now? Is it 100% compliant with any schema/type/extensions? A lot of our users are using extensions so we have to make sure that if we start experimenting with Neon, they will be able to transparently create their test / temporaries databases.
Is Neon process run as a whole database server? So can we create multiple databases, and then create multiple branches on each of these databases? The idea here is that our users take transformed snapshots of their production data, and I’d like to create one database per snapshot, then only use short lived branches when they want to code against their snapshots.
Reading the doc, it’s not clear to me how to setup the storage part of Neon. I understand that we can store the data directly on S3, but is it also possible to have a mounted volume and store the data there (locally)?
Thanks for this awesome project, I think it will truly disrupt the Postgres ecosystem for the best!
Our cloud service doesn’t give you root database access, so one can’t install external extensions. In a self-hosted setup, it is possible to install any extensions. Most extensions should work fine unless they interact with files/WAL directly. The only such extension that I know is the old version of cstore_fdw – but it was rewritten using table access methods and integrated into Citus extension, so it should work now.
Is Neon process run as a whole database server?
Not sure that I get the question. You read a bit more about architecture here: Architecture decisions in Neon — Neon. Compute node is a patched Postgres which is more or less standard Postgres, with the same DDL/Auth/etc. We are hooking into replication and disk access code paths, so the execution part is mostly intact and should be totally compatible with vanilla Postgres.
I understand that we can store the data directly on S3, but is it also possible to have a mounted volume and store the data there (locally)?
Sure, you can configure pageserver to avoid any S3 interaction and store everything locally. The process is not documented yet, but you can look at neon_local sources (it spins up everything locally) to understand how all the services are configured.
Thanks for the reply @Stas_Kelvich ! It seems really promising for our use case!
What I meant was, is branching firing up a new posgresql instance based on $PGDATA? Or is it something more granular?
We are using Fly Machines (based on Firecracker) to host our temporary databases, I’ll try to replace our vanilla postgres docker image with neon and look into having the data being persisted on a mounted volume.
I would love to have a more direct line with someone at Neon and talk about a partnership with Snaplet, I really think we can find great workflows together!
