Great product. Eagerly waiting for more stuff being shared around neon database.
We are currently testing out our staging and testing systems on neon in combination with branching and an easier workflow. Everything works great and the API is clear and easy to use (even though an SDK would be great).
The only problem that we are running into is that we receive connection errors several times a day.
We are using prisma to do all of our DB related stuff
Our guess is/was that the database was automatically scaled down to 0 and our system tries to send new queries and then fails with:
Server has closed the connection
Timed out fetching a new connection from the connection pool. More info: http://pris.ly/d/connection-pool (Current connection pool timeout: 30, connection limit: 17)
We activated connection pooling on the project and also telling prisma via the DB_URL to use ?pgbouncer=true
In order to solve the issue with Can't reach database server at... we added ?pgbouncer=true&connect_timeout=15&pool_timeout=30 so that the neon DB has some time to scale up from 0.
The connection pooling issue in my opinion should be solved by activating PGbouncer and telling prisma to use it, but this doesn’t seem to work. Is there any need to use ?connection_limit= on the database url string?
It would be great to get any guidance on on to be able to solve these two issues with our Prisma → NeonDB.
This is the workaround I used for the connection-timeout-when-endpoint-paused issue (below) and feature suggestion I had for the Neon team to make this a bit easier (maybe there’s a better way than what I proposed):
Suggestion for “Endpoint available” API route: Another example of something that’s totally possible today but requires a bit of effort: waiting to confirm an endpoint is ready for use.
Something not technically required but very nice to have could be an endpoint e.g. GET /v2/projects/{project_id}/endpoints/{endpoint_id}/ready - which would return 200 OK if there are no pending start/update branch operations, and something like e.g. 423 Locked if there are. Use-case is ensuring CI jobs don’t fail due to trying to access an endpoint that’s not ready yet and timing out.
I’ve just written something to repeatedly POST /v2/projects/{project_id}/endpoints/{endpoint_id}/start and parse the returned .operations array, retrying until it’s empty. Which seems to work just fine(?) - but 1) feels like a bit of a hack, 2) doesn’t feel 100% accurate (if there are ever operations ongoing that don’t mean the endpoint isn’t ready for use, e.g. availability checks, perhaps others(…??)), and 3) would just be nice if I didn’t have to write additional code and could reuse what I already had that checks for a http status code below 400 to indicate success.
So 7 months later we solved the Server has closed the connection issues but still see a lot of the Timed out fetching a new connection from the connection pool.
The sad thing is, queries sent via the pgbouncer instance -pooler with light/moderate traffic are 50x slower (0.07s non pooled connection vs 3.5s with pooled connection), whereas at the same time queries sent to the same DB with a non pooled connection are super fast.
Moreover, support recommends to use direct DB connections knowing we are using serverless infrastructure + distributed long running instances…