some of my CI runs are failing at the branch creation step (POST https://console.neon.tech/api/v2/projects/$PROJECT/branches) with this error “project already has running operations, scheduling of new ones is prohibited”.
I’m only running max 1-2 CI jobs at a time at present; such strict concurrency limits would seem to render the use case of branching for CI relatively unworkable (unless I’m missing something).
Is this expected behaviour? Anything that can be done to resolve and support this use case creating potentially several branches within a given project at the same time during parallel CI runs? Thanks!
Yeah, sorry for that, I clearly understand the pain point.
In order to clarify this a bit. We are currently in the middle of the migration process from the point ‘one project = one branch = one endpoint’, where the rule ‘one operation at a time’ worked pretty well. To the point of ‘many branches and endpoints’.
Allowing concurrent operation on different branches / endpoints is on our roadmap, but so far this limitation is a protection from many bad concurrency issues we can have internally.
A few workarounds I see right now:
if you use some script to create branches, then it’s better to add a retry procedure to all non-read-only request to the API for the same project with some reasonable time threshold like < 60 secs. Code 423 Locked should which is returned with this error should be 100% safe to retry
Would your setup permit modification and use of terraform? If yes, I’d like to kindly suggest the terraform provider to provision branches. It has a backoff-retry mechanism implemented so you won’t have to deal with handling the “locked” http response status code.
