Branching beta feedback

General
1

Use case: configuring CI/CD (using GitHub actions) to create temporary branch on push → confirm migrations apply to branch (using Prisma migrate) → apply migrations to root branch if successful.

(Actual implementation more complex with dev + production environments, but that’s the basic idea)

Inspired by the example at GitHub - kelvich/branching_demo_ts (as demoed in Nikita’s Prisma Day talk)

Things I discovered so far:

  1. V1 API (as used in demo repo) creates branches that appear as projects rather than branches (I’m guessing this is the older behaviour prior to the most recent branching beta release last week?)

  2. → tried the V2 API, which successfully creates branches as per web interface.

  3. → V2 API requires additional work to achieve the same result vs. V1 API:

# V1
curl -o - -X POST https://console.stage.neon.tech/api/v1/projects/$PROJECT/branches \
  -H "Authorization: Bearer $API_TOKEN" |
  jq -r '.id'

# V2
curl -o - -X POST https://console.neon.tech/api/v2/projects/$PROJECT/branches \
  -H "Authorization: Bearer $NEON_API_TOKEN" \
  -H 'Content-Type: application/json' \
  # POST body now required
  #
  # i. calling from shell: need to wrap string in double quotes for env vars to be parsed
  #     and therefore escape double quotes within JSON string
  #
  # ii. need to specify parent_id - doesn't default to root like v1 API
  #     -> need to obtain root branch ID via API GET projects/.../branches
  #          OR from URL for 'main' branch in web interface (can't see it elsewhere)
  #
  # iii. need to specify branch name - doesn't auto-create like v1 API
  #      -> must not exist already
  #      -> user now needs to find a method to create unique ID
  #           (here using $GITHUB_RUN_* env vars within GitHub Actions)
   -d "{\"branch\":{\"parent_id\":\"$PARENT_BRANCH_ID\", \"name\": \"ci-actions/runs/$GITHUB_RUN_ID#$GITHUB_RUN_ATTEMPT\"}}"

Now working - but would have required less work if there was an API that creates new-style branches with simplicity of V1 API (defaulting parent_id to root and auto-creating branch name if not specified - for “throwaway” test branches).

  1. Wish: Branch rebase feature [maybe - not sure whether I’d actually have a need for use this yet - still figuring how best to implement] - so I can update a branch’s data in-place to a newer version of the original parent branch without having to create a new branch. (New branches require updated host name in connection string, which would require redeploying test environments with updated env variable).

  2. Endpoint for connecting to newly-created branch:
    V1 API demo used ${{ branch.id }}.cloud.stage.neon.tech. → removed .stage. to make it work via ${{ branch.id }}.cloud.neon.tech.
    Doesn’t work with V2 branches → connection errors with “Console request failed” (doesn’t explain cause of error/my mistake).
    Realised I need to construct different endpoint for V2: "${{ branch.endpoints[0].id }}.${{ branch.endpoints[0].proxy_host }}"
    Would be helpful/less work if: 1) full endpoint host name returned in V2 API result; 2) easy/predictable way to get working host name just using branch ID/single toplevel response field without having to drill into array of multiple potential endpoints, as with V1.

Hope helpful to share!

3 Likes
2

Update: added point 5 re: endpoint hostname

3

Thanks for trying it out and such a detailed feedback! Here are some thoughts:

    1. Yeah, v1 is a legacy implementation, which is gone from UI and going to be completely removed rather soon.

  2. ii. sounds reasonable to default to the root branch in the API, iii. also don’t see any problems in setting ID to the name in the API if not provided

  3. Hm, tricky question. Proper rebasing means that we merge data changes from the branch with new changes in the base branch, which doesn’t sound realistic for binary data generally, at least at this stage.

    If you are OK with just resetting the data in branch to some newer state of the base branch (hard reset in git), then we can implement something like this, which internally will just replace this branch with a new one right from the base branch. That’s doable.

    Yet, if the main thing you care is keeping the connection string, then we going to allow replacing the branch under endpoint. In other words, you can create a new branch and update your old endpoint to run on it, so that connection string will be preserved. We 100% going to implement this really soon. Maybe this one will solve your use-case?

  4. Ugh, that’s a good question we thought a lot and didn’t come with a good solution so far. With project creation we return a ready to use connection string as we know all the components (host, db, role, password), but when you create a branch, there could be several roles / dbs and we don’t know which one to use. However, I now see that I don’t see any reason in not returning at least the complete host with endpoint id.

To summarize, 3 and 5 look like something immediately actionable to me, 4 will be somewhat addressed in the near future, but cannot be done completely right (data merge) in the near future. I guess we will have some discussion on this matter, so cannot guarantee that it will work exactly as I explained above, but that’s how I’d do it :slight_smile:

3 Likes
4

Thanks! That all sounds great.

For (4) I was mistaken in using the word “rebase” - I actually meant “hard reset”, as you suspected :wink: Data merging is not what I meant (not even clear that could be done effectively without knowing application-specific implementation details!). Being able to switch the branch under an endpoint sounds an even better solution.

Re: (5) yes - just the hostname would be perfect. Credentials I’d expect to handle separately, as with the example using the V1 API. The desired outcome is not writing my own code that reconstructs Neon’s logic for hostname generation (which I’ve no idea if/when may change…)

4 Likes