I need to run tests in GitHub Actions where the tests connect to the Pub/Sub emulator. GitHub's recommended way to start something in the background that runs throughout your workflow, for things in your workflow to use, appears to be "services" (https://docs.github.com/en/actions/using-containerized-services/about-service-containers).
But I wasn't able to get this to work. I wasn't able to find a way to override the command used to run the Docker image. People suggested workarounds like creating a container where you use the gcloud Docker image as a base image and use your own command to start whichever emulator you need. But I don't like to idea of having to set up and maintain a system to keep an up to date Pub/Sub emulator image available for our workflows.
I was able to put together a workflow that appeared to work. The entire example, with the readme documenting my findings, the workflow file, and the Go source code that the workflow runs, is at https://github.com/mattwelke/github-actions-pubsub-emulator-example.
Just in case that repo isn't available at some point in the future, here's the YAML for the workflow, which I think is the important part. What makes it work for me is adding & to a step that starts the emulator so that it runs in the background and my Go code can then run, connecting to it:
on:
pull_request:
# Sequence of patterns matched against refs/heads
branches:
- main
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/[email protected]
- uses: actions/[email protected]
- uses: actions/[email protected]
with:
distribution: 'temurin'
java-version: '17'
- name: 'Install Cloud SDK'
uses: google-github-actions/[email protected]
with:
install_components: 'beta,pubsub-emulator'
- name: 'Start Pub/Sub emulator'
run: |
gcloud beta emulators pubsub start --project=testproject --host-port=0.0.0.0:8085 &
- name: Run program
env:
PUBSUB_EMULATOR_HOST: 'localhost:8085'
run: |
go run main.goIs this the recommended way to do this use case? If not, is there a better way?
For some added context, we're porting our workflows over from CircleCI, and we have this at the top of one of our CircleCI jobs:
version: 2.1
jobs:
run-unit-tests:
docker:
- image: circleci/golang:1.13
...
- image: google/cloud-sdk
command: gcloud beta emulators bigtable start --host-port=0.0.0.0:8086It looks like we were able to use a command to control how the google/cloud-sdk image starts, and that's how we got it to start the emulator as a CircleCI service for the entire job to use.
There's a community docker container that runs the emulator, which you could run as a service: https://github.com/marcelcorso/gcloud-pubsub-emulator. You could also build your own container and publish it to a registry:
FROM google/cloud-sdk
ENTRYPOINT "gcloud beta emulators bigtable start --host-port=0.0.0.0:8086"
GitHub does not support arbitrary args to service containers (https://github.community/t/how-do-i-properly-override-a-service-entrypoint/17435/8). Unfortunately we have no control over this.
Please let me know if you have other questions.
@sethvargo Thanks for the response. If I choose not to go the route of maintaining my own image that runs the emulator, or choosing a community maintained image, and instead run it the way I have in my YAML here, do you foresee any problems?
@mattwelke the biggest problem I see is, if you were to use self-hosted runners, or if GitHub were to ever allow managed runners to be re-used, that process is not killed when the job finishes. Thus, it could continue to run forever and conflict with future jobs.
For a more full-proof solution, you could write the PID ($!) to an output and then have an step that is always() that kills the PID.
@sethvargo That's helpful, ty. I don't think it's that hard to create a simple image like your example and update it once in a while, so that's probably the route we'll go.
| Owner Name | google-github-actions |
| Repo Name | setup-gcloud |
| Full Name | google-github-actions/setup-gcloud |
| Language | TypeScript |
| Created Date | 2019-11-05 |
| Updated Date | 2023-03-23 |
| Star Count | 1505 |
| Watcher Count | 65 |
| Fork Count | 548 |
| Issue Count | 11 |
| Issue Title | Created Date | Updated Date |
|---|