Syncing Adobe Launch/Tags with Github - Part 2: The Github Push Webhook

The Github push webhook

After the setup that I mentioned in the first post is completed, I can tap into the Github push webhook to kickoff the syncing between Github and Launch. This webhook will notify my API that some file modifications have occurred in Github and that those changes that may require updates to Launch components. I say may here because the Github push webhook will notify me of all file modifications in my repository and it will be up to me to determine what, if anything, needs to be synced with Launch.

The Github push webhook also introduces the first layer of security in the form of a Webhook secret. This allows me to verify calls made to the API. More on that later.

So what exactly is webhook? Off the top of my head, and by that, I mean according to ChatGPT, a webhook is:

a way for two applications to communicate with each other in real-time. It is a type of API (Application Programming Interface) that allows one application to send data to \ another application automatically whenever a certain event occurs.

Webhooks work by sending a notification to a URL endpoint that is specified by the receiving application. This notification typically contains a payload of data that describes the event that triggered the webhook.

In order to start receiving notifications from Github about file updates via the push webhook, I need to do 2 things:

• Build and API endpoint for the webhook to send its POST requests to
• Configure that endpoint as a webhook in my Github repo

I'll start with building an endpoint...

My API for receiving Github push webhooks

Fortunately for me, going super in-depth about building and hosting API endpoints is beyond the scope of this little blog I got going here. If you are new to such things, I highly suggest reading about building APIs in Flask for Python (what I used) or Express for Node. Once built, you then need to host your endpoint server somewhere public so Github can access it. Lots of options here, the most obvious (and easiest IMO) AWS web hook gateways.

Below is the Flask route for my endpoint that will receive and handle pushes from Github. If you've never worked in Flask or made an API before, it might look a little complicated, and yes it is truncated for brevity, but it really only does a couple things:

• @verify_github_webhook_secret is a Python decorator that makes sure any POST requests to this route not only come from Github, but come from me, specifically.
• modified_data_element_files is a list of all the data element files that have been updated in this push. If there are zero changed data element files, like if I updated a README.md file, the entire flow stops here. No need to sync with Launch.
• The dict called data contains all the information I will need about this push to look up the actual content of the files. The push webhook only says that contents was modified, but not specifically what those modifications were.
• Finally, if everything checks out, I invoke the next Lambda in the flow which will call the github API to extract the specific file contents which will prepare them for the Launch API. In my actual code FunctionName is the name of the next lambda I call.

@app.route("/github-webhook-handler", methods=['POST'])
@verify_github_webhook_secret
def github_webhook():
    try:
        try:
            payload = request.get_json()
            modified_files = payload['head_commit']['modified']
            author_name = payload['head_commit']['author']['name']
            owner = payload['head_commit']['author']['username']
            repo = payload['repository']['name']
            ref = payload['ref']
        except Exception:
            return make_response(jsonify(error='Error parsing github data from request'), 400)

        # find modified data elements files
        modified_data_element_files = [
            file for file in modified_files if file.startswith("dataelements")]

        # the request was successful, no action needs to be taken
        if len(modified_data_element_files) == 0:
            return make_response(jsonify(message='No files in the data elements directory have been modified'), 204)

        # everything is good - trigger the lambda - the lambda does the launch updating!
        data = {
            "message": "OK",
            "modified_data_element_files": modified_data_element_files,
            "author_name": author_name,
            "owner": owner,
            "repo": repo,
            "ref": ref
        }

        # invoke the next lambda here
        client = boto3.client('lambda')
        response = client.invoke(
            FunctionName='some fake name here',
            InvocationType='Event',
            Payload=json.dumps(data)
        )
        # return success
        response_data = {
            "statusCode": 200,
            "body": data
        }
        return make_response(jsonify(response_data), 200)
    except Exception as e:
        return make_response(jsonify(error=f'Error {e}'), 500)

By far, the easiest way to create and deploy AWS Lambdas and API Gateways is through the Serverless framework. It takes care of the drudgery of uploading packages to AWS and has a nicer UI. Highly recommended.

Once I get this API deployed, I'm going to have a publicly reachable route to POST to. First task complete. It is this URL that I will then configure inside of Github.

My deployed API will have a route that will look something like the URL below. Notice how the path github-webhook-handler is the path I defined in @app.route above.

https://fakedatahere.execute-api.aws-region-name-here.amazonaws.com/github-webhook-handler

Setting up a Push webhook on a Github repo

Now that I have my endpoint URL all set up and running in "the cloud", I need to tell my Github repo to post to it on Push events. To do that, I go into my Repo and go to Settings > Webhooks > Add Webhook.

The endpoint URL I made goes in payload url and I need to generate a random string for a secret and save it somewhere safe. Again, my Secret value is used in the @verify_github_webhook_secret decorator to verify requests that the API receives.

Once the webhook is setup, I can QA that it's working by doing some push events and checking the Recent Deliveries section. If something is broken, like I pasted the wrong URL or there is an authentication error, it will show here.

In action

So there is not much to look at, but here is a recap of what happens in the first step of the *Github to Launch sync *flow when I update a file on Github that needs to be synced with a Launch component:

1. I push to the main branch of my Github repo
2. Github triggers the push webhook and sends a POST request to my API
3. My API reads the data from the webhook request and determines if there are file changes that need to be updated in Launch.
4. If changes need to be synced, the API gathers the information required to pull the actual file contents from Github (which takes place in the next step) and triggers the next lambda in the flow.

Here is the response from the Push webhook showing my success message when I update my host.js data element file.

I can't show the whole JSON response, but it should look familiar:

data = {
            "message": "OK",
            "modified_data_element_files": modified_data_element_files,
            "author_name": author_name,
            "owner": owner,
            "repo": repo,
            "ref": ref
        }

Next up

So I've been notified that a change has occurred that needs to be reflected in Launch. Next up, grab that actual contents of those file changes and prepare to send them to the Launch API.