Hi… how ya been, its been a month already, gosh times flies by really fast. Status update: I’ve been doing good, When I am in home I complete the work that I plan… far better than when I am in Chennai. The corona virus pandemic is still around the world at large, US right now is the largest affected country with over 70k dead people as of writing this. The lockdown in India has been extended another 2 weeks.

Note: I have edited this post, because of the changes I made, after publishing this post.

This post is going to be about github actions.

Why

Automate a process after commit, schedule a process to be done timely…etc the possibililties are endless. A little background… lastweek github announced unlimited private colaborations and made a ton of premium features completely free.

I am going to use this feature entirely for a different purpose, It further backs-up my saying..literally anything can be done.

Goal

  • I have this github repository which is XKCD viewer, which utilises api ‘https://xkcd.com/info.0.json'. The api does not provide cors.

  • I want to clone the api, (the clone must be updated daily).

  • should provide cors support.

  • should not contain any external dependencies like server to set it up.

Solution

  • github offers unlimited time constraint for github actions(public repository)

  • using github actions I could setup a bot that runs timely.

  • I can mainatain a local copy of the api, compare it to the offical every day,

  • if there is difference, then download the new contents and update the local state.

  • commit the changes directly to the repository.

Approach

  • Im going with python in conjunction with workflow file, since I want to brush up my python skills.

  • The python logic will maintain a local state, add new contents, update itself i.e. the state.

  • I am going to use json to maintain the state.

  • The api data will be stored under a folder with every entry having a unique folder.

  • Use github actions to commit the changes to the directory.

  • Use github secrets to maintain a token, (token used to commit diretly to repository)

Result.

After coding away the mentioned logics above. I am left with this workflow file.

name: fetch new content from api
on:
  schedule:
    - cron: '0 */8 * * *'

jobs:
  fetch-content:
    if: github.repository == 'aghontpi/xkcd-declutter'
    name: execute python logic
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - name: install requests pip3
        run: pip3 install requests
      - name: run python logic
        run: python3 ./.github/workflows/update_xkcd_official.py
      - name: status of the new folder
        run: git status xkcd-offline/ --porcelain
      - name: add new files
        run: |
          git config --local user.name "xkcd-content-fetcher-bot"
          git add xkcd-offline/
      - name: commit new content
        run: (awk '{print "added content " substr($3, match($3,"\""), match($3,"}")-1) "."}' xkcd-offline/state.json) | git commit -F -
      - name: push the new content
        uses: ad-m/github-push-action@master
        with:
          github_token: ${{ secrets.TOKEN }}

See it in action, downloads new content automatically and put it under the folder xkcd-offline.

Link to project github

Updates/Post Edit

After I published this post, Github made changes to how an environment variable is accessed, changes contains the following.

  • updates to accessing environment variable.
  • decided to log while every run, that is persistent in the repository.
  • run every 12 hours instead of 8 hours.
  • added github actions bot as the author in commit message
  • update instructions to how to use this API (supporting cors)

updates to yaml

name: Sync with Api
on:
  schedule:
    - cron: '0 */12 * * *'

jobs:
  fetch-content:
    if: github.repository == 'aghontpi/mirror-xkcd-api'
    name: execute python logic
    runs-on: ubuntu-latest
    env:
      new_content: true
    steps:
      - uses: actions/checkout@v1
      - name: install requests pip3
        run: pip3 install requests
      - name: run python logic
        run: python3 ./sync.py
      - name: status of the new folder
        run: git status api/ --porcelain
      - name: add new files
        run: |
          git config --local user.name "sync bot"
          git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
          git add *
      - name: commit new content
        id: newfiles
        run: (awk '{print "added content " substr($3, match($3,"\""), match($3,"}")-1) "."}' syncState.json) | git commit -F -
        continue-on-error: true
      - name: not proceed if there is no new files
        if: steps.newfiles.outcome == 'failure' && env.new_content == 'true'
        run: |
          echo "no new content was fetched"
          echo "new_content=false" >> $GITHUB_ENV
      - name: push the new content
        if: env.new_content == 'true'
        uses: ad-m/github-push-action@master
        with:
          github_token: ${{ secrets.TOKEN }}
          branch: main

how to use this as an api(supporting cors)

Instructions/Usage

End

See you soon, untill then hope you do well. - Gopinath aka aghontpi aka bluepie