subDimension

Publish to GitHub from Editorial

After I decided to move my blog into GitHub, I got quite excited about learning to use the API. I downloaded Editorial a few weeks ago but hadn't gotten around to using it much yet but it has an absolutely fantastic Python scripting engine built in with all kinds of really cool modules. I set aside a whole weekend to get stuck in an learn the arcane invocations required to activate the API and mate it with Python.

I ended up being really disappointed - the GitHub API is incredibly easy to use and the modules built into Editorial made it an absolute breeze to get everything working! I especially liked the keychain module for safely storing API keys out of the source.

Here's my script Gist:

#coding: utf-8
import keychain
import console
import editor

import time
import re

import requests
import json
import base64

SITE_BRANCH = 'master' # either master or gh-pages
COMMITTER = {'name': 'Joe Bloggs', 'email': 'email@example.com'}

#keychain.delete_password('GitHub', 'username')    # Uncomment these lines
#keychain.delete_password('GitHub', 'token')       # to change the details
#keychain.delete_password('GitHub', 'repository')  # stored in the keychain

# Get Username, API Token and Repository
username = keychain.get_password('GitHub', 'username')
if not username:
    username = console.input_alert("Username", "Enter your GitHub Username", '', "Save")
    keychain.set_password('GitHub', 'username', username)

tokn = keychain.get_password('GitHub', 'token')
if not tokn:
    tokn = console.password_alert("API Token", "Enter your GitHub API Token", '', "Save")
    keychain.set_password('GitHub', 'token', tokn)

repo = keychain.get_password('GitHub', 'repository')
if not repo:
    repo = console.input_alert("Repository", "Enter your GitHub Repository name", '', "Save")
    keychain.set_password('GitHub', 'repository', repo)

# Mangle the post ;)
post_text = editor.get_text()

post_sections = post_text.split('---')
if len(post_sections) > 1:
    yaml_header = post_sections[1].splitlines()

  # Find the title in the YAML
    post_title = None
    date = None
    for line in yaml_header:
        if line[:6] == 'title:':
            post_title = line[6:].strip()
        elif line[:5] == 'date:':
            date = line[5:].strip()[:10]

    if post_title:
        safe_title = re.sub('[^a-zA-Z0-9\s]', '', post_title).replace(' ', '-')
        safe_title.replace('--', '-')
        if not date:
            date = time.strftime('%Y-%m-%d', time.gmtime())

        post_filename = '_posts/%s-%s.markdown' % (date, safe_title)

        URL = 'https://api.github.com/repos/%s/%s/contents/%s' % (username, repo, post_filename)

        header = {
            'Authorization': 'token %s' % tokn,
            'User-Agent': username
        }

        get_data = {
            'path': post_filename,
            'ref': SITE_BRANCH
        }

        response = requests.get(URL, headers=header, params=get_data)
        response_json = response.json()

        if response.status_code == 404:     # File doesn't exist, create it.
            data = {
                'path': post_filename,
                'content': base64.b64encode(post_text),
                'message': 'Blog Post - %s' % post_title,
                'branch': SITE_BRANCH,
                'committer': COMMITTER
            }

            response = requests.put(URL, headers=header, data=json.dumps(data))

            if response.status_code == 201:
                console.hud_alert("Blog post created successfully.", 'success', 2)
            else:
                console.alert("Commit failed.")
        elif response.status_code == 200:   # File exists, update it.
            data = {
                'path': post_filename,
                'content': base64.b64encode(post_text),
                'message': 'Blog Post - %s' % post_title,
                'branch': SITE_BRANCH,
                'committer': COMMITTER,
                'sha': response_json['sha']
            }

            response = requests.put(URL, headers=header, data=json.dumps(data))

            if response.status_code == 200:
                console.hud_alert("Blog post updated successfully.", 'success', 2)
            else:
                console.alert("Commit failed.")
        else:                        # Something went wrong!
            console.alert("There was a problem with the server.")

    else:
        console.alert("Couldn't find a title.\n\nAction Halted.")

else:
    console.alert("No YAML header found.\n\nAction Halted.")

I could have stored the branch and committee details in the keychain too, but I thought 5 pop-up dialogs might be two too many!

I also adapted it to pull in a post from GitHub using the YAML front matter; you have to add the date field for that to work.

I'm really impressed with Editorial and I think it has some huge potential.

Moving home

I've spent the last couple of days moving my website from my virtual host over to GitHub Pages. I've been overtaxing the server it sits on and it kept falling over. I want my website to be a little bit more stable!

I'm going to write up some of the problems I came across over the next week or so, but I just wanted to see how easy it was to write a new post directly on the website!

Things I still need to do:

  • Implement some kind of search system
  • Trawl for broken links
  • Write a Python module for publishing posts right out of Editorial
  • I also need to fix the way posts are displayed in the RSS Feed!

Advertising - An Update

Back in January I decided to try a short experiment by adding ads to my site, just to see what happened.

I only intended to leave it in place for a couple of months, but I think I've established by now that I'm at the very least a chronic procrastinator.

In the 10 months that I have had adverts on my site, I have made a grand total of £2.66; since Google puts a payment threshold of £60.00 on payments, it would take me about 20 years just to get paid.

I don't think I'm uncovering any Grand Truth or anything here - my blog traffic is small, but when I looked at the quality of the adverts I was displaying, I realised that it just wasn't worth it.

I have removed ads and will henceforth be writing for the sheer pleasure.

Never tell people about your projects

I last posted here on the 1st of July. Not the longest outage, but possibly the most irritating.

I'd been working hard on my mo.lecul.es project and was getting myself psyched up to actually release it.

Then I went on holiday for a month, came back and went back to my day job which is very demanding and simply didn't have the time or enthusiasm to get things moving again. I still use mo.lecul.es every single day and it works fantastically for my needs. There are a few bugs here and there and the web app client still leaves a lot to be desired but it happily does its thing every 15 minutes, works wonderfully well with Reeder 2 and I love it.

Talk about RSS aggregator services however seems to have completely died. Everyone seems to have picked a replacement service for Google Reader and moved on with their lives! I've pretty much convinced myself that at this point, if I fixed the bugs and sorted out a payment service, I'd get all of about zero customers. This is mostly why I will never be a Silicon Valley Gajillionaire, but I think I'm mostly OK with that.

What it did do though was make me too afraid to write on my blog. I got anxious about it, thinking that if I posted about anything that wasn't "I made a startup, pay me money now please", people would see that I had failed. Maybe I have, although ultimately, I have a nice little RSS aggregation system that keeps my thirst for stuff to read sated.

I worked out the other day that of the 24 hours I get in each day, after taking off my job, looking after my children and sleeping, I get about 2 and a half hours of my own, during which I eat and spend time with my wife! This is why Silicon Valley Gajillionaires are almost entirely very young, single and childless.

I still love to tinker with stuff, and I'm hoping that by coming to terms with putting mo.lecul.es to sleep for the time being, I'll be able to let myself actually play with other things:

  • A pixel art game heavily inspired by FTL
  • An RPG type game to let me relive my glory days in Ultima Online
  • Learn Ember.js so I can write a cool web app for mo.lecul.es
  • A project management type app that that brings my Todo App, Bookmarking App and stuff all together in a really useful tool.
  • A Gallery thingy so I can put my photos on my website
  • Finally create an iOS app

Service Un-interrupted

As I write this, Google Reader is still available but its lifetime is measured in hours.

For the last 3 days, I have been using mo.lecul.es via Reeder on my iPhone, using the Fever API option, and the web app that I started building on Friday on everything else!

It is my intention to launch mo.lecul.es as a paid service in the next week or two. I had hoped to be finished with it before Google closed the shutters on Reader, but life got in the way a little.

The system is functional, but it is still a long way from properly usable. Some things that still need to be done:

  1. Add a way to subscribe to feeds without having to log into the database and manually type in SQL!
  2. Add a way to un-subscribe, move feeds into groups and rename them.
  3. Add a 'logout' button
  4. Add a way to change your password
  5. Create a signup page
  6. Integrate with a payment processor
  7. Make a nice landing page with tour screenshots
  8. Implement the Google Reader API
  9. Miscellaneous design/layout tweaks

So just a few things there. Longer term, the web app I built is awesome, I'm really pleased with it but I build things old school - no fancy javascript frameworks until everything works 'sans-script'! (actually, no fancy frameworks at all, I like to build things myself). That means there are a lot of page refreshes - mark an item read; reload the whole page, switch feed; reload the page. This works great all the time it is just me, but I can't see it scaling particularly well!

I realise too that the payment processor part is going to take a little while, but I'm giving myself a little extra time on that one - all accounts will start off free for the first month. I think paying for software is really important, but I also like to take things for a test drive before I plunk down cash. I guess the clock starts ticking on that one the moment I get my first customer!

Tasks for today are numbers 1, 2 and 3 on the list.

Great news

Mr. Reader got an update today, it now supports a whole slew of sync services including Fever.

I'm still working hard on mo.lecul.es, my RSS Sync Service and updated it today to work with Reeder (iPhone), Mr. Reader and Sunstroke. So currently the API side of things is in Private Beta and I'm working on a web frontend to enable management of subscriptions and groups.

Making Progress

Cadence Emilia

Things have been moving slowly in the last week on account of a new arrival in our house!

However, I have built the very basic skeleton of an RSS sync system that uses the Fever API. Here's a screenshot of Reeder syncing with my test server (currently only subscribed to the feed for this website!)

mo.lecul.es syncing with Reeder

The Fever API doesn't allow for editing feeds, so currently in order to add a new subscription to my account, I have to log onto the database server and manually add it, but it's a good start and I think I've got the hardest 2 parts done. Just the huge easy bit of building a webapp to manage it all to go!

I'm planning to enter Ludum Dare tomorrow, just to start easing myself back into coding, so I'll probably be writing a fair bit about that as I go.

RSS Sync and Google Reader

Google announced yesterday that they are shuttering Reader. I'm rather sad about this since I use Reader and a number of apps that interface with it many times every day. It's the first tab I open when I start my browser.

Back in 2011 I wrote a post about developing my own system. I've noticed that it's getting quite a few pageviews at the moment! I did indeed begin developing my own service, but it was a bit of a forever project for me. I have a domain, a database and a skeleton api but nothing fully working; yet.

I'm going to switch priorities on this now and start building something that I would be happy to use myself in place of Google Reader.

Maybe my forever project could become my forever career

Pew! Pew!

I've really been enjoying noodling about learning HTML5 Canvas and JavaScript and whatnot, I have plans!

A couple of weeks ago I started running Computer Club at the school I teach at. Last week we started playing around with a space shooter akin to Xenon 2* and its ilk.

The code is available on GitHub and you can also Play the Game.

IMPORTANT NOTE: The graphics are intentionally crappy! The idea was to offend the kids' sensibilities so much that they made their own art!! (The game is rubbish because making games is hard)

* Cute fact, my brother and I called this "Exanon" for far longer than either of us care to admit when we were kids.

Static Blog - The New "Hello World"

I've been thinking about a comment I read on an HN thread introducing a static blogging engine someone had written.

It's difficult to tell the tone the line was written with:

Are static site generators the new "hello world?"

when reading anything on the internet, one's immediate reaction is usually that the person is criticising or disparaging, sometimes this is even without merit!

I kept thinking about it though. In April of 2012, I re-wrote my PHP based static blog engine using Python; not because there was anything particularly wrong with the system I had already built and was using, but because I wanted to learn Python.

"Hello World" is a deeply dissatisfying first step in learning a language. I remember sitting in the computer lab at my school, aged 13, entering into a BASIC prompt:

10 PRINT "HELLO WORLD"
20 GOTO 10

and feeling very foolish when I had to ask someone how to make it stop on the unfamiliar machine I was using! Following that, I was left with the question "What now?"

The great thing about building a static blogging engine is that it touches on most of the fundamentally important aspects of creating a useful application:

  • Reading and writing files
  • Capturing user input
  • Formatting Strings
  • Loop and other flow control
  • Arrays and other simple data structures
  • Incorporating external libraries

You start off (ironically) creating a simple HelloWorld.txt entry and learn how to open the file and read in the contents. Then you start to think about outputting that as HTML, working in templating and markdown libraries, and writing it to a file. And so on.

It's clear from some of the other comments in that thread that some people believe that once there is a good solution for a given problem, there is no merit in creating more. I'd say that my current static blogging engine is a long way from the best option, but I enjoyed writing it and I learnt a lot by doing so.

Whilst I would recommend building one to anyone thinking of learning a new language, nothing compares to solving an actual problem you have right now.