Making Progress
Things have been moving slowly in the last week on account of a new arrival in our house!
However, I have built the very basic skeleton of an RSS sync system that uses the Fever API. Here's a screenshot of Reeder syncing with my test server (currently only subscribed to the feed for this website!)
The Fever API doesn't allow for editing feeds, so currently in order to add a new subscription to my account, I have to log onto the database server and manually add it, but it's a good start and I think I've got the hardest 2 parts done. Just the huge easy bit of building a webapp to manage it all to go!
I'm planning to enter Ludum Dare tomorrow, just to start easing myself back into coding, so I'll probably be writing a fair bit about that as I go.
Posted by Mike Watts On Friday, 26 April 2013 @ 19:06 | permalink
RSS Sync and Google Reader
Google announced yesterday that they are shuttering Reader. I'm rather sad about this since I use Reader and a number of apps that interface with it many times every day. It's the first tab I open when I start my browser.
Back in 2011 I wrote a post about developing my own system. I've noticed that it's getting quite a few pageviews at the moment! I did indeed begin developing my own service, but it was a bit of a forever project for me. I have a domain, a database and a skeleton api but nothing fully working; yet.
I'm going to switch priorities on this now and start building something that I would be happy to use myself in place of Google Reader.
Maybe my forever project could become my forever career
Posted by Mike Watts On Thursday, 14 March 2013 @ 08:52 | permalink
Pew! Pew!
I've really been enjoying noodling about learning HTML5 Canvas and JavaScript and whatnot, I have plans!
A couple of weeks ago I started running Computer Club at the school I teach at. Last week we started playing around with a space shooter akin to Xenon 2* and its ilk.
The code is available on GitHub and you can also Play the Game.
IMPORTANT NOTE: The graphics are intentionally crappy! The idea was to offend the kids' sensibilities so much that they made their own art!! (The game is rubbish because making games is hard)
* Cute fact, my brother and I called this "Exanon" for far longer than either of us care to admit when we were kids.
Posted by Mike Watts On Monday, 11 March 2013 @ 12:28 | permalink
Static Blog - The New "Hello World"
I've been thinking about a comment I read on an HN thread introducing a static blogging engine someone had written.
It's difficult to tell the tone the line was written with:
Are static site generators the new "hello world?"
when reading anything on the internet, one's immediate reaction is usually that the person is criticising or disparaging, sometimes this is even without merit!
I kept thinking about it though. In April of 2012, I re-wrote my PHP based static blog engine using Python; not because there was anything particularly wrong with the system I had already built and was using, but because I wanted to learn Python.
"Hello World" is a deeply dissatisfying first step in learning a language. I remember sitting in the computer lab at my school, aged 13, entering into a BASIC prompt:
10 PRINT "HELLO WORLD"
20 GOTO 10
and feeling very foolish when I had to ask someone how to make it stop on the unfamiliar machine I was using! Following that, I was left with the question "What now?"
The great thing about building a static blogging engine is that it touches on most of the fundamentally important aspects of creating a useful application:
- Reading and writing files
- Capturing user input
- Formatting Strings
- Loop and other flow control
- Arrays and other simple data structures
- Incorporating external libraries
You start off (ironically) creating a simple HelloWorld.txt entry and learn how to open the file and read in the contents. Then you start to think about outputting that as HTML, working in templating and markdown libraries, and writing it to a file. And so on.
It's clear from some of the other comments in that thread that some people believe that once there is a good solution for a given problem, there is no merit in creating more. I'd say that my current static blogging engine is a long way from the best option, but I enjoyed writing it and I learnt a lot by doing so.
Whilst I would recommend building one to anyone thinking of learning a new language, nothing compares to solving an actual problem you have right now.
Posted by Mike Watts On Tuesday, 05 March 2013 @ 12:42 | permalink
Advertising
There are only really 2 ways to make money from writing on the internet:
The problem with a Paywall is that you really have to have a fairly large audience to start with. The problem with Advertising is that it's advertising.
Really though, there are others, these two are just the obvious ones.
In around February of 2011, I decided to take a slightly different direction with my online writing. Up until then I'd just written about whatever came to mind, which was usually a brief opinion on something that someone else had written. I was trying to emulate the blogs that I followed myself, but I'm not that strong a writer and I don't have access to the juicy inside information (or any information really). I was just adding to the echoes.
I started writing about my adventures in tech, explaining how I solved various coding-related problems. I was emulating a different set of blogs that I follow (of which there are fewer), but also the kinds of resources that I was using myself every day to solve the problems I would then write about.
My intention was to add value - where I had to cobble together 3 or 4 different blog or forum posts to find a solution, those that came after would find my single post that explained exactly how to do what needed to be done. If nothing else, I'm creating a resource for myself, so that when I come across the same problem next time, I can simply search my own blog.
A few of my posts seem to have served their purpose though, my Deploying Flask to Apache post has nearly 400 hits at time of writing.
So far this year, I've noticed an increase in traffic of about 50% overall, so I decided to try an experiment. I've added Google Ads to the sidebar, just beneath the main navigation. It's not in the perfect place for maximum eyeballification, but it's there, and I'll be interested to see what happens with it.
Personally, I rarely, if ever click on ads, my brain has mostly learnt to filter them out so I don't think I'll get many pennies from this endeavour. It would be nice to make enough to pay for my hosting, which is currently very cheap (no kickbacks, I feel nothing but positive about them!). I might experiment with writing some reviews of stuff I buy and including affiliate links, although again, I want to make sure I'm actually adding value somewhere.
I've enjoyed writing my blog far more since I switched to a more technical slant, and a brief look at my post archive suggests that I've been keeping it up for longer than ever before! It has had an interesting effect on my coding too - I find myself working on things in my spare time much more often, so that I can generate things to write about in my blog!
I shall give it a few months and report back with what I find.
Posted by Mike Watts On Tuesday, 22 January 2013 @ 12:34 | permalink
Toggling field values in PostgreSQL
It's important to me that each time I do something new, I move forward with knowledge of the technologies I'm using.
This weekend I created Ticks, a super simple TODO list. I needed a way to toggle a task between complete and incomplete. This status is stored simply as a timestamp field. If the field is NULL then the task is incomplete, otherwise, it was completed on the date stored.
UPDATE ticks
SET completed =
CASE
WHEN completed IS NULL THEN CURRENT_TIMESTAMP
ELSE NULL
END
WHERE task_id=%s;
Nice and simple, with just 1 query.
Posted by Mike Watts On Sunday, 20 January 2013 @ 22:26 | permalink
New Design for 2013
My website has a new design this week. I liked the simplicity of my old design, but I felt a little constrained by it. The width made me feel pressured to only post long articles, I'd like to be able to post twitter-style updates from time-to-time and I felt that they would look lost. The new design gives everything its own little box and I think this gives me more flexibility. My intention is to make a few small tweaks to my blogging engine over the next week or so to facilitate different kinds of posts.
The design is not so much inspired by my GitHub profile page as it is a complete rip-off. I really like the layout, the sidebar with the avatar communicates the ownership of the writing here. I hope they don't mind too much. There are still a few little bugs to work out, so things might look a bit wonky whilst I work through them all. I'm not sure how the monospace font comes across, but I spend most of my free time looking at code, so again, I think it communicates a little of my personality.
In preparation for my new design, I was perusing the Internet Archive. I was trying to get a feel for the last couple of designs I'd had. Unfortunately, the Way Back Machine hadn't archived any of my old style sheets, so I just got text. What they had archived though was a batch of old blog posts that I no-longer had! I spent a few hours copying and pasting the content into text-files and those posts are now preserved here. Reading through a few of them I was struck by just how poor I was (still am?) at writing and how I really didn't know what to write about. I think I've found my voice a little more over the last year or so, writing about developing the things I make. I don't need to worry about awkward opinions looking strange 3 years from now. I leave the posts as they were, aside from a few notes about things that have disappeared since I wrote about them.
I've also noticed that a lot of my posts start in exactly the same way, which often sounds a bit odd, so I'm going to work on that this year! I found a few tips on opening sentences, so I have somewhere to start!
Posted by Mike Watts On Thursday, 17 January 2013 @ 09:16 | permalink
Developing Games in Javascript
I was inspired by the Ludum Dare competition at the beginning of last month. I wanted to make a game!
Initially I started working in Python but sadly the current state of game libraries' OS X compatibility leaves plenty for another post. I thought I'd give Javascript a whirl.
There are many, many javascript game libraries. I invented 3 criteria for choosing one in the end:
- Can I understand the code, armed with just my early-2000's knowledge of Javascript?
- What's the documentation like?
- How cool do the example games look?
basically, #1 and #3 are inversely proportional and #2 is really hit-and-miss.
JawsJs ended up being my choice. It totally fulfilled #1, #2 was good enough that I could work out where to go next and #3 looked pitched at the sort of level I thought I could manage myself!
I found that I really enjoyed using it. It even became my first contributed repository on GitHub - I implemented A* pathfinding on TileMaps (which I'll probably write about soon).
My first game is still very much in progress, and it represents the absolute zenith of my artistic abilities, but I'm proud of it:
Play 'Rouge' here. It'll get better.
Posted by Mike Watts On Friday, 11 January 2013 @ 10:50 | permalink
PostgreSQL triggers in plpython
I wanted to run a query that inserted rows into table B whenever a new row was added to table A.
The first step took quite some time. You have to start off by adding the procedural language you wish to use to the database you're working with. I'm happy with Python, so I chose plpython:
$ psql
# CREATE LANGUAGE plpythonu;
but it didn't like that, I got an error:
ERROR: could not access file "$libdir/plpython": No such file or directory
Googling failed me! I couldn't find the answer anywhere. This time, a bit of "aptitude foo" came in useful (I'm running Ubuntu):
$ aptitude search postgresql
there were a lot of results, but neatly nestled towards the bottom was:
p postgresql-plpython-8.4 - PL/Python procedural language for PostgreS
so I installed that:
$ sudo aptitude install postgresql-plpython-8.4
back to psql and try creating the language again:
# CREATE LANGUAGE plpythonu;
CREATE LANGUAGE
success!
Next to create the function that I wanted to use in a trigger, I discovered a useful command in psql:
# \e
this pulls up the environment default editor to make writing queries a little less horrible!
CREATE FUNCTION unread_trigger()
RETURNS TRIGGER
AS $$
qry = plpy.prepare("""INSERT INTO unread_messages
SELECT m.message_id, s.user_id
FROM messages m
JOIN conversation_subscriptions s
ON s.conversation_id = m.conversation_id
WHERE m.message_id = $1;""", ("INTEGER",))
res = plpy.execute(qry, (TD['new']['message_id'],))
$$ LANGUAGE plpythonu;
When you exit the editor and save the temporary file, psql executes the query. I'll explain what's going on here in a minute. Another useful command in psql lists the functions created:
# \df
The next step was to add the function as a trigger to my table:
# CREATE TRIGGER set_unread
# AFTER INSERT ON messages
# FOR EACH ROW EXECUTE PROCEDURE unread_trigger();
so, every time a row is inserted into the messages table, the unread_trigger() function is called and inserts a new row in the unread_messages table for each user that is subscribed to the conversation that message belongs to. In that function, there is a dictionary called TD, this dictionary holds the various values relevant to the event that triggered the function, it's documented in the PL/Python PostgreSQL docs.
The PL/Python language includes a module that allows access to the database: plpy.
Finally, I didn't get the function right first time. I couldn't work out an easy way to modify a function, so I ended up DROPping and re-CREATEing it:
DROP TRIGGER set_unread ON messages;
DROP FUNCTION unread_trigger();
EDIT: I just realised you can use CREATE OR REPLACE FUNCTION ... and it will change the function definition, instead of dropping and re-creating it every time!
Posted by Mike Watts On Wednesday, 19 December 2012 @ 20:43 | permalink
PostgreSQL unicode
I was having trouble storing unicode strings in my postgresql database.
DISCLAIMER: I'm still really struggling with Unicode in Python, but I think I got a little closer to understanding today!
When I installed and configured PostgreSQL for the first time, the default setting for character set was LATIN-1, which is annoying. I kept running into problems saving bookmarks in subMarks when the page titles had unicode characters in them (the emdash has a lot to answer for).
In order to fix this, I had to convert my database to UTF-8. You can't actually do this though, instead, you have to create a new database, and dump the data into it. Creating a new UTF-8 database is fairly straightforward:
$ createdb -E UTF8 -T template0 NEW_DB_NAME
I used some instructions I found for transferring the data, however, I didn't drop the cluster and create a new one - I have other databases running and I really couldn't be bothered with all the hassle, I just created a new database and left the old one where it was. Basically, I did this:
$ pg_dump --blobs --oids OLD_DB_NAME > OLD_DB_NAME.latin1.sql
$ pg_dump --blobs --oids --encoding=UTF-8 OLD_DB_NAME > OLD_DB_NAME.utf8.sql
then:
psql --set ON_ERROR_STOP=on NEW_DB_NAME < OLD_DB_NAME.utf8.sql
That all went fine, but the next step was causing me some serious problems: connecting to the database from my python web app.
Since I'm running PostgreSQL 8.4, I couldn't specify the client_encoding as part of the DSN, it needs a separate step:
conn = psycopg2.connect(app.config['DSN'])
conn.set_client_encoding('UTF8')
But that isn't quite enough. UTF-8 is not Unicode so Python was getting upset with the encoding of the strings it got from the database.
My Googlefu was strong at this point though, I discovered the missing piece in a bug filing for Django. Adding the line:
psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
to the top of my database code did the job. You can view the code change in this commit.
This got me a little closer to finally getting Python Unicode!
Posted by Mike Watts On Tuesday, 11 December 2012 @ 22:04 | permalink