Faces and Feeds

I think I might have to develop an app for reading Facebook the way I think it should work.

There was an article doing the rounds the other week about how “our minds can be hijacked,” which was all about how terrible social networking is for us. I skimmed part of it, but got annoyed when it seemed to be about rich Silicon Valley entrepreneurs deciding to go “off-grid.” That’s all very well for them, but most of us have to make a living.

More pertinently, since the main target for the attack was Facebook, it annoyed me because I use Facebook to keep in touch with people that I might otherwise not. For that, it can be very good.

And yet… it struck a chord with, me to some degree. I realised that Facebook has increasingly become more of a time sink than a pleasure. Not that I spend vast amounts of time on it each day, but when I do open it up, I often end up spending longer than I’d have wanted to. And not reading updates from friends and family, but following links to articles and quizzes and nonsense, most of which I wish I hadn’t bothered with.

By comparison, a similar length of time spent in my feed reader lets me read blog pieces by people I actively want to hear from, and which I’m generally glad I’ve read.

But they mostly aren’t friends and family.

And then there’s the fact that the Facebook algorithm is tuned to show me what it thinks I should see, not what I want to see. What I want to see is all the updates from my friends, in reverse-chronological order. And that’s all. But there’s no guarantee that it will show me everything everyone posts, and the order is close to random at times.

One way to work round this is to visit people’s individual Facebook pages. You could see all your the posts by all your friends by going to each of their profiles in turn. But that would mean you’d have to keep track of all that: remember who you visited and when, and somehow manage the list of people.

Keeping track of things is what computers are good at. The software should be doing that for us.

So I’m thinking that what I want is an app that will do that for me: that will keep a list of my Facebook friends, and show me all their posts (which of course is what Facebook used to do).

As far as I know, no such app exists. This seems strange and unlikely, but I don’t think Facebook make a public API available for third-party clients, so such an app would have to work by scraping the web pages, which is neither good practice nor much fun.

Of course, what this means is effectively turning Facebook back into a set of RSS feeds — or now, especially as I have some experience with them, a set of JSON Feed feeds. Which would then be usable in all sorts of other places.

Web scraping may be bad and painful; still, I think I want to write this thing. Watch this space.

The Kickstarter Corporate Communication Conundrum

Today I chanced to see an email in which a manager was asking his staff to work for extra hours. Well, ‘asking’ is putting it generously, to be honest. There didn’t seem to be much that was optional about it.

The Kickstarter connection, though: you’ll be familiar with the idea of ‘stretch goals.’ If not, the idea is that the basic target is to make X amount of money, but if we make X + 10%, or whatever, we’ll be able to do these other things. Develop additional features, make the item in more colours, or whatever. My guess is that the term originally comes from sports.

So this email included in the subject the phrase ‘stretch targets.’ Meaning we want you to do more this week/month/whatever, than we originally planned. It was clearly written by someone who thinks that the way to develop software faster is to work your staff to the bone. When in fact that’s much more likely to result in people taking shortcuts and making mistakes.

In this team they’re already working weekends, and now they’re being ‘stretched’ even more. It bodes ill. But perhaps co-opting the language of positive things for something so negative is worse.

Some Open-Source Software for Your Delectation

I have made a thing, and pushed it out into the world. Well, really, this is me pushing it out into the world, because nobody will have noticed it before now, and with this, there’s a chance they might.

A couple of months ago Manton Reece and Brent Simmons announced the existence of JSON Feed, a new syndication format to sit alongside RSS and Atom; but using JavaScript Object Notation or JSON, instead of XML.

They invited people to write parsers and formatters and so on for it, and I quickly realised that no-one had yet written one in Java. As far as I can tell that is still the case. Or at least, if they have, they haven’t made it public yet.

No-one, that is, but me, as I have written just such a thing: a JSON Feed parsing library, written in Java. I’m calling it Pertwee. That’s the product page at my company site (more on which later). It’s open-source, and can be found at Github

As software projects go, it’s not that exciting. But it is the first open-source project that I’ve released. I hope someone might find some use for it.

Wondering why people recruiting for senior development positions often ask low-level JVM type questions. Doesn’t hurt to know that stuff, but who keeps it at their fingertips?

Swim, Test, Shop, Film, Sleep

Yesterday I kind of wilfully skipped a day. At some point in the evening I realised I wasn’t going to write a post, so I just said, “Fine: that’s allowed.”

Today I started by going for a swim. After my new regime of exercise last summer, I got out of the habit once I started a new contract. So it was good to get back to it. (Which is not to say I haven’t swum or gone to the gym in all that time, but it’s been a few weeks at the moment.)

After that I took a HackerRank test for a new job opportunity. It’s a site that does programming tests. This one was, I suspect, a disaster. I hate doing that kind of thing: you’ve got a timer running, and the problem you’re trying to solve is unlike anything you’d have to do professionally… Anyway, suffice to say, it didn’t go terribly well.

This evening was all about falling asleep in front of the telly. We tried to watch 20,000 Days On Earth, the film about Nick Cave from a few years back. I got it a few Christmases or birthdays ago, but hadn’t got round to watching it till now. I enjoyed what I saw of it, but there was definite falling asleep on the sofa and missing chunks. Oh well, it’s a DVD: we can always go back.

Oh yes: there was also a trip to Westfield, the time-void where hours go to die.

Some Thoughts On Software Development

Before the job interview that I mentioned the other day, the company asked me to answer some questions in writing. I didn’t get the job, but I was pleased with my written answers (and they presumably helped me to get the interview, at least). So I thought I’d reuse them as a blog post. None of this should be surprising for anyone who knows anything about the software development field, but it’s interesting to reflect on how things have changed across my career.

What are some of the fundamental changes in your approach to software development you have adopted in the last few years?

There are two main changes that are fundamental and independent of languages and deployment environments: agile techniques and test-driven development (TDD).

Agile

Moving from waterfall to agile development was probably the most significant change to development practices in the industry. We always knew that breaking work down into smaller units led to better estimating, more modular code, and just better software. The genius of agile was to extend that understanding to the period of time spent on a block of work. A two-week sprint, with its work being specifically estimated, planned and developed, is just infinitely more manageable than a project phase lasting months.

Add to that:

  • self-organising teams which include someone from the customer or end user — or at least someone whose role is to represent the user;
  • accepting that change will happen, and embracing it;
  • and the discipline of saying that some features won’t be developed;

and we have a recipe for success.

TDD

Good developers always understood that testing was essential, and did it. But they used to follow a written test plan, or just have an idea of what needed to be tested and work to that. Testing was manual, hard to repeat, and error-prone.

TDD brought automation. So instead of writing a document listing the required tests, we can write code. That inherently makes the tests rerunnable, so regressions get caught before they become a problem.

But almost more important than that is the idea of writing the tests first. In an ideal world you write a comprehensive set of tests, write functional code until all the tests pass, and you’re done. It may not always work out exactly like that — in particular, adding tests to a mature codebase can be problematic — but writing tests first encourages us to write code that is easy to test, which tends to lead to better-designed, more modular code.

An added bonus is that the tests can help to document the code, by showing our expectations. And of course they make refactoring easy and safe, as long as they are in place before you start.

If you were to start your last project over again, what would you do differently?

The project I’m thinking of involved rewriting the product’s GUI into a modern, responsive, browser-independent form, using HTML 5 and Twitter Bootstrap.

The existing version was an old frames-based web app that only worked fully in Internet Explorer, and had to be tweaked when each new version of that browser came out. We had long wanted to modernise it, but there were always other demands on development time.

Eventually I got a chance to try a proof of concept for the change. The application uses JSPs and Struts action classes, and the brief was to continue using these as much as possible. I decided to start with one of the main display pages, the one that users spend most of their time in. The idea was to give a quick demonstration of what was possible; and it did, to a point. But what I hadn’t realised was that frames are not part of HTML 5. There are ways to keep using them, but it’s not easy, and not good practice.

So while the new look and feel of a single page was clear, it was far from clear how the various pages would interact, how they would be brought together to form the whole UI, without frames.

If I were to start the project again now, my first step would be to work out how to link the pages together into a single interface, in the absence of frames. Most likely I would use one or other of the forms of JSP includes.

However, if there was the budget to do a more complete rewrite — by which I mean one that did not necessarily seek to use the existing JSPs — I would probably make much greater use of JavaScript and Ajax, and use the action classes just to provide data to the Ajax calls.

What is your approach to testing, and how would you test your application?

I would use a mixture of automated unit testing using JUnit, automated GUI testing, and actual user testing, if at all possible.

This fits well with what I was saying above. There are, broadly, three levels of testing: unit, integration, and system. Though writing automated unit tests is a development activity, rather than a testing one. Certainly we wouldn’t expect dedicated QA testers to work at the unit-test level.

So let’s assume that we have satisfactory unit-test coverage and we are interested in testing the application as a whole. Automation is obviously key here, as well, both because it allows us to easily repeat the tests regularly — for every checkin, in an ideal world (and see below); and because it removes the need for testers to manually step through a written script, which is boring and error-prone.

I have used Selenium for automated GUI testing, with some success. It takes a significant amount of development work, because it’s doing a significant thing, but the effort should pay off.

However, even after all that, there is still no alternative to having someone sit down and actually use the application. Automated testing might pick up outright errors in how the user interaction works. But it won’t catch fine details like misaligned elements, typos in onscreen text, or just generally how it feels to use the application.

What are the benefits of Continuous Integration?

Continuous Integration takes us beyond the traditional daily build. It does more than just building, and does it more frequently than just daily.

At the simplest level it ensures that, for every commit, an incremental build of the complete product is made, and all the unit tests are run. In the most advanced case, as well as building and testing, the product can be deployed to test servers and integration tests such as the automated GUI tests mentioned above can be run. Realistically those tend to take longer, so it’s unlikely that you would do them for every commit, but they can certainly be run multiple times daily.

So we get the following benefits:

  • frequent builds catch problems in code integration;
  • unit tests are run frequently, catching any regressions;
  • integration tests are run regularly, catching other problems;
  • general confidence in the product is increased;
  • developers are happy to commit changes frequently.

Tip: using Pandoc to create truly standalone HTML files

If you’re using the excellent Pandoc to convert between different document formats, and you:

  • want your final output to be in HTML;
  • want the HTML to be styled with CSS;
  • and want the HTML document to be truly standalone;

then read on.

The most common approach with Pandoc is, I think, to write in Markdown, and then convert the output to RTF, PDF or HTML. There are all sorts of more advanced options too; but here we are only concerned with HTML.

The `pandoc` command has an option which allows you to style the resulting HTML with CSS. Example 3 in the User’s Guide shows how you do this, with the `-c` option. The example also uses the `-s` option, which means that we are creating a standalone HTML document, as distinct from a fragment that is to be embedded in another document. The full command is:

pandoc -s -S --toc -c pandoc.css -A footer.html README -o example3.html

If you inspect the generated HTML file after running this, you will see it contains a line like this:

That links to the CSS stylesheet, keeping the formatting information separate from the content. Very good practice if you’re publishing a document on the web.

But what about that “standalone” idea that you expressed with the `-s` option? What that does is make sure that the HTML is a complete document, beginning with a `DOCTYPE` tag, an `` tag, and so on. But if, for example, you have to email the document you just created, or upload it to your company’s document store, then things fall apart. When your reader opens it, they’ll see what you wrote, all right; but it won’t be styled the way you wanted it. Because that `pandoc.css` file with the styling is back on your machine, in the same directory as the original Markdown file.

What you really want is to use embedded CSS; you want the content of `pandoc.css` to be included along with the prose you wrote in your HTML file.

Luckily HTML supports that, and Pandoc provides a way to make it all happen: the `-H` option, or using its long form, `–include-in-header=FILE`

First you’ll have to make sure that your `pandoc.css` file1 starts and ends with HTML `

Then run the `pandoc` command like this:

pandoc -s -S --toc -H pandoc.css -A footer.html README -o example3.html

and you’re done. A fully standalone HTML document.


  1. It doesn’t have to be called that, by the way.