Just a quick note on what is going to be posted in the next few weeks – I’ve got a few significant pieces in mind for various topics – including:

  • Doing the work elsewhere – asynchronous queues

This is going to be a series of articles – and to support it, I’m rewriting some code that I had originally wrote for my last job (v2, and so significantly improved over the original). First though, I’ll tell you what is planned, and just how asynchronous queues are used and how they can be incredibly useful for scaling up any significant website, and not just in the obvious ways.

  • Mail queuing, on a vast scale

Dave Marshall has just posted an entry on Using message queues to improve user experience where he queues up some emails in order to spool them out over the course of a few minutes.
For the last 18 months, I’d been doing something very similar, on a far larger scale, with PEAR’s Mail_Queue.
I’ll show you how I did it and I’ll show you how the messages were generated quickly, and how they could be sent out – and more importantly without destroying the system it was running on. As a bonus, I’ll show you how if you run some form of internal mail system, you could save gigabytes of database space and give yourself vastly more flexibility.

  • Self stubbing mocks

Using the Mocking functionality in PHPunit & Simpletest can be complicated with the various calls that are required. There’s just not much documentation around the PHP-world on how to run it.
Another method, which can be easier to understand, is self-stubbing – putting your code into the test class. I’ll show some examples of how to do that.

Finally, I’m going to be doing my ZCE exam in the next week or two – quite possibly on Thursday 3rd June (2009). Keep a close eye here for the results, and a follow-up.

Tagged with:

Back at the PHP London Conference at the end of February, iBuildings was offering a little test, with prize for people that could do well answering the sort of questions that are on the ZCE exam. Never one to turn down something useful for free, I took ten minutes to answer the eight questions. A few weeks later, I get an email from them/Zend to say I’d won the chance to take an exam – ZCE, or ZFE (Zend Framework). Although I use ZF, I don’t know it well enough to begin to pass any exam, so as I’ve still not had the chance to take it, I figured, why not take it on their dime?

About 14 months ago, I’d bought 5 tries on the PHPArch-based ‘Vulcan’ test prep exam. Today, I’ve come back to it, and gone through it again. Like last time, the test (practice and real) is scheduled to take up to 90 minutes, but I had whipped through them all in 45 minutes, I have finished the 70 questions.

I’m amused by the fact the only part of this I failed was ‘Basic Language’. The first time around it was design patterns. Either way, now I’ve got some time, I’m going to schedule the test for quite possibly later this week and see about getting the paperwork for it.

It’s also still 7 ‘EXCELLENT’s, and a fail – just in different places 🙂

Category Grade
XML & Web Services PASS
Arrays PASS
Web Features EXCELLENT
Basic Language FAIL
Streams and Network Programming PASS
Database Access PASS
String Manipulation and Regular Expressions EXCELLENT
PHP 4/5 differences EXCELLENT
Security EXCELLENT
OOP EXCELLENT
Functions EXCELLENT
Design EXCELLENT

Overall : EXCELLENT

Tagged with: ,

Well go figure. I’ve just won $50 (Canadian, that’s about $3000 USD by now) of books and ‘stuff’ from PHP Arch, care of its publisher, Marco Tabini’s, blog.

He’d put a little puzzle up last night, some long numbers, and a few short. I recognised them as almost ISBNs – it wasn’t hard to figure them as having dropped a zero from the front, making them “php|architect’s Guide to Programming with Zend Framework” and “php|architect’s Zend PHP 5 Certification Study Guide, 2nd Edition”. From there, guessing the other numbers were page, line and word counts was easy.

So, what should I buy? I’ve already got a subscription to the magazine – PDF edition (it’s so much easier to ship bits over the atlantic…).

As I mentioned in my second post, ZCE prep – and dumb tests – about open book tests (like Brainbench), having a copy of all the relevant documentation can be incredibly useful, if only from a speed issue. Knowing you can just open a new tab and type a few words to get the information on a function, or concept from the manual takes away so many problems.

I mentioned there that I have a local copy of the main PHP manual – and I wanted to tell you how I keep it, and a couple of other manuals up to date, as well as other documentation.

Mirroring PHP.net

It took a little exploring to first find it, but there is a page on how to get make a mirror of the site – http://www.php.net/mirroring.php Although they aren’t looking for more mirrors (unless your country doesn’t already have a couple around), there’s no reason you can’t make a private mirror for your office, or just for yourself, as long as you don’t update it too often and so cause excess load on the source website. I have a cron-job to sync my own copy once a week on a Sunday afternoon – when I think it was fairly quiet.

If you have a linux server (or a Mac OSX would be able to do it as well I expect), and Apache already running, then setting up a virtual host and dragging down a local copy isn’t hard. You don’t need Mysql running, but if you have SQlite3 installed, you will be also able to use the quick-lookup facility – very handy if you can’t remember the order of parameters for strpos – http://php.net/strpos).

Here’s my script I use – taken almost verbatim from the above mirroring.php page:

It may look a little more complicated than it it – but in short, it goes to the server at rsync.php.net, and gets the English version of the manual, but not the others, and it also doesn’t bother downloading the large source files. In fact, the two largest parts of the site that is mirrored are the manual itself (currently some 44MB) and the manual backend (which includes all the notes, and most usefully, the lookup functions).

The first time you download the site, it’s going to take a while – it is, currently, 133MB (though the -z in the rsync command line will compress the bytes on the wire, so there is less to download for the same results). However, most of that does not change week to week, and so the rsync protocol comes into it’s own by only downloading parts of files that have changed (where they are sufficiently large to make a difference). Here’s an example:

  • sent 71838 bytes received 9768512 bytes 57045.51 bytes/sec
  • total size is 116855428 speedup is 11.88

Here, the total size it would have downloaded was: 116,855,428 byes, 111MB. However, it only downloaded 9,768,512 (9.3MB) – an over eleven-fold improvement for the same results.

Next time – What else you should be reading, keeping around for reference, getting updated automatically, and generating from your own codebases!

Well, I’ve just completed the PHP Arch ‘Vulcan’ practice test – the first of up to five such practice tests I’ve purchased. I have quite deliberately not gone through what study materials I have on hand before I took this test (I wanted to get a baseline), but none the less got an ‘EXCELLENT’ final score, and the same ‘Excellent’ on seven of the twelve sections the pre-test is broken down into. ‘Pass’ on four others, and just one ‘Fail’ on the design (patterns) section. If the real test worked much the same – and with a composite score, rather than having to pass all sections – I doubt I would have a problem to have gotten a passing grade.

Although the test (practice and real) is scheduled to take up to 90 minutes, after 45, I have finished the 70 questions. I’d set about half-dozen to review, but I don’t think I changed any of those answers, and so after a little more than 50 minutes – I called for the results.

It was much as I expected, with just some occasional requirements to know some parameter orders and specific use (the kinda thing where you test it, and if you didn’t get it right first time, I would trivially look it up to check, for functions like sub-string searching). Also several questions on XML handling, which I muddled along with.

Streams, strings and web-features are something I will have to look at more carefully for next time, but for me the big one is design patterns – it was my only failing section.

All in all, I’ve very happy with this evening’s events and the (practice) results.

Category Grade
XML & Web Services EXCELLENT
Arrays EXCELLENT
Web Features PASS
Basic Language EXCELLENT
Streams and Network Programming PASS
Database Access EXCELLENT
String Manipulation and Regular Expressions PASS
PHP 4/5 differences EXCELLENT
Security EXCELLENT
OOP EXCELLENT
Functions PASS
Design FAIL
Tagged with: ,

Extending PHP_CodeSniffer by Raphael Stolt shows how to quite easily add to a tool that will report what parts of your PHP source needs a clean-up, from the built in ‘sniffs’ for coding standards, and now adding to that for some slightly more opinionated choices on the maximum number of lines per function, or functions per class.

This could be a start of a whole collection of additional classes for additional checks.

The only challenge I see at the moment (and I don’t think it’s a big one, though I’ve not tried it, so it might not even exist, I’ve just not had the opportunity to look yet) is having to put code into the PEAR directory path, since that is where PHP_CodeSniffer is looking for the base library. I do see that they use long PEAR class-names, so it may just a matter of having the phpcs tool look elsewhere for the base coding style class.

[edit: 18:06]
Ahah, some investigation – and copious use of echos in the phpcs script, and /usr/share/php/PHP/CodeSniffer.php class:

Creating your own standards, and using them from anywhere on the filesystem – not having to link from inside the PEAR/PHP/CodeSniffer directory:

The coding standard class with your extension:
The class is called PHP_CodeSniffer_Standards_Example_ExampleCodingStandard – the two <em>’Example'</em>s being the name of the coding standard. The file is called ‘Example/ExampleCodingStandard.php’.

And in the subdir: …/Example/Standards/
– Raphael’s ToManyMethodsSniff.php and MethodLengthSniff.php files from his post.

To call it:

Tagged with: , ,

This week I’m going to take the first of my PHPArch.com’s ZCE prep test – then I’ll read the book and see they they expect me to know.

Going for the Zend Certification is something I’ve been thinking of doing for a couple of years, and especially now that it covers PHP5 – and increasingly good practices and security topics. It’s not that I need to get the ZCE, I’d go for it , for the intellectual challenge if nothing else. It’s also the closest thing I would have to a professional qualification since I completed a HNC computer studies in 1992 – and that was just 1/day week day release over the course of a couple of years.

Of course, it’s not the first PHP test I’ve taken – last year, just before the PHP London 2007 conference, Allegis had come along to the PHP groups’s early-February meeting, to plug their services (and they got business from it, one guy interview the following day, a Friday, and started work on the Monday) – but they offered to have anyone that wanted to do the Brainbench test, paid for, by Allegis.

At the time, I had just started a couple of days before at a job near Covent Garden, but then left it after a week for a better gig (where I still am now, some 13 months on) – but I took Allegis up on the offer, and finally got the results at the conference. I never did get a copy of the exact numbers, but I was told the headlines, so these may not be exact, but they are certainly in the ballpark.

  • Time: 28 minutes (apparently this is very good)
  • Score 4.73 (out of 5.0)
  • Better than 98% of other test-takers.

I’m told that the harder the questions you answer, the harder the next questions get – so getting from 4.0 to 5.0 is a lot harder than getting from 3.0 to 4.0 – if I’m wrong about that, then please let me know.

The thing is, the Brainbench tests are open-book – they pretty much have to be, you take them at home, though they are against the clock. I can certainly appreciate the logic of it – after all, which serious developer doesn’t have an internet connection and a quick bookmark to http://php.net (and I’ve look at the other language sites – php.net is by far the best), or at least a copy of the documentation around (.CHM file or just a bunch of HTML pages – or, like I do, a weekly rsynced copy of the php.net manual!). Just as well I did, the tests I’ve seen always throw in some pointless questions like how to use LDAP, or how to connect to an Oracle database. I’ve never used either of them, so I don’t bother to learn them – but if I did need them, I’d figure it out in a few minutes reading – or, more likely, I’d have some kind of library, like the Zend Framework which did the hard work for me – plus, I’d only end up writing that kind of code once anyway before I threw it into a function and forgot the minutiae.

It saddens me when people are too dumb to do well on such a test though – how hard is it to read the manual, at least well enough to know where to refer to for more advice?

The ZCE is a closed book exam – or, as I call it (for all the reasons the brain-bench is open-book) – unrealistic. If I can’t recall whether the $haystack or $needle come first in in array or string search – it’s but a moment to look it up.

Even though I’ve listed my scores above, I don’t bother to promote myself with them on my CV – indeed Allegis is the only company (recruiter or not) that know them – they did pay for it after all. Because I can get those kind of scores with what I consider so little effort (about half-an-hour’s worth in fact), then either the test is bad, or 98% of the other people that gave taken that test are. Frankly, I’ve got to think it’s mostly the latter.

Keep reading my posts, and I’ll tell you want you need to do to ace your tests – and not look a fool when it comes to developing something I might set you.

Tagged with: ,

I’ve read http://www.svn-checkout.co.uk/2008/01/19/how-to-release-new-versions-of-websites/ via http://www.lornajane.net/posts/2008/SVN-Deployment-and-a-New-Site and while I consider revision control an essential tool (a few years ago, my job was the only one in the previous five years where I didn’t have to install my own RCS), I somewhat disagree on the idea they suggest.

That first link, ‘how to release new versions of websites’, suggests checking out a version of the site as a working copy, (it’s certainly something I’ve done before now), but then it goes on to use the ‘svn switch’ capability to move between versions (of course, they are also doing in in TortoiseSvn, and there for the live web-server is likely to be running Windows, no way I’d run a server on Windows – not even for testing). There is however some trouble with revision switching – especially on a busy site that has to keep running even while the new version is being put into place. While SVN does atomic commits – the new code goes into the repository all at once, or not, it’s harder to do the all at once part on a non-transactional file-system – such as a webserver. The bigger problem is that rolling back will also take time – and in an emergency, the time taken to do something is crucial.

Here’s what I do – Whenever I want a new version to go live, which might be from every couple of days to as often as a couple of times per day, I’ll update run the script below with the specific revision number to export (and a date/time, but that’s just for easy reference). When it runs, it also symlinks the given version as ‘dev’ – which is part of the path to the site ‘dev.example.com’, wham, instant new version, and I can trivially delete the symlink and on the same command line symlink the older version (with ‘rm dev && ln -s 1234.20080102 dev’). After a quick wander around the newly checked out version, maybe run a set of unit tests, just for security, it’s just as easy to put a similar ‘live’ symlink into place with a new link.

If there is ever a problem, rolling back to a previous live version is just as easy. Other configuration changes are made within the code on the apache servername, or the machine’s own hostname (more useful for CLI scripts, generally run from cron).

There are some downsides – with a complete checkout of a 40-some megabyte website (it’s mostly the Zend Framework and other libraries from which I use a number of files, though rarely all), it takes a little while (not too long, it’s a gigabit link between the repository and the main webserver) and there are also some potential caching issues (Etags are usually based on the file inodes), but as we plan to move to a multi-machine cluster – and the images aren’t being served from Apache, but a dedicated image webserver, that’s not a significant issue – and even on Apache we don’t have Etags enabled (Yslow from Yahoo also suggests that).

Update:  Since I first posted this, I’ve started using Capistrano. Look out for new posts on how to best use that.

Tagged with: , ,