Rasmus proved how hard it is to stay simple when writing an application or a library, and explained that the main victims of complexity are scalability, performance and security. He used a good number of frameworks to prove his point, showing the overhead in performance for an “Hello world!” application along with the graph of all the included files for generating such a simple request. Of course your application will be more than just an “Hello world!” but this shows just how much is loaded by the various frameworks for doing such a simple task.

I found the results to be frightening. So I looked at how simple our framework is compared to the others.

The first day

After firing siege, I got a very bad result: only around 120 trans/sec when PHP alone did more than 4000 trans/sec. It wasn’t surprising however, considering no performance optimization was ever done in the project. With the project reaching beta soon, it’s time to change this and multiply the speed.

You can take a look at the inclued file from that day:

Optimizations

Firing up the XDebug profiler, we could observe this call graph:

It’s easy to see what takes time… The weeAutoload::addPath method takes 76% of the execution time, and the configuration file loading 12%. Both can be cached and are only subject to change in a development environment or when upgrading, so we can automatically cache them when DEBUG mode is off and ignore the cache otherwise.

The cache method used is pretty simple. The cache files have no expiration time, they’re just deleted when upgrading an installation to a new version. If something goes wrong, the global constant NO_CACHE will happily ignore the cache files. And in development, cache is ignored as long as you have DEBUG enabled, which you should have all the time when developing.

The first time the script runs the data (an array in both case) is written to the cache file as valid PHP using var_export. Subsequent runs will simply require this file to load the cache. To prevent security problems, the file mode is set to 0600, disabling its access to anyone other than the webserver’s user.

Today

Using siege today I get these results:

essen@karen (0) % siege -c 5 "http://localhost/wee/trunk/" -b -t30s > /tmp/wee-today
** SIEGE 2.66
** Preparing 5 concurrent users for battle.
The server is now under siege...

Lifting the server siege...      done.
Transactions:		       40995 hits
Availability:		      100.00 %
Elapsed time:		       30.16 secs
Data transferred:	        3.44 MB
Response time:		        0.00 secs
Transaction rate:	     1359.25 trans/sec
Throughput:		        0.11 MB/sec
Concurrency:		        4.97
Successful transactions:       40995
Failed transactions:	           0
Longest transaction:	        0.04
Shortest transaction:	        0.00

Not bad at all, considering all we did really was to cache the autoload and configuration data automatically.

The inclued graph looks like this on the first load:

And like this when the cache files have been generated:

As you can probably guess, we won’t be able to improve much more on this part of Web:Extend.

Comparison with other frameworks

I computed a table with results from a few other frameworks. I used the same method as Rasmus used in his presentation. There’s probably a few frameworks missing, because they’re too complex, because they require me to set up a database, or because I missed them. If you would like one added to this table, feel free to give me its name and instructions on how to write an hello world page with it.

For those who really think an “Hello world!” proves nothing, I also added an example of Web:Extend with a few queries for comparison. The queries include an INSERT statement recording the hits in a table, along with an UPDATE statement and a SELECT statement both working on a 100K rows table. The UPDATE sets one column of one row to NOW() while the SELECT randomly selects 25 rows and display them in the template. This is not a prepared statement so it can be faster.

Tests were done with Ubuntu’s default PHP configuration file, on Ubuntu.

A few notes about ZF: since it’s the only framework requiring the setting of include_path, it was tested separately with its library path configured specifically for these tests. The include_path started with the path to the library. Other tests didn’t have any modification of the configuration.

I was kinda surprised to see Kohana being so slow compared to CodeIgniter. As you might not know Kohana is based off CodeIgniter. Maybe there’s a reason? If you know something that could make Kohana slower than CodeIgniter, tell me and I can run the test one more time.

As you can see, there’s a huge difference in performance today between Web:Extend and the other frameworks. Performance-wise, if you’re running PHP5, Web:Extend is your best bet. If you’re still running PHP4, CodeIgniter will get the job done.

Tomorrow

There’s still a few things we want to improve in our framework in the next few weeks.

Lazy-loading of the application’s modules

There’s currently 3 application modules: output, database and sessions. We already do a lazy start of the output module, but not of the 2 others. This means that the application will always connect to the database, even when no query are performed. Or that the application will always start the session, even when you don’t need it. Unless of course you didn’t activate these modules in the configuration file at all.

With lazy-loading of the modules, we’ll be able to connect to the database only when weeApp()->db is called. Or start the session only when weeApp()->session is called. The modules won’t be initialized at all if they’re not required for the current page.

Send output to browser early

Currently the framework generates the templates after the request handling ended. Sometimes you might have some heavy operations to do, like logging, saving a search result, doing some maintenance on the database that the user doesn’t care about. We’ll make it possible to send the output to the browser as soon as possible and perform these other tasks without making the user wait.

Jayson Minard wrote about this recently at devzone: Do Not Make Users Wait for Things They Do Not Care About.

It might also be interesting to send parts of the page early, like the header of the HTML page.

Send cache early

It might be possible to send cached pages directly from the index.php file, which would be really fast but would not be a good choice for all types of applications. We’re still not decided on this. The pros are that the file would be sent very fast, without loading the framework: it would be almost as fast as fetching an HTML file directly. The cons are that it won’t be useable for anything other than simple pages, common to every users. This would be a good choice for vitrine websites or any page that doesn’t require user’s input. We probably will implement it, since our current cache is already “restricted” to common pages. And for other pages the best solution is to use APC or memcache anyway.

Integrate caching methods semi-transparently in the model

Allow the developer to cache the results of a query without having to worry about the caching mechanism used (be it APC, memcache or anything else). The framework would choose the best available on the system and provide a standard interface directly in the model classes. For example, instead of $this->query you could use $this->queryCache to cache the query automatically, with an optional timeout value passed as parameter. You could also use $this->deleteCache to delete the cache corresponding to the parameters given. It could be all the cache for the current model, or only part of it. We would of course provide an option to allow you to choose the caching mechanism used in case the framework doesn’t choose the one you want.

Automate scaling

We saw a few days ago how we could analyze all our SELECT queries in one script. This is a functionality offered by our framework today. But this is not enough. We’re still missing a data generator that can handle relational data generation (if you know a good one we could use, please leave a comment!); a builtin profiler to test queries and functions; and a tool to create reports based on the data generated by query analyze and overall benchmark. As you already saw, the analyze of the SELECT queries returns a PHP array. This wasn’t because of laziness, this was because we intend to give you a nice report of these analysis, complete with emphasis on the problematic values (at least for MySQL and PgSQL, anyway).

Simplicity is a philosophy

To quote wikipedia, Simplicity is the property, condition, or quality of being simple or un-combined. It often denotes beauty, purity or clarity. Simple things are usually easier to explain and understand than complicated ones. Simplicity can mean freedom from hardship, effort or confusion.

Code simplicity can be achieved by following a few rules:

1. Define code and naming conventions and stick with it.
2. Never load what you don’t need.
3. Refactoring duplicate code is only worth it if it is used at least 3 times.
4. Use the Standard PHP Library.
5. Use of the right tool for the job. Use SQL for querying databases, XPATH for XML files.
6. Prefer learning a technology than learning the libraries built around it.
7. Automatically cache what you can; if you can’t, provide an easy-to-use interface for caching.
8. Slow code often means high complexity.

And if you ever need guidance in the path of simplicity, ask this good man.

Thanks for reading.

Disclaimer: If you feel like exploring the framework then have fun, but remember that the documentation is still lacking and that some parts of the API may change before we reach beta. We advise you to wait for the beta release before trying to use it, unless you feel adventurous or just plain curious.

Today I’d like to start a series on website optimization. The order in which I will do things is not necessarily the best order to optimize a website. However it will be full of tips and explanations for every single aspect of website optimization. A better structured article will later be based off this series and put in Web:Extend’s documentation. We know, the documentation is pretty empty right now. We intend to fix this before we reach beta.

Context

I currently have a contract with a client whose website had huge performance issues. Our job was first to port his code to our framework, then optimize both the code and the server configuration. When he contacted me he had peaks of 130 registered users at the same time with loads between 15 and 30. Some pages could take up to 30 seconds to load.

The rewrite

So I did just that, port the code to use Web:Extend instead of various PEAR and non-pear libraries. At the same time we would remove the biggest problems in the database schema and in the queries. Let me get this straight: the website was slow not because it used PEAR; it was because the queries would sometimes select thousands of rows that would then be filtered by PHP, and because some operations generated a huge number of queries because JOINs were not used as often as it should have been. The use of the framework was motivated by the client’s will to move to PHP5 instead of the now deprecated PHP4, and this framework was chosen because I’m its main developer and I know it very well.

Once the code was fully ported and tested (fixing at the same time a few bugs in the original code), we put it on the production server. The load reduced to an average of 3 at peak times. That’s nice, but it can be better. I should point here that any user on the website produce an AJAX query (producing several SQL queries) every 10 seconds. That means at least 13 queries per second at peak times. It’s actually more, since that number does not include non-registered users.

The biggest point of slowness

The queries were not optimized yet.

We looked into it and decided to first try to find the biggest performance problem. The chat functionality was the biggest problem. Whenever someone posted a new chat message, a noticeable delay would happen. All chat messages since the launch of the website were stored in one table… containing about 1.5 million rows. The table was big and not properly indexed nor optimized (it’s still not perfectly optimized but I’ll get back on this point later). Most chat messages in this table were never used since they were read and long forgotten. We decided to archive the table periodically (deleting wasn’t an option). A copy of the table was thus created and a cron job would run the PHP code to archive all the read messages sent more than 15 days ago and all unread messages sent more than 3 months ago. This gave us a nice 150 thousand rows on the main table instead of the 1.5 million we started with.

This last point helped a lot; the slowness went away. But this didn’t fix everything. More queries needed help, and the database schema was still in need of huge changes.

EXPLAIN ALL QUERIES

This part is very technical. Do not hesitate to post a comment asking for answers or clarification. Thank you.

This brings us to the most interesting part of this post. I needed to analyze the SELECT queries used by the application. All the queries. My first thought was: is there a way to analyze everything in a completely automated way without having to copy/paste the queries or rewrite anything? If such a beast exist, I didn’t find it. So I wrote it myself.

This website uses MySQL. A good way to check whether your queries are optimized is to run an EXPLAIN statement. An EXPLAIN statement is basically just a SELECT query with “EXPLAIN ” put at the start. Instead of running the query, it will analyze it and return various informations about how MySQL will run it. This basically means that I just have to put “EXPLAIN ” at the start of all my SQL queries to get a full report on all the queries. And this is basically what I did, but without changing the application’s code.

In Web:Extend, we used the concept of sets to define the model part of applications. A set contains elements, just like a table contains rows, or like a path contains files, directories or links. You can retrieve either “exactly 1″ or “n (with n>=0)” elements from a set. Let’s take an example: the users set. This set have one user with nickname "essen", and 42 users with the "invisible" option activated. In our framework, "essen" is an individual element while the "invisible" users are a subset of the users set. Mathematically, "essen" is both an individual element and a subset, but for the sake of simplicity we consider it to be nothing more than an individual element.

Web:Extend defines base classes for both a set and an individual element. All operations performed on a set (e.g. all SELECT statements, along with the UPDATE and DELETE operations potentially affecting n rows) are written in the set class; all operations performed on an individual element (e.g. INSERT, UPDATE, DELETE affecting only 1 row) are written in the element class. When a set returns elements, it creates the correct instance for the element associated with the given set.

You probably understood by now that all SELECT queries are stored in only one type of class, the set classes, and that these classes do not contain any overhead related to the handling of individual elements. In other words, we’re free to instanciate them at will and call its methods without any worry. Most methods are of the type count, fetch or fetchAll, which respectively count the number of elements in the set (or in a subset), retrieve exactly 1 element and retrieve n elements.

All we have to do now is put an “EXPLAIN ” at the start of all these queries. Now that we saw how it works, it should be quite easy, right? As long as we resolve the following problems:

1. When fetching exactly one element, the model will return exactly one row and not the result of the query containing all the rows returned by the EXPLAIN query.
2. When counting rows, the row is fetched from the database and the value returned directly.

Let’s take this code for example:

class demoUsersSet extends weeDbSet
{
	protected $sModel = 'demoUsersModel';

	/**
		@return int The number of users.
	*/

	public function count()
	{
		$a = $this->getDb()->query('
			SELECT COUNT(*) AS c FROM users
		')->fetch();

		return $a['c'];
	}

	/**
		@param $iUserId The user identifier.
		@return demoUsersModel The requested user.
	*/

	public function fetch($iUserId)
	{
		return $this->getDb()->query('
			SELECT * FROM users WHERE user_id=? LIMIT 1
		', $iUserId)->rowClass($this->sModel)->fetch();
	}

	/**
		@return weeDatabaseResults All the users.
	*/

	public function fetchAll()
	{
		return $this->getDb()->query('
			SELECT * FROM users
		')->rowClass($this->sModel);
	}
}

We have the array access in count and fetch methods to bypass. We are going to bypass them by writing weeDatabaseResult child classes that will always return $this for both array access and fetch methods. Let’s call the resulting class weeExplainSQLResult (no pun intended).

This won’t be enough, however; we still need to tell the database driver used by the model to return “EXPLAIN” result objects instead of normal results. We can do this simply by extending the database driver classes, make the required changes, and tell the model classes to use that driver for their queries. Extending was easy for both MySQL and PgSQL.

We finished writing the classes that will bypass the execution of all the queries in the model and execute EXPLAIN queries instead.

All SELECT queries explained

Let’s try it!

$oDb = new weeExplainMySQLDatabase(array(
	'host'		=> 'localhost',
	'user'		=> 'wee',
	'password'	=> 'wee',
	'dbname'	=> 'wee_tests',
));

$oSet = new demoUsersSet;
$oSet->setDb($oDb);

foreach ($oSet->count() as $aExplainRow)
	var_dump($aExplainRow);

foreach ($oSet->fetch(42) as $aExplainRow)
	var_dump($aExplainRow);

foreach ($oSet->fetchAll() as $aExplainRow)
	var_dump($aExplainRow);

When ran, this code will output this:

array(10) {
  ["id"]=>
  string(1) "1"
  ["select_type"]=>
  string(6) "SIMPLE"
  ["table"]=>
  NULL
  ["type"]=>
  NULL
  ["possible_keys"]=>
  NULL
  ["key"]=>
  NULL
  ["key_len"]=>
  NULL
  ["ref"]=>
  NULL
  ["rows"]=>
  NULL
  ["Extra"]=>
  string(28) "Select tables optimized away"
}
array(10) {
  ["id"]=>
  string(1) "1"
  ["select_type"]=>
  string(6) "SIMPLE"
  ["table"]=>
  string(5) "users"
  ["type"]=>
  string(5) "const"
  ["possible_keys"]=>
  string(7) "PRIMARY"
  ["key"]=>
  string(7) "PRIMARY"
  ["key_len"]=>
  string(1) "4"
  ["ref"]=>
  string(5) "const"
  ["rows"]=>
  string(1) "1"
  ["Extra"]=>
  string(0) ""
}
array(10) {
  ["id"]=>
  string(1) "1"
  ["select_type"]=>
  string(6) "SIMPLE"
  ["table"]=>
  string(5) "users"
  ["type"]=>
  string(3) "ALL"
  ["possible_keys"]=>
  NULL
  ["key"]=>
  NULL
  ["key_len"]=>
  NULL
  ["ref"]=>
  NULL
  ["rows"]=>
  string(1) "3"
  ["Extra"]=>
  string(0) ""
}

By applying this on all the queries of our application we will get the complete analyze of all our SELECT queries, available for review and optimization to anyone working on the project. These “EXPLAIN” classes will allow us to test the results of changes on the database over all the queries rather than looking at them one by one and possibly missing things by making changes that affect a query we already looked at and fixed.

Feel free to use, improve or adapt this method in your projects. If you do, please let us know about it so we can use your improvements.

Conclusion

We’ll take a break from the query optimization in the next post of this series to talk about Simple is Hard and its implications for our framework, our thoughts about it and what are our plans to improve performance as a whole in the framework. I’ll only say for now that our framework performs really, really well. 3rd part will talk about query optimization and the importance of database design, highlighting errors and explaining good practices.

Thanks for reading, see you soon.

Disclaimer: If you feel like exploring the framework then have fun, but remember that the documentation is still lacking and that some parts of the API may change before we reach beta. We advise you to wait for the beta release before trying to use it, unless you feel adventurous or just plain curious.