In the past, am talking about the year 2000, when we designed websites or apps (we simply called them applications), we didn’t have broadband. We knew that the bandwidth consumed by users was not ours to enjoy. So we had to be careful and optimize everything that went onto the website. The images we optimized via tools, people crushed PNGs, wrote optimized code, deleted variables that we didn’t need, didn’t create objects unnecessarily. Everything mattered. Information mattered more, not just style.
It’s quite essential that the clients get the information that they request quickly. Every time you click a button in an app, you are probably requesting a web service to ask for information. For e.g.
When you click your friend list in Facebook, you are asking the facebook web services to provide that list to your app.
When you check the weather in an app, you are asking for the current weather conditions by probably sending the current location in the request.
In games whenever your game shows info about your profile, updates your HUD, it fetches information most probably from a web service.
All said and done, the web services can sometimes become quite heavy to download and will cause a timeout if
- There are too many users connected at the same time to the web service
- The XML/JSON returned by the web service is quite heavy. Remember, sometimes even 30 KB is quite heavy when your clients are not having good connectivity.
- All the above
In this tutorial, I will attempt to compress data that is sent via a web service and try and save time and size. Try this with your own apps and games and you will see lesser connection drops and failures.
Note
Important
Tip
Warning
Help
- Well, sometimes users complain about constant timeouts. Developers will claim that the web services are well architected and don’t need optimization.
- Frequent disconnects could occur at user end, especially on mobile devices.
- Increase in costs, especially for bandwidth consumption. To accommodate the growing number of hits, hosting providers might be asked to increase bandwidth, resulting in additional costs.
Solution
Whenever we used to send files earlier, before the advent of drive and solutions like Dropbox, we would send large files by zipping them. Why cannot we do the same with data? Data if zipped can travel faster across the same medium.
Well TCP/IP layers offer their own form of compression so that information is delivered faster across the internet. But imagine if you zip it further, your information would travel further. Over here I will be providing a simple walk through on how to achieve the same with the help of PHP and AS3. Some investigative statistics of compression v/s non-compressed methods are also provided.
So let’s begin.
Writing a web service
We will begin with creating a simple web service. I will be using the sample books.xml found from Microsoft’s website.
<?xml version="1.0"?>
<catalog>
<book id="bk101">
<author>Gambardella, Matthew</author>
<title>XML Developer's Guide</title>
<genre>Computer</genre>
<price>44.95</price>
<publish_date>2000-10-01</publish_date>
<description>An in-depth look at creating applications
with XML.</description>
</book>
You will get the source files by clicking the link below. The flashsrc folder in the zip file contains two projects, you may want to use FlashDevelop to run them.
You could use this XML or create your own or read from a database and generate XML, your call. This is what the web service will do.
Case 1
In this case, the web service will return the XML in uncompressed form.
<?php
header("Content-type: text/xml");
$file = 'books.xml';
$xml = file_get_contents($file);
echo $xml;
?>
The idea is to print the XML whenever this PHP is requested.
The idea is to output the XML whenever the web page is requested. We will check the download time from a client as3 code. Host this PHP and the XML file on a web server. For test purposes, I’ve hosted in a WAMP server. You could download one here.
You can find the as3 source code below to fetch data from the webservice.
package
{
import flash.display.Sprite;
import flash.events.Event;
import flash.events.TimerEvent;
import flash.net.URLLoader;
import flash.net.URLRequest;
import flash.utils.Timer;
/**
* ...
* @author Elvis Fernandes
*/
public class Main extends Sprite
{
private var theXML:XML = new XML();
private var theURL:String = "http://localhost/xmlwithoutcompression.php";
private var theRequest:URLRequest;
private var theLoader:URLLoader;
private var startTime:int;
private var endTime:int;
public function Main()
{
if (stage) init();
else addEventListener(Event.ADDED_TO_STAGE, init);
}
private function init(e:Event = null):void
{
removeEventListener(Event.ADDED_TO_STAGE, init);
// entry point
theRequest = new URLRequest(theURL);
theLoader = new URLLoader();
theLoader.load(theRequest);//load the webservice
startTime = (new Date()).getTime();//storing time the request was made by the client
theLoader.addEventListener(Event.COMPLETE, onXMLLoaded);//fired when the xml is received
}
private function onXMLLoaded(e:Event):void
{
endTime = (new Date()).getTime();//store the time when the xml is received
trace("finished loading took " + (endTime-startTime) + " ms");
//trace(theLoader.data);
}
}
}
Case 2
Now, in this case, the web service will return the XML in compressed form.
<?php
header("Content-type: text/xml");
$file = 'books.xml';
$xml = file_get_contents($file);
$data="<?xml version='1.0'?>";
$data=$data."<data>".base64_encode(gzcompress($xml))."</data>";
print_r($data);
?>
This PHP will generate an XML file with the following output. You can try accessing the URL by hosting the XML and PHP files in localhost.
<?xml version='1.0'?> <data>eJy1WE1z2zYQvXem/2HHPeQQyZJsJ6kzijK2mjhp7MYTuW1uHYhYihiDAAOAUtRf3wfS+kgsZqg0PtkkPrh47+HtroYvP+ea5uy8subFweCwf/By9PNPw0QEoe0M/xLRcGrtLSn54mB6O+gPDuq3cUCUIbNudCHyqXCStRYduhIhZLwY9u4G15ODCppHH68u6Tees7YFu0eeLkoledirB9dzZ2wcj8Y2L8rAbtirn9fDhVMJj05ODk+fDHv1w2asnGrls3+kCDw66vf73UG/2x9g3vbAerpknzhVBJx/dGZIma7kImSk45lFoMSxCMrMSBSFVsAFEz2tli8UpuJIh8Pe9kY1bL2I2y4Ij+5D+EFo7Tv0TuVNwF0padQsC/RBKNMA2GthgvDLBryetITrqDt42gouSq3L2ZFwSaYCJ4GmIF+zp8S6wjqspH9tPlWMo622EIZ4rjR56xJ27DEkjKQM+9iFIeykZWatpGBpyonNmT6VzGa9gU0JAqOFdVruh/vxfdzH1nFACK/mohF4AbnSmU/YSOD7oNAPuoNnraBPcS0qGBKLW1d4jrAIMsJYEJEZi9u7XEPmbaI4LKFuemVmGnh3qsVLW0LZvnRzNbfOkxbL6v1qXYpxWUk+Uh3358Vqs/2gP/ku6N9P2VkDl7jkmUiWDwf9oNs/hk+0gf6tocL60BWFTYReRui/wDRfenCjbLlxCYHAAt2aqG9r9JKEp/pokL0ufJR65TN8B7FW6TYJNdPKZGKqAg7rI9mX1khrDmnCuB46blELdbVsLdj9iHryXUTdIL4JtIILLenCCaUflKvTllzFsMLCkhQlnBNZLgJXwdShTOi061Uka8udagMDS6ACVym6UsQ/sSY4q+PyO7K3gV8t/kqu+wH/dFdSAIFad2i8NCFTjehfWmRwOldO+gbYP9hcmISbEmk7dwLs/aM2sP+dwa7HwmlBOUM6dC1KHZMpvN86g5z5pT0B3RTKQYC4QpwXkai0viYpbkUWn2ccyJVpqnlP0392H9ebrHTAo1PF1YjqpIjHJPwRPntIXOH67XA9I8lckGdBUkXKU2Uk3MOVjHplHrUOo9ngilOVPiZXzWJWIi1P2UQ8KzvBLvsB+et9IN8Z61yHJoFTYZpwHDsEvaSxEwuNSqAByTfYyTZWeq0rl367ysVAiisV4k5n1cfJB+sQIYmpLaE2K5IMlctapcBWFSxjMeNjgVNVglXxUhuFMh41kN8P1dMdqNZV0TVXpe9uUK+FE9J+RhbwTRXJBGka2qTXKomBNED79EdrtKpMRKykhYRGY76CQlUB0TkLG0aKe8PKs4FZzlZb/IlAMVVF9aI7UNEJfhc5uJgIvUBVIhUgj31Kpd3C2anm3G/XhFOOVfqnEt8r8704GPTvc/D+0TmkAIu4+VZFnjjrbRro8I9XN88pZptrZ2dO5HmM5VwhyO9rbY7b0gLFn7ahZR0rslOMFvSooFDawTWUJ/5caBtzt9rU2ZJBiI5vkHt8bT3F3fGq1s3xnWvvh/aOBrIV2hP0WcfP6Ywido6RY3yM/n80kPug3KqBjArYqKKOmArhfHSH2BXNeSfInbqRRO0RPQZlIYqKqlF+f4XJOEAqkmg7HyeXN5EDPHgobFO2TM4+VkaUg8T92NjRi16g9cfHrtQtN5Hxl/Kl0DD+UipLz34oKyenLavAk5Z96oaRr8L+WvlV57/2+/gLQNX4B+SHxWrtufAq6ayexo8fozr7pW5hzybXj9ELb0rwSN0sdsFxd1AqoIBtkGT9O0geu4PVGjZzhSIyvvsmkcPe6jea/wBisG55 </data>
We can now read this only after decrypting it. A sample as3 code is given below.
package
{
import flash.display.Sprite;
import flash.events.Event;
import flash.events.TimerEvent;
import flash.net.URLLoader;
import flash.net.URLRequest;
import flash.utils.ByteArray;
import flash.utils.Timer;
import mx.utils.Base64Decoder;
import flash.utils.CompressionAlgorithm;
/**
* ...
* @author Elvis Fernandes
*/
public class Main extends Sprite
{
private var theXML:XML = new XML();
private var theURL:String = "http://localhost/xmlwithcompression.php";
private var theRequest:URLRequest;
private var theLoader:URLLoader;
private var startTime:int;
private var endTime:int;
public function Main()
{
if (stage) init();
else addEventListener(Event.ADDED_TO_STAGE, init);
}
private function init(e:Event = null):void
{
removeEventListener(Event.ADDED_TO_STAGE, init);
// entry point
theRequest = new URLRequest(theURL);
theLoader = new URLLoader();
theLoader.load(theRequest);//load the webservice
startTime = (new Date()).getTime();//storing time the request was made by the client
theLoader.addEventListener(Event.COMPLETE, onXMLLoaded);//fired when the xml is received
}
private function onXMLLoaded(e:Event):void
{
endTime = (new Date()).getTime();//store the time when the xml is received
trace("finished loading took " + (endTime-startTime) + " ms");
var theXML:XML = new XML(theLoader.data);//retrieve the data
var b64:Base64Decoder = new Base64Decoder();
b64.decode(theXML.toString());
var bArray:ByteArray = b64.toByteArray();
bArray.uncompress(CompressionAlgorithm.ZLIB);
//trace(bArray);
}
}
}
This is it, we are done with the code. Now off to testing how much truth is in it. We need to really evaluate whether compressing XML data actually helps transfer files faster.
Results
To test the web services, I used Apache JMeter which is quite a versatile tool and provides detailed information about your web service and time taken for data retrieval. I will not be covering how to use Apache JMeter in this post, maybe in another post.
Click the above link to download a zip containg the jmx file used by Apache JMeter. For the first test case, I tested both the web services locally.
[table id=1 /]
And another test
[table id=2 /]
As you may note, in the above case, compression is the winner, Bytes transferred over HTTP were almost double that of with compression. But we didn’t save much when it comes to the time taken do download. Each sample is around 2-3 ms. This isn’t the case with the AS3 client, though. In an actual web server (not localhost), the time taken to download is huge and quite different, hence we cannot conclude based on the above results. I then hosted the web service on an actual web server. The test results are given below.
[table id=3 /]
And as you see below, the web service without compression is not fairing well.
[table id=4 /]
This is despite the fact that mod_deflate is enabled in the Apache server. The mod_deflate module provides the DEFLATE output filter that allows output from your server to be compressed before being sent to the client over the network.
My Verdict
This is what the aggregate graph will look like in JMeter.
[visualizer id=”723″]
The average time taken for compressed data to travel is 107ms while that for uncompressed web service it takes around 211ms.
The compression method is a winner already. The time taken for uncompressed web services to reach the client increases drastically when the size of data is huge. You could try it yourself. Create an XML file about 10MB in size and check it out yourself.
Conclusion
There are pros and cons with both methods, compressing data would make the data unreadable via tools like JMeter usually used to conduct tests. But it does offer some level of protection. A word of caution, I do not suggest to use the same kind of compression that I’ve suggested. Also, one should keep on changing the compression algorithm time and again.
Till next time, bye bye.
0 Comments