Many passwords in linux are encrypted using the crypt() utility. The user is usually not aware of the difference between a crypt and a MD5 encryption. Well it can turn out to be important, especially if crypt uses the default DES-based scheme to perform the encryption.

The problem with crypt() + Traditional DES is that it truncates the password length to 8 characters. Users are not usually aware of this and assume that the entire length of the password has been saved and encrypted. Take the apache tool htpasswd for example. It uses crypt() to encrypt passwords (It may also use its own MD5 routine) into a password file. The following command creates a new user in a password file

htpasswd password_file new_user

After this command is executed, you are prompted for a password. If the password is greater than 8 characters, for example – 123456789, it will still be accepted and no warning will be provided that it was truncated. So providing the password 12345678 will also allow you to be authenticated into the system. Why is this bad ?

• The time taken to crack 8 character passwords is shorter (in relative comparison).

• It is also likely that the password was truncated in these scenarios, so an attacker may well target passwords that are exactly 8 characters in length.

• Some people have the knack of prefixing the password with the username first. Bad idea if your username happens to be 8 characters long

• The user may not even be aware of the problem, since he/she assumes that the password is strong and greater than 8 characters.

So the next time you provide a password to a system, you might want to know how they get saved into a persistence store and what encryption is used.

I came across an interesting application today. Google has released an application named Jarlsberg that is full of security holes. The intent is to make developers learn how these holes work and put them in a position to combat the security vulnerabilities.

You can visit the app to learn more. Security flaws to be detected are classified under the following categories

• Black box. You dont know the code
• White box. Requires you to see the code to understand how to break it.
• Gray box. Some code will be made visible.

I also came across an instructor’s guide that has problems to be solved in the application, graded by their difficulty level.

What better way to learn an exploit than to perform it on a test system ? Some of the exploits involve

• XSS and related challenges
• Path traversal exploits
• DOS
• Buffer overflow
• SQL Injection

and so much more. Give it a try now

I use remote linux services often and exposing them as local services can be performed securely using SSH. For example you can access a tomcat server or email server hosted at IP 1.2.3.4 by opening a secure SSH tunnel between your local machine and the target address – 1.2.3.4.

The OpenSSH tool can be used to perform SSH related activities on your machine. Simply install it with yum, apt-get or Yast, if it is not already available. Once you have it use the following command to open multiple SSH tunnels to your services

Open tunnel and execute commands:

1

ssh 1.2.3.4 -lmyUser -L 3098:1.2.3.4:21 -L 3099:1.2.3.4:80 -L 3100:1.2.3.4:443

The command is explained below

1.2.3.4 – Your target IP

l – The user to login as

L – A local tunnel to a remote port

3098 – The local port to use when establishing this tunnel

21 – The remote port at 1.2.3.4 to which the tunnel will be established

Multiple tunnels can be opened by specifying multiple -L flags. For example http://localhost:3099 will now redirect to http://1.2.3.4:80/ That is a fancy way of saying all HTTP requests ( 80 is the default port ) for 1.2.3.4 can now be reached locally at port 3099.

Note that the above command will also log you into the remote system. If you want to open the tunnels alone, use the -N switch and the -f switch as shown below

Open tunnels only:

1

ssh -f myUser@1.2.3.4 -L 3099:1.2.3.4:25 -N

The -f switch asks SSH to work in the background and -N asks SSH not to execute any commands.

I logged into a gmail inbox today, surprised to find ‘Google Buzz’ asking me if I wanted in. I clearly said ‘No’. Not yet another social network. pfft. So after I said ‘No’ there it was sitting comfortably on the navigation bar and telling me people were following me and I could follow them.

There is a small link on the bottom of your Gmail page that says ‘turn off buzz‘. Click on that and you get disconnected from the social network. I wonder what google plans for Orkut.

After project owners were asked to move out of kenai.com, Oracle now admits that it did a poor job at communicating its plans for the project. An updated post from Oracle says that the future of kenai.com projects is secure. Those projects will be migrated to java.net and continued as is. Here is an extract from the post

We don’t believe it makes sense to continue investing in multiple hosted development sites that are basically doing the same thing. Our plan is to shut down kenai.com and focus our efforts on java.net as the hosted development community. We are in the process of migrating java.net to the kenai technology. This means that any project currently hosted on kenai.com will be able to continue as you are on java.net. We are still working out the technical details, but the goal is to make this migration as seamless as possible for the current kenai.com projects

Most developers seem to be moving out of kenai.com after being asked to leave. For those of you that do not want to move, there may still be hope

I ran into a weird little error while trying to tune the performance of a query in the oracle database. A table had an index on a VARCHAR2 column. After a fair number of inserts were made the population of this table grew to 1.5 million records. A query that did an join on the VARCHAR2 column was talking ages to get the results. Well, it was taking 900ms actually but the SLA for the API call is way below that.

An “explain plan for QUERY” statement was run followed by a select from plan_details. It turns out the index is never used and a full table scan was used to get the data. Hence the problem.

So all I have to do is find out why the index is not being picked up. It probably has to do with statistics, and a rebuild on the index might help I thought. Or may be I need to analyze the index. So I went about trying to do some of these things. There were no DBAs around so I had to experiment a little.

1
2
3
4
5
6

analyze INDEX YOUR_INDEX_NAME validate structure;
 
Error: ORA-01418: specified INDEX does NOT exist
 
SQLState:  72000
ErrorCode: 1418

This was totally weird. I knew that the index did exist and the name was correct. To double check I ran this query

1

SELECT * FROM all_indexes WHERE INDEX_NAME='YOUR_INDEX_NAME'

And it did return my index correctly. After trying several other commands related to indexes they all returned the same thing. I tried to prefix the SCHEMA name and that did not help.

So it turns out, I do not have permissions to create indexes or do any operations related to them and that is what throws this error. The error is pretty misleading since there already exists an error code for insufficient prvileges for performing a given operation – Error: ORA-00990: missing or invalid privilege.

I passed the index related queries on to a DBA and the performance of my application queries were back on track. I hope this saves a developer the time I lost trying to find out why the index did not exist.

PS: Oracle 11 also has an index visibility option which can be verified with

1

SELECT INDEX_NAME,VISIBILITY FROM USER_INDEXES WHERE INDEX_NAME='YOUR_INDEX_NAME';

A few days ago the EU had approved the takeover of Sun microsystems by Oracle. This brings to an end the long awaited acquisition. There have been several blog entries about the fate of the open source projects that Sun currently supports. I have read some of the arguments put forward about why developers should move away from / use certain projects.

However I also find myself in a dilemma. I installed GlassFish and deployed a few apps on it recently. I found it interesting and was impressed with it within a few minutes of using it. MySql is also a great open source project. There are many tools built around the database and it competes well with PostGresql. Netbeans is also a great IDE and supports glassfish.

I am very tempted to continue using these projects for the development of new pet projects and to learn new specs. Right now I expect the support for these projects to continue for a few years, and I do not really expect to see great innovation of any sort. The best case scenario is that Oracle leverages the usefulness of these projects and decides to keep them alive while giving them healthy financial support. One is allowed to dream

So it boils down to the following question. “Will you as a developer / architect / <Insert technical position here> continue to use projects like GlassFish, Netbeans, MySql etc ?” Or are you looking to move away to other open source projects immediately ? Voice your thoughts through this poll

Note: There is a poll embedded within this post, please visit the site to participate in this post's poll.

I have been a tad busy fixing some weird little bugs lately. They helped me appreciate the multitude of things that can go wrong in a live environment and served as a gentle reminder that you should always be on your heels.

Here they are

1. LDAP and the user

A web application product was configured to use an LDAP directory structure. The directory was segregated into roles / groups / OUs, the usual. One of the users had trouble logging into the product. This was weird because this person was a valid user and Outlook seemed to recognize him. Outlook uses the same LDAP tree. So I dug into it. The easiest way to check what is going on is to use a LDAP directory browser. I use the one provided by Novell for free. The LDAPs support is not great but it will do for basic lookups.

So as I dug into the tree, it turns out this user was mapped as a group. Yes a group. The product was configured so that only the “user accounts” LDAP directory was looked into for valid users. So since this user was, ummm a group, the application was unable to find him. My only thought was ‘wow ! how did this go under the radar for so long ?’. The mistake was understandable though since groups and users are under a similar looking structure. The LDAP admin must have had too many doughnuts at lunch and probably dozed through when configuring the user into the system

2. Authentication failure

This one involves the LDAP too. We built a web app recently that was supposed to check for users under a specific tree node. The logic was to bind to the LDAP and if that is successfull then it must mean that the user exists. Unfortunately the negative scenario to this logic was not tested all that well. The API did not throw an exception when the Bind failed. So this meant that I could login to the application without a password. This also meant I could use any user name I wanted to and login. Even one that does not exist. I was tempted to do some operations as MrBunnyRabbit76 but fixed the bug instead.

3. FTP and the CPU

Now comes this little gem. A FTP process had been scheduled in CRON to run every 10 minutes or so. Its job is to ensure that a local folder and a remote FTP folder are in sync with each other. The program used to do this is lftp with the R switch (for reverse). It so happens that the FTP account gets locked because it ran out of space. The account is unlocked the next day. However the lftp processes did not terminate for some reason when the account was locked.

I come back to check on a web application on the server and it gives a 503 error. hmm… weird I thought. Everything is fine, tomcat is up, apache is up and yet a 503. The problem was that the lftp processes that did not terminate caused the CPU to be overloaded. All other processes were begging for some CPU time. The lftp processes were sitting quietly and drinking up precious CPU power for almost 2 days. Once the processes were killed things went back to normal.

Weird things happen in PROD whether it is your fault or not. No matter how much you test, it always pays to be on your watch

I chanced upon a post a while back about how a switch statement should be replaced with the strategy pattern. If you have not had a chance to go through it, please do. The post is not very long.  I found myself disagreeing very strongly with the author of the post and I was surprised to find that people thought this was a good idea.

The gist of the post was that using the Strategy pattern was better than using switch statements to determine which logic to execute. Here is why I think the idea used there was bad

1. The introduction of the Strategy pattern, in the example, introduces three new classes. So for every case in a switch statement, we should go about replacing the corresponding code with a new class ? This could easily lead to an explosion in the number of classes.
2. Distributing logic across classes makes it harder for a developer to follow. When I traverse 2 / 3 links down a code path I might lose track of where I started.
3. Introducing a new case means I have to introduce one more class, remember to add that as a strategy and switch to that strategy correctly when the case is presented.
4. Introducing a new class to handle a ‘case’ might not make sense. Classes are supposed to cohesively carry out a function. Introducing a new class for every function that a class is supposed to perform, dilutes the purpose of the class.
5. The code that was supposed to perform the switch case is simply done else where and called a strategy.

Applying patterns where they do not belong, can be an anti pattern by itself. I can relate with what the author is trying to do. When I finished the ‘Head first design patterns’ book I was racing to implement a pattern for everything that was around me. The book stopped me right there and warned ‘HelloWorld does not need a design pattern’. And they are absolutely right. For a design pattern to succeed, you need

1. The right problem to apply the pattern.
2. Correct implementation of the pattern to solve the problem.
3. Knowledge and relevant documentation if necessary, to let the maintenance developer down the line know that this is how you used the pattern.

Patterns are meant to solve common design problems but trying to use them everywhere can lead to overly engineered code that is hard to read and maintain. Use patterns only when they are necessary and help you solve the problem.

After a long wait I got a google wave account. yay ! Took the wave for a spin over the last few days and there were some interesting things that I observed. I wrote my first java wave robot and it was pretty cool. But an explanation of how the robot works should be left to another post all together. I will share my general observations in this post.

Deleted welcome messages:

The first thing that was weird was that welcome messages are often deleted by wave users or by bots. This is nuts. The wave welcome messages also have a lot of noise amidst them with quotes like ‘Please do not delete this !’ in bold red with a big font size. Wave still does not have a feature to disable edits. It is coming soon but it is not yet active.

Lonely waves:

Almost every one is complaining about this. They have a wave account but the 8 invites that were promised have not yet arrived (for some). The reason being that google wave is expanding its user base very slowly to avoid a massive crash of the system. It is also difficult to find waves centered around a topic. I tried to find java discussion but my searches ended up with nothing If you are a java geek and have a google wave account simply add your wave account name in the comments section. At least those reading this post will know how to contact you.

Miscellaneous / Tit bits:

• Google wave was pretty slow. But I expected that since this is a DEV preview.

• A funky sidebar was present on each window. I did not like this. I am used to clicking somewhere within the side bar’s empty space to move up or down. Fortunately the mouse wheel works with the side bar.

• Some robots are creating havoc. Editing comments left by others and deleting wave replies. This leaves some people vexed.

• There are some cool robots out there. I tried out the ISBN wave robot that replaces the ISBN number with a picture of a book and a link to O’Reilly. I can think of many applications where such robots can come in handy.

• Editing / deleting wave replies (blips) involves 2 clicks. You click a drop down and then select an option from it. Usability of the blip can be enhanced by placing small icons near each blip, which you can click to perform that action. Just a thought.

• Removing yourself from a wave discussion is not yet supported. You can add contacts without asking the contact’s permission first.

• There are wave ‘groups’ out there that discuss specific topics. You can subscribe to these wave groups. But sometimes, like in a forum thread, the discussion takes a U turn and towards the end of the discussion the participants are talking about something entirely unrelated to the root topic.

• Searching for waves is done in a variety of ways. You can search by tags / contributors / authors / wave ID / gadgets etc etc. Each wave can be tagged to particular keywords like a blog post.

This is just a tip of the ice berg. The java robot was very exciting to create and it was more exciting to watch it in action. There are some good tutorials out there that tell you how to write a bot but I see a gap where the flow of events and the overall picture of the robot is not brought out. I will try to cover that in my next post. You can subscribe to the RSS feed if you would like to keep an eye on it.

Let me begin this post by saying that I am not writing this so that you can read this and become a haCkEr. I am writing this post so you can learn to identify a vulnerability and try to avoid an embarrassment.

Google is an amazing search engine. The problem is that it is too good at what it does sometimes Here are some ways that google can reveal vulnerabilities on your website by mistake.

You allowed google to index a critical file:

This happens more often than you think. WordPress for example houses important files under the wp-* folders and it is no one’s business except yours to look at these files. Other files like .htaccess htpasswd are critical to your site’s security (if you are using apache and ‘allow overrides’). Do not allow google to index them. You can prevent that by placing a robots.txt file on the root path of your website. More on that here.

The better option is to put in place a configuration that will not allow the sensitive file to be displayed in the first place. Not all robots will obey what you instruct using robots.txt. The FilesMatch directive on apache can help you protect your site.

You can double check that google can read your robots.txt using google analytics. You can check the files that google has indexed using the query‘site:yoursitename.com

Google indexed a service page that is being served on a non regular port:

Examples of this are login pages or services that do not require a password. Searching for such pages can be done using the “inurl” keyword in searches. Here is an example inurl:8080. There are ways to tweak that search string to reveal more information about services on other ports. When you complement inurl:something_unique_in_the_url with a search using quotes, like inurl:1234 intitle:”Administration blah”, it can yield some very interesting results. Pick your favorite admin tool and replace the port and title with the admin home page equivalent. The search works on many major application / web servers.

Remember that google indexes your page. Even if you correct the problem, the damage is done and is still being done. With cached pages, a service that does not ask for user name and passwords (yes there are important services that do not require a username/password) will be completely indexed. Yikes ! The data that your service exposes is cached and indexed for everyone to see. Not what we want.

To avoid this simply shutdown services you do not need. If you need a service but you want that service to be private, block the port with a firewall.

You can optionally tell google bot and other bots not to index the page in question. But that is not really a solution. Be proactive and secure the service. A cached page can end up earning you some DOS attacks.

Google cracks MD5:

I realized that google could be used to crack weak passwords from this post. If the encryption is done without salting, the password will result in the same hash every time. A weak password can be guessed easily using this technique.

The lesson here is to use a strong password that no one will guess. The other lesson is to ensure that the links on your site do not pass along sensitive information. Here is the google search in case it interests you

Cached directory pages:

Your web server is quite capable of displaying a directory listing. What this means is that besides displaying HTML, if I were to request for a directory name instead, your web server will reveal the contents of the directory to me. Why is this bad ? It helps find more vulnerable files that are housed inside those directories. You can ask apache not to serve directory content by configuring the same in httpd.conf. The line of configuration will look something like this



Options Indexes FollowSymLinks
# More stuff here

Remove the word Indexes.

The related search query in google is intitle:”index of /”. Tweaking it will provide better results.

Before you make any configuration changes, always make a backup. Read about the changes you are making and understand what you are doing before you do it. Try these tricks on your site and check if it is secure. Be creative. Think about other sensitive terms like jsessionid, username, passwd, password, id etc.

I have seen a few estimation nightmares in my time and have been unfortunate enough to be in some of them. Let me narrate a few anecdotes first

Anecdote 1:

I used to work with a re-insurer. This company had a legacy application that was written in fortran. Yes fortran. It did some very important things. It was capable of making estimations for a given market and it crunched a lot of numbers into meaningful data. Because this application was written in fortran, finding the right engineers to maintaining it was difficult. So they decided to shift the application to a better supported platform / language.

The work came to IT and a manager said ‘Lets convert it to VB’. This person, did not know fortran and was not a master of VB either. No developers or architects were asked for advice. It was simply decided that the application should be converted to VB from fortran.

But wait, the fun is just about to begin. After making the decision to convert it to VB, a developer was asked to judge how long it would take to make this migration. This developer worked in Oracle technology and knew nothing about fortran and very little VB. This developer was chosen to do this estimate because the others were busy. A ‘ball park’ estimate of 4 months was given to the manager. The estimate was not based on any fact / data. It was arbitrary at best. The manager then went to the user and said ‘We will convert your application from fortran to VB in 4 months’.

The user could not believe what he was hearing. So much so that the business department said ‘If you cannot convert the application in 4 months IT will pay X dollars per day for each day that the project is defaulted’. The IT managers agreed and all hell broke loose. Two developers were assigned to the project and they worked like crazy over weekends and put in 16-20 hour days to finish the project. The project was delivered 3 months late and IT lost money implementing the project.

Anecdote 2:

An application cluster which housed 3 nodes had a problem. The cluster would simply shutdown without warning every now and then and had some applications that were used by external users. Here external users = internet users that are customers to the application. Since the problem involved external users it was of higher priority since a crash is an embarrassment.

I was assigned the task of finding out what the problem was. A manager called me up and said ‘Give us an estimate on how much time you need to fix this’. I had no clue, so I did some preliminary investigation and tried to find various root causes. I narrowed down some possibilities and was about to give her a number. She said ‘Your estimate has to be within X working days since I told the users we will give them an analysis by Thursday’. I was like ‘What !?‘. I still had to do tons of research and it would have taken me a few weeks to go through all the problems in all the applications in the cluster and try to figure out what was going on.

There were no logs indicating a problem and very little signs of what was going wrong. In the end I was forced to deliver a half boiled “analysis”. The problem was then forgotten. It came back to bite us several times until we finally solved it a year and a half later.

There are several other cases where an estimation nightmare caused losses in time and money. I have learned some lessons the hard way over the years.

• When your manager will not listen to your reason about why an estimate is high, run the other way quickly.

• Assert yourself. Projects will take time to get done. You do not have to lower estimates just because some one thinks it is high.

• Include contingency effort in estimates. This is usually 15-20% of the total effort.

• If you hit a road block, communicate it immediately. If you wait for the end date and then tell everyone that you cant deliver on time, they are less likely to listen to you.

• Do not be shy to say ‘It cannot be done within X days.’ There is nothing wrong with saying this. No one will think you are a bad developer if you take 5 extra days to get something done. If you overshoot by 5 days however, people will think ‘you missed the deadline’ by 5 days. This is a big difference.

• Always over estimate. If you think you can get something done in 5 days, you are probably thinking about the ideal scenario. Users will come up with feedback sometimes and will still expect you to deliver in 5 days.

I am not trying to blame managers or developers for under estimations. There are managers that know nothing about the technology in question and make estimates in place of developers. Then there are developers that make estimations based on an ideal scenario. Both are taking risks. Think carefully before committing estimates. Getting them wrong can cost you dearly.

I joke with a friend about this all the time. ‘We are surrounded by Google programmers‘ he said, and I couldn’t agree more.

There is a difference between a ‘Google programmer’ and a ‘programmer that works at Google’. Google programmers simply search Google for a piece of boiler plate code and stick it into their app. It wreaks havoc later and causes trouble for a lot of people.

I chanced upon this blog post a while back that was complaining about lazy developers at stackoverflow. The complaint being that some developers ask silly questions, the answer to which a Google search will easily reveal. It seems some people are so lazy to use google that whole websites are dedicated to Google something for you

So there are sections of the programming population that do not want to use Google, they will ask a question in a forum, expect a silver bullet for a reply and nag till they get it. The other section Googles for some code, copies it into the application and god knows what will happen when it goes kaboom. There is a third section of developers that do use Google and do see code that comes out of a Google search, but do not inject the same code into their application. Instead, they understand the code and research a little more to arrive at a good solution.

Now although I agree with the sentiments in the blog post mentioned above, why single out stackoverflow alone ? A comment left in the post agrees that the problem is prevalent across other sites

We have this problem at dream.in.code as well. I think it’s a problem that creeps in to most Q&A type sites.

The rest of the comment makes a suggestion that programmers that are not employed in the US are lazy, but lets ignore that. Laziness transcends all barriers. The comment is quite true that most Q and A sites have people come in and ask very basic questions. The problem might not be with the people who ask the questions but with the people who answer it. When a slacker asks a question with an obvious answer, the right thing to do would be to point to a javadoc or a user guide pdf instead of giving the answer straight away. This will encourage the person that asked the question to lookup the answer next time.

When some one asks you a question with an obvious answer, do not be rude; give them a direct answer; or ask them to RTFM without giving them a link. For all you know this person may really be a newbie that has absolutely no clue about the subject. Instead post a helpful link and nudge them. If they repeated ask silly questions, simply ignore it after a while. Quoting a text from the post

This behaviors will produce tons and tons of copy&pasted code in our programs and will create a generation of developers unable to think by themselves

These programmers are already here. But I don’t think that we are creating a generation of them. Those that are lazy and simply want to copy and paste code will always do so. They can be ignored after a little nudging. Those that are ‘cats on the wall’ can be nudged to the right side. Then there are programmers that will research code and use Google in ways that it should be used. These are the programmers that will triumph in situations that involve solving classloaders problems / SSL handshake exceptions /performance issues etc etc.

If you are a web developer, you would have encountered one of the following challenges.

1. Ensure that your web page contents are not cached by the browser.
2. Check if your clear text componenets are gzipped by the web server.
3. Add additional request parameters / headers for testing.
4. Determine if a stolen cookie can be used to login to the site.
5. Analyze request / response times.
6. Verify web server response status / headers.

Well, I could go on and on and the list is certainly large. Do you see anything common in the list mentioned above ? They all require a tcp monitor / web debugging proxy to solve. There are several that are available out there. If you have never used one before, its never too late. I was able to save several minutes of time when programming java web apps after marrying them with a debugging proxy / monitor.

I use a tool named charles. Charles is a commercial product and a trial version will let you debug for 30 minutes. It is pretty cheap to purchase and gives you a central dashboard where you can focus your efforts. The trial version also shows a pop up now and then that will wait for 5-10 seconds before it lets you use the tool again. I have found these limitations to be bearable, if you are using the tool for free.

Here are a few use cases where a debugging proxy can prove to be useful.

Cache control:

It is important that you mention an expires header or cache control policies for your web pages. This allows your browser to display cached content if the content has not yet expired. You can use Charles to track these headers and check if they are actually being returned in the response

Start the charles debugging proxy. If you are using firefox, there is a firefox plugin that integrates with this tool. Now simply visit the site whose expires headers you want to check. Here is an example.

Expires header:

The report from charles shows that the expires header for a resource was set to Sep 22 2009. If you want to tweak that value, you can always change the params in the web server, reload the configuration and check back here.

Response times:

Now, if you want to check the request response statistics of your web site and how well it is performing, simply visit a couple of links on your site. The left side bar should look like this

Charles:

Click on the Chart tab on the right side bar and this will give you detailed information about the site performance. The chart can display data by time line,size, duration, or resource type.

Chart:

Arguably, a tool like apache JMeter is more suited to perform load tests and visualize test results. But a TCP monitor / debugger can also help here.

The tool has many more advanced capabilities which I will be writing about in future posts. If you would like to keep up to date, subscribe to the RSS feed.

I hope that the tool saves you time. Have fun programming

PS: I do not have any affiliation with Charles. I just think it is a cool tool. Other web proxies that you might be interested in are given below

Fiddler – From MS

Tcp Mon – A cool tool from java.net

Wireshark – A very advanced network monitoring tool that captures packets for analysis.

The RSS cloud and the pubsubhubbub (PuSH) way of notifying RSS feeds has gained some interest of late, thanks to wordpress supporting the former format. If you have never heard of these protocols, heres the low down. The RSS feed works by having a client ‘pull’ data from a feed. That is, clients keep nagging at the server every X minutes saying ‘Hey do you have something new for me ?’. The server responds with a no about 99% of the time. This leads to a lot of inefficiencies. Instead if the clients were to use a push model, the data is actually ‘pushed’ to a client automatically without the client asking for it. This is great since the client no longer makes unnecessary calls to the server. The diagrams below illustrate this.

Classic update:

Rss Cloud update:

The cloud is supposed to be an EC2 cloud. Thus the name Rss cloud You can get an overview of PuSH here .

So what are the differences between the two ?

What does this all mean for a regular RSS user ? It might mean zilch since the actual feed content does not change. The new model is similar to the Observer pattern, where clients behave like Observers and the server feed is the Observable. This model leads to a more efficient feed.  Clients may well receive faster updates but would a client care ? There are a few questions open to both models, which might raise concerns about reliability.

• How will these models address the problem of clients behind a firewall ? A geek can open up a HTTP tunnel or whatever, but the average user would be clueless. This is a major problem for both protocols. Dave Winer acknowledges this. Here is an excerpt of what he has to say about being behind a firewall.
  

  The subscriber must have a known address, therefore must not be behind a firewall or NAT. For client apps, they need some kind of proxy that has a known address. This limit is signficant, but certainly not insurmountable.

Technically, the problem is indeed surmountable, but when it comes to usability, this problem is a major headache.

• Will the new model actually mean faster updates to a feed ? Yes – if the cloud / server can find the client successfully.

• When using the RSS cloud, what happens to clients that use a dynamic IP address ? If they disconnect, the notification no longer works since the remote IP address is used for notifications.

• How do you handle authentication for a feed ? The PuSH model tech spec clearly states that only unauthenticated ATOM feeds can be served.

• Will both Atom and RSS be supported equally well. PuSH currently favors ATOM but RSS support is also mentioned in the tech spec. Rss cloud supposedly supports both Rss and Atom.

It would be interesting to see how robust this protocol / format is. It might lead to a better feeding mechanism if nothing changes for the client.  Ideally, a client should be able to get better update rates for a feed and not have to know anything new about Rss cloud or PuSH. It shouldn’t matter to the client which format is used. Whether that is possible is to be seen.