You will probably speak and write a language that's written left to right. In fact, most languages are written that way. I think it would be an accurate guess to say that a lot of software doesn't support bi-directional text, because it was developed in a Western country or at least a country where they use a left-to-right language. This is the cause for a relatively large problem: this kind of software is unusable. What can we do about it?
A few languages, or more accurate: scripts, are read right to left. While seemingly unimportant this causes a lot of software not to support those languages, unless they are written in, say, the Latin script. In other words: your stuff is of (nearly) no use to people mainly using such languages.
Let's take a look at this screenshot of a basic Drupal 6 installation using English, but set to RTL. You will notice Druplicon, the Drupal mascotte, facing away from the center of the page to the border of the browser window. I can't think of any reason why Druplicon should find the window border more interesting than the page content, can you?
Another problem is string concatenation. In a lot of cases you will want to concatenate strings that need to be displayed in a particular order, because otherwise they won't make sense. At other times you are working with strings that consist of important and additional content, someone's name and phone number, for instance. Logically the phone number is the least important bit of information, so you will want to display it after the person's name. In Western languages, like English, after translates to right of the name. In Arabic, however, after would be left of the name.
Since this blog is about creating websites I will only present a solution for this particular kind of software. Converting your ordinary websites to good software that can be used anywhere in the world, independent of the language it's being used in requires two things:
Implementing solution #2 might take a lot of time, depending on the system you're using and in how many cases it's actually applicable to that system. Something to take into account before converting anything is your target audience. If you are only able to sell and support your software in a region where no living soul would ever use an RTL language, it's just rubbish to make your system support that kind of languages at all.