The integration offered by Django and RoR really makes one feel that not using these frameworks is just the thing why software developers are in demand today. Hire fifteen to do a job in php which two could do using Django.

Some may say these facilities are just keystroke-savers. Agreed. The important question is why do you not want to save those keystrokes . A common application development (without the right kind of framework) spends a significant amount of time (I’d hazard a guess of more than 25%) doing plumbing work. Lets take a Java example for an “edit” screen, which shows the user already stored data, and allows him to make changes:

• Take data from repository, probably as a generic List
  • Oh yeah, connect to the DB first, and run a DB Statement in a try-catch block, right within your business logic
• Populate a data bean with all the data. All manually. If the DB cursor contains 25 columns, have a object with 25 attributes, 25 getters, 25 setters and populate all attributes one by one.
• Send the data bean to the view
• In the view, have hard-coded UI components for each attribute of the data bean to be displayed. For any related queries to be executed, hard code page links which will fetch those.
• On event of user clicking OK, program a set of validations to ensure user has not entered bad values. Most of these are not business validations, but checks to ensure no bad characters are present in the input.
• Populate user entered values obtained through form into a data bean
• Code a SQL Update script which will take each value from the data bean and update the backend
• commit (don’t forget that!)
• Rinse-and-repeat for all related queries

Any framework support (struts, spring, hibernate) is a keystroke saver in addition to being responsible for formalizing (a subset of) the programming for the application. Django and RoR simply take the integration between MVC to the next level. I had mailed a friend a few months ago about what Rails has:

1. Integration with AJAX: You see whether the type of request from the client (browser) is ajax or plain HTTP. You use those conditions to provide a specific response. In case it is an ajax request, you can play directly with elements rendered in the browser (e.g. Put text “Saved Successfully” in the information bar and blink it once). This integration, and resulting 5 lines of code is far faster than anything we would do by ourselves. (client side use ajax-specific library functions to send request, then check XML returned and perform activities in browser using javascript again). This integration is thanks to use of inbuilt prototype and scriptaculous libraries

2. Test Integration: I know that if a nice high-level test framework didn’t exist already, we would never get serious about writing real automated tests. It would always be like “lets develop this cool feature now that we have time… we’ll check out tests later”. Fortunately, tests are easy to write, and we can start small. (full page tests and all can be written later). We can also write Model level tests for ensuring relationships etc. N e a t.

3. MVC Integration: Neat MVC code, easy to write. Controllers and views nicely integrated. All variables declared in controller are directly available to views. Which should have always been the case, but in other barebones J2EE/php, you need to pass a data object explicitly, because MVC is not mandated.

4. Fixtures: Stuff to fill your “Test” database with data. Easy to write YAML (byebye XML). This will auto-run when you run tests, so your test database is ready with required data. If you don’t know yet, Rails will auto-create dev, test, prod databases for easy demarcation. Rails will also auto-clone the dev database schema to test database prior to loading fixtures.

The learning curve is a bit high, and I am spending a lot more time understanding the components in comparison to J2EE. Yet, given the benefits, I think it is well worth the effort.

Although the Mindstorms RCX has pretty good capability, the programming interface bundled alongwith it is for kids. For any program spanning more than two cases of flow control, we need a real programming language.

The two languages I used were NQC (having almost C syntax, and named Not Quite C) and LeJOS (Le Java OS), based on Java 1.2.

NQC has inbuilt support for functions as well as multithreaded modules, but quite frankly the threading was not upto the mark. With two threads competing with each other for execution time, sometimes there would be several seconds where one of the threads waited for execution. LeJOS, on the other hand, worked beautifully with multithreading.

NQC also has a rather limited memory available for processing in comparison to LeJOS, which is quite surprising considering Java VM of LeJOS occupied around half of the memory. Yet, I was able to form large data structures in LeJOS while arrays of as small as 30 integers started giving trouble in NQC.

LeJOS has its set of troubles as well. The VM installation is actually a one time thing, but with the amount of testing you perform on the bot, batteries get changed, RCX units change and thus you find yourself running LeJOS installation a dozen times, which is quite a miserable thing to waste one’s time on. In addition to that, we have the great blocked I/O problem for Infrared Sensor which we discussed in our last post.

Limited memory and processing capacity puts stringent limits on nested methods, garbage collection and size of data structures. As I discussed earlier, a workaround to blocked I/O calls involved creating two separate threads. In my case, I had to keep both threads running infinitely. Having the child thread exit after receiving the data and having the parent thread spawn a new child when necessary worked for a while, but then the program would hang, presumably because there were too many threads getting created and the garbage collection may not have been able to cope up.

All in all, its fun working on the RCX, as long as the program is tuned to minimum resource usage, which goes for any micro platform. If you have experienced any more glitches or gotchas, please feel free to post comments!

However, LeJOS was based on Java 1.2, which had blocked I/O calls. This means that an I/O write was not completed until a corresponding I/O read was executed at the receiver. Similarly, an I/O read was not executed till someone wrote to the buffer.

How is this different from unblocked I/O? In the latter scenario, we poll the buffer status like this:

while(true) {
  sbuf = inputStream.read(byteArr);
  if (sbuf != null) {
    //code to process byteArr
  }
  //other processing to be done
}

In case of blocked I/O, the program waits for inputStream.read to get executed, till a point that the input buffer actually contains some data. Of course, this makes the check sbuf != null worthless, but more important is the fact that the “Other processing” we’d like to do never gets done until the input buffer is populated. Why is this important? Because in most cases we want to process a lot more than a single data stream. For example, in my LeJOS program, I was handling inputs from touch sensors, a light sensor and an Infrared sensor.

(A bit about my LeJOS program: I was trying a basic car driving program which could understand walls with touch sensors, car driving in front using light sensor and vehicle coming head-on using infrared sensor.  Although my Infrared sensor could be accessed using the usual I/O calls, it meant blocking up all the processing till someone actually sent me a signal.)

The easy way out of blocked I/O is multi-threading. The parent class polls the buffer data held in the child class (where both classes are threads and run independently). If there is valid data, we read and respawn the child class. Else we continue (in an infinite loop) with other processing and return later to check data. The child class is the one which stays blocked, but that thread is dedicated to receiving data only. Our program execution continues safely.

For those interested, LeJOS code for basic car driving (with inputs from sensors) with Lego Mindstorms RCX kit is available here.

Actually the beauty of the post lies in the author not trying to be in preach-mode. It’s just a “screw it all I’m goin’ home” treatment handed out to display:block; float:left; layout, which has irked some gentlemen of the CSS church. For these kind-hearted people, the author takes some time out to coin a term CSS-Trolls. And could you believe it, the very same association of people replied on this post with complete disgust, on how terrible the advice is, and how the author ought to die of shame. Well, some comments were actually written by well-mannered folks, who put their point and signed off with a bow.

But let’s not talk on people. Let’s talk on the point in question. Tables or CSS?

Well it really doesn’t have to be an exclusive decision. CSS does provide a lot in terms of

• Flexibility in presentation
• Easy alterations in future
• Accessibility for visually impaired
• Pixel-level control of all objects

and other goodies. I’m a CSS fan ever since I built my first CSS-enabled website 3 years ago and realized how much I sucked in web design before.

Yet, CSS, with all its control on page components, remains a browser-based implementation. Wait, even tables are in the same league! Yes, but tables give you one assurance: If you want “Y” in the left cell and “Z” in the right, that’s how you are going to get it. No side effects with anything: pages resized, font sizes increased, browsers changed, you try it. It may look ugly, sucky design, but “Z” remains to the right of “Y”.

This one single feature of consistent placement gets screwed up when you try to use CSS. I sat with my friend once to get a layout with 3 columns, with a horizontal bar at the bottom of all three columns. Quite a number of times, the bar appeared on the right of the columns, sometimes behind the columns. It took us a good one hour to get things sorted out, after adding a multitude of attributes for the last column and for the horizontal bar. I can surmise that whatever attributes we used would produce at least some side effects in one of the browsers.

You can well say that I am not as good of a CSS coder as I think. Well, that’s the whole problem. The layout worked perfectly in one browser and it wouldn’t on others. Why does my knowledge on markup have to involve inconsistencies rendered by design specs of different browsers?

Lets take another example. A friend working in a UI framework team discussed an issue with me one day: The div appeared two pixels to the right of the desired location on FireFox, while it rendered at the perfect spot for IE. Now that’s an amazing problem, where the (heavy) component is derived from 5 other smaller units. Turned out to be a difference of rendering a border between FF and IE.

Coming out of bedtime stories, we have situations where “the boss wants the code complete by evening”. It is useless to sit and gaze at the beauty of CSS at times like these. Just get yourself some nice tables where necessary, use CSS where easy and roll it out, as many have preferred.

At first, a little introduction to the havoc wreaked by WebAccelerator: It sits with your browser, and “clicks” links intelligently on the page you have visited. This ensures that your next click opens the new page instantly. However, “intelligent” behavior started to trouble web applications where links happened to update/delete records in Admin Consoles.

Although the bigger question raised was regarding privacy concerns (Google indexes pages prefetched by WebAccelerator, which includes pages unreachable by its crawlers), lets keep that out for a moment and revisit the issues faced by web developers. As Web Accelerator is no longer active, you may wonder why we need to recap history. The reason is, you never know what plugin the users of your app have installed on their browsers. Yesterday, it was Google. Tomorrow, it may be something smaller, having auto-installed with another package, and no one will have an idea that your pages are being prefetched.

As always, information websites with links sprinkled around do not need to bother about prefetch. Its the websites with user authentication required that mostly fell prey to this activity.

I’ve not tried GWA, and there are comments stating that GWA doesn’t do a lot of things which have been alleged. However, our work here is not to discuss merits of Web Accelerators and their conformance to standards. All we want to do is strengthen our own website. So lets take a look at some problems faced, and graceful solutions or workarounds opined.

1. “Logout” link prefetched once the user logged in: This threw the user out before he did any other activity. Quite irritating. The “Best Practices” supporters came out in strong defense of Google here. Why would developers keep Logout as a link (GET) and not a POST, they asked. Except that Logout is really an idempotent operation! A user can logout once or ten times, and it is always the same result, in almost all cases. Our little tweak to the Best Practices helps in deciding that POST is better for Logout.

A safer deal is to have form method as GET when the application state does not change at all

2. “Delete” links prefetched in Admin consoles: Well, this is pretty straightforward. You cannot have “Delete” as a GET operation. But here’s where we get out of utopia. In the real world, navigation and look and feel of the application is largely decided by the UI team, and the developer has little say in the matter. If the designers feel that links alongside 10 items feel “cool” and buttons don’t, well, you need to keep a link. The workaround here is to have a href = "#" and code a form submit on the onclick event of the link.

3. Links which involved heavy database operations: …and thus increased server load were prefetched. A way out here is to limit the number of “heavy” operations performed by a user per minute. This seems like a fair balance between a hack to redirect to 403 and a puritan approach of removing links altogether, making pages accessible only through Javascript or POST operations.

4. Links which retrieved data but also imposed exclusive locks on the data: The first user to come along could end up locking quite a bit of system data, thanks to prefetch operation. However, isn’t a lock on data change of application state? The change needn’t be in a database operation. Any change of state should require (scream) POST.

Well, that’s quite an interesting list of 4 points with repeated gyaan which has, no doubt, also been written  before by others. But, as long as reading this post helps at least one developer, I’m happy. Benefited developer, please post a comment so that I stay vindicated

After thinking about it, I’m inclined to agree.

Older browsers didn’t have this problem: they were not multi-tabbed. And window switching is certainly not disabled by any modal pop-ups.

Browsers evolved, tabs arrived. Today I have 15 tabs opened in my browser. And then say someone sends me a nicely obfuscated script which simplifies to this:

javascript:eval("for(i=0;i<554;i++) alert(123)");

Works nicely in FireFox. Stops me from accessing other tabs altogether till I succumb to the pressure 554 times (and I might not even know the actual figure if its jumbled code)

And lets not forget that this error can be caused without malicious intent. Say some webpage gives out two or three alert messages in a loop, and the loop counter goes haywire because of a corner case which wasn’t tested. It still is a block on all activity in the browser, whether the user likes it or not.

I’m not saying this is some awesome loophole in browser security. But it certainly is more than an irritant if I’ve paid by credit card in another tab, and the site is waiting for me to perform the next step.

The graceful solution for this is to have dialog boxes modal with respect to their own tab. This satisfies all legacy requirements of an alert as well. Currently, browsers have just provided a knife to play with.

Update: This problem has been reported in Mozilla as a bug, verified as a FireFox DoS as early as November 2000.

I think this is really a common problem for web-developers. I’ve fallen into this trap several times myself (accidentally created an infite loop around my debugging alert()). But instead of aborting all scripts, I’d like to have something similar to the “A script on this page is causing Mozilla to run slow” message with the option to abort the script.

Another user writes:

The page in the URL (don’t open it unless you know what you’re doing!) “locks” the user in an endless stream of JavaScript alerts. There is no way out; closing the popup just opens a new one; UI is unresponsive in *any* place except the popup; you can’t cancel the loading of the page (or do something like ESC to stop the script) since the UI is blocked by the popup. The perfect anti-Mozilla DoS.

Today this very pop-up is a major hassle for users, just because in multi-tabbed interfaces, the user cannot switch between tabs when an alert is displayed to him. So you have an irritated user who wants to urgently book a flight in the next page, but is stuck on your tab because you have provided some text for him to understand and press OK.

This is still ok when the alert is shown on the press of a button. There is a higher probability of the user being still focussed on your page. But alerts on page load, notifying “Successfully performed operation” are nothing but rot.

To grab user attention while user is on your page, a far more elegant way is the use of div based message boxes. This keeps the user interface consistent with the remaining website and also grants user space for some other (possibly precious) activity. The new CSS opacity attributes also let you gray-out the page background whilst displaying your error — pretty neat.

While you are proceeding towards unobtrusive interfaces like a good kid, why not also give a thought to what ought to be the messages warranting a pop-up? Oracle ADF, which we use for development, has 3 levels of messages:

• Info, Warning: Display at the top of the page with appropriate icon
• Error: Display as a div based pop-up to the user, gray out the remaining page, ensuring the user cannot perform any activity till he selects an option to handle or acknowledge the error.

The good part is that the behavior of messages is built into the framework. So if tomorrow a usability test suggests that warning messages better be shown as a popup so the user doesn’t miss them, there’s a 3 line change in the framework to enable this. Clean, easy, cool. That’s the way we want it.

Its safe to assume that not all developers will be using these features for mainstream websites unless our very dear IE supports them. So we still have time to see these beauties in action. But we have a lot of gray area for what happens when these are used together. The implementations are still juggling with these questions.

For example, what happens when opacity of a box is 50% and a drop-shadow attribute is used for the box? Or, what happens when a shadow drops on a box which is having less opacity? Should the shadow have rounded corners for a box with rounded corners? How should a shadow with bigger fade (black to transparent area) render on a shadow with smaller fade?

These are just some tricky “beginner” questions with no straight answers. Fortunately the community is pretty enthusiastic about rendering these effects with markup. Lot of such issues and fundamentals have been discussed.

In an image editor you can manipulate the effect rendered depending on which operation you perform first. This is not really clear for CSS, and with browser implementations which can vary, it might be a bit of a hassle for us to have consistent cross-browser CSS3 markup ready to use.

Code for Python (Write to file)

f=open('scratch','wb')
for i in xrange(1000000):
f.write(str(i))
f.close()

Code for Java (Write to file)

File f = new File("scratch");
PrintWriter ps = new PrintWriter(new OutputStreamWriter
(new FileOutputStream(f)));
for (int i = 0; i < 1000000; i++) {
ps.print(String.valueOf(i));
}
ps.close();

Discounting the mandatory include statements for java.io, class name, try catch block for any and all I/O operations, we are still left with around 4 classes to be used for 1 write operation to a file.

So the question newbie posts in a forum is: I am writing to a file, but I’m not sure whether I should use BufferedOutputStream or BufferedWriter

The reply generally holds this: If you are inserting chars, use a Writer, else use the Stream

This is fairly accurate, but for a slightly deeper understanding let us break down the modules

Streams transfer bytes. Period.

So when you want to transfer any binary data, you use streams. You open the file, wrap it with FileInputStream, take out the data and process it. When you save it back, you make use of the FileOutputStream. Similar classes to be used for transferring though other streams.

In case you would like to use some text-friendly methods for your ascii text, you need chars or a char[] array. To get this, you convert bytes to chars while reading (and similarly chars to bytes while writing), with a simple wrapper, which happens to be InputStreamReader for reading and OutputStreamWriter for writing.

To have data formatting conversion done by Java, wrap the Writer objects in a PrintWriter

To increase performance, you wrap the objects in a buffer. Java uses internally allocated memory space to provide efficient read/write operations. The working of the buffer is transparent to the programmer, except for few methods like flush()

(Any) OutputStream => (wrap in) BufferedOutputStream

(Any) InputStream => (wrap in) BufferedInputStream

(Any) Writer => (wrap in) BufferedWriter

(Any) Reader => (wrap in) BufferedReader

The Buffer classes also provide some friendly methods: readLine() in BufferedReader and write(String, int, int) in BufferedWriter. These are more of a convenience which can be coded anyway in 2-3 lines. In case your app can’t afford to wait for the buffer to get full before transmitting or receiving data, it is always better to avoid using Buffers than to use and flush() them throughout your program.

The big question often happens to be which method to use. The basic wisdom you get from the guy in the next cubicle (or from Microsoft) is

The GET method[...] appends name/value pairs to the URL. Unfortunately, the length of a URL is limited, so this method only works if there are only a few parameters. The URL could be truncated if the form uses a large number of parameters, or if the parameters contain large amounts of data.

Also, parameters passed on the URL are visible in the address field of the browser. Not the best place for a password to be displayed.

As always, there are some more things one ought to know as a responsible developer to get this distinction correct. Firstly the definition as per the HTML specification (which is extremely useful)

If the processing of a form is idempotent (i.e. it has no lasting observable effect on the state of the world), then the form method should be GET. Many database searches have no visible side-effects and make ideal applications of query forms.

If the service associated with the processing of a form has side effects (for example, modification of a database or subscription to a service), the method should be POST.

The keyword “idempotent” says a lot. If the state of your application does not change with the number of times a request is sent with a parameter (set salary of X as 10,000 bucks) then the operation is idempotent.

A safer deal is to have form method as GET when the application state does not change at all (search for text in a file or database).

We will look at other practical differences between the two methods which further influence our choice (some of them are already covered by Microsoft. Yay!)

I attempt a tabular distinction below. Lets hope it is clear enough.

Some more detail on this with additional perspective is available here and here. Some perspective from grandmother’s viewpoint is available here