David Herron's Blog

David Herron has worked for 7 years in the J2SE Quality Engineering team. He has developed test suites and GUI test automation tools, and was a co-author of the java.awt.Robot class. His prior experience is in development roles, in cross-platform GUI toolkits, and electronic mail systems. David graduated from the University of Kentucky in 1988.

Mustang Beta approaching - we want to know about your bugs and regressions

A useful testing technique to find deadlocks related to invokeAndWait

Tom Hawtin writes about a deadlock he found in jedit 4.2. As he says: There was, however, a use of invokeAndWait. Sure enough the EDT was waiting on monitor locked by the thread calling invokeAndWait.

What's most interesting is a class he provides to help one automatically find those situations. It relies on a Mustang ability provided through the JMX support to query all held locks. The code looks to see if the Event Dispatch Thread is called with a parent stack frame in invokeAndWait, and then checks the locks. Really cool.

Developer scenario testing

Today I want to talk about something I've been thinking about a lot lately. I work in the Java SE Quality Engineering team. We do a lot of the testing that goes into ensuring the quality of Sun's Java. We see the new features in Java before anybody else does. As one of my colleagues says "we are the first customers of Java". Our job is to create "functional" test suites for new features in Java, the functional test suites being a different style of testing than either the unit tests or conformance tests others in the Java SE team do. I've covered some of this in previous blog postings.

The thing I've been thinking about is the testing that we cannot do. That is, we cannot test your applications in your environment.

We try and put ourselves in your shoes, to think about what you might do with the features, and write tests from that viewpoint. But, try as we might, we are not you, we do not have your environment, we do not have your applications, we do not have your mindset, and we can only go so far.

What's most important is your experience with Java when you develop and run applications. Even more important is that your applications continue to run well even when we make a major Java release. e.g. the Mustang project is wending towards its finish date (and, no, we have not disclosed the Beta/RC/FCS dates as yet), and it will be really great if your applications were to execute just as well or better on 1.6 as they do on 1.5.

As I said, we do the best we can. The team as a whole tests Java in several different models (conformance testing, unit testing, performance testing, stress testing and functional testing). But, as I said, we are not you and we cannot have covered all the functional combinations that you have implemented in your applications. There is an effectively infinite number of ways to combine Java API calls and features, so it is clear that you will have come up with ways to use Java that we cannot have dreamed of.

This is where you come in. You can test your application in your environment on the Mustang early access releases today. We really want to know about problems you see with your application on Mustang. The earlier we know about those problems the greater a chance there is we can fix the problems before Mustang is finalized.

We've been providing the weekly snapshot builds on mustang.dev.java.net for several months now. It's easy to download those builds and try them out.

So, "why". Why do this testing for Sun? Let me suggest that you do it for yourself. You will eventually want to run your application on 1.6 anyway, so why not try it out today and see how well it works?

The non-public classes in Sun's Java implementation

Romain Guy has written a blog entry about SwingUtilities2, and I find it a little confusing because in one paragraph he's suggesting the reader to use this class, and in the next he is saying to steer clear of it. Maybe he's quoting someone else and didn't make that clear?

In any case, Sun ships in our JDK a whole lotta private classes. It's relatively easy to dig around and find interesting classes or methods to call. And the exercise is somewhat like the old past-time of Windows programmers of finding undocumented API's to call to do goofy things.

I want to write a little to clarify what I think Romain was saying.

I'm sure the SwingUtilities2 class has some useful features. No doubt that's why it's there. But if one uses that class, one should absolutely expect their programs to break.

We treat the public documented API's as our contract with you, the developers of Java applications. We go to great length to maintain compatibility with the public API's. But when something is undocumented or otherwise clearly a private API, we feel very free to make changes.

The SwingUtilities2 class which Romain mentions is in the latter camp. We mave made zero contract with the public about that class, just like all the other private classes. Those are all implementation details which are free to change at any time as we refactor the internals to fix bugs and add features.

For the public classes, the ones for which we gaurantee compatibility, we have review processes up the yinyang to check for changes. I work with one of the review processes, called the "CCC" (which is probably short for "Change Control Committee", but I like to think of it as the "Committee for Concerned Citizens"). For "substantive" changes we go through a review process and generally strive to keep behavior, exceptions, signatures, etc, the same from release to release. The process gets a little tricky from time to time, especially when the current behavior of a method is buggy and fixing the behavior means an incompatibility which will break software that depends on the buggy behavior. Fortunately those are rare instances.

In any case, this is going on a little longer than I intended.

The short message is a strong recommendation to stay away from using private classes. If you do this, your program will be tied most strongly to Sun's Java, and will probably break in a later release.

Re: Why Sun never make Java on FreeBSD

I saw this question Solaris vs FreeBSD: Why Sun never make Java on FreeBSD and thought to say a couple things. Note that I am not involved with the management decisions around this, and don't know the reasons why Sun doesn't ship the JDK for FreeBSD. But what I see is that Frans Thamura's blog posting ascribes conspiratorial schemings on Sun that probably aren't accurate.

One thing is that Java is hardly absent from FreeBSD. There is an independant port whose home page is here: http://www.freebsd.org/java/ and it appears to support 1.5 (a lot depends on what is meant by "stable")

A very simple consideration should be fairly obvious. And if you understand Occam's Razor, you might want to follow the simple explanation rather than a complex one.

When Sun "supports" a new platform for Java there's a lot of cost and resources that have to be lined up. There's developer expertise required (in knowing the special features of the OS), a lot of the native code would have to be ported to the new platform, and the testing team has to add multiple platform instances to the test matrix. All that (and more) adds up to a considerable cost, making the simple plausible explanation to be that Sun's management would want to see a business justification for spending that money.

See? No need to ascribe conspiracies and scheming! Maybe it's just about the money. (NOTE: I do not know this one way or another ... I'm just offering this plausible explanation)

Automated visual verification is hard

In the Quality Team we try to automate our testing as much as possible. This is easy for tests of the core library or other functionality where there's no GUI. But when you bring in a GUI like for AWT/Swing tests then the test complexity goes up dramatically, because for some scenarios you need to verify the graphics rendered correctly.

There's a general strategy we have in GUI testing we call "layered testing". There are always aspects to the GUI API's that we can test programmatically. E.g. if you click on a Button, you can programmatically test the ActionListener gets called, etc. We use that trick to avoid automating visual verification in most cases.

But I've been thinking about a really hard problem in this area for awhile. A few months ago I watched a video shot by Scoble (that Microsoft guy) about this ??sparkle?? product they're going to ship (they==Microsoft). I don't remember the name of the technology, but that doesn't matter. The capability is to change GUI components so they render as a pseudo-3D animation. In the video the team showed radiobuttons who would dance around if they were selected, or listboxes who'd been subclassed beyond recognition to shift their contents around a 3D space.

Looks cool ... but if you can't afford to test it, then how good will the result be?

That is -- what I've observed from 16 years in software development is there's always a limited budget for testing. In testing the GUI you can rely on manual testing, but it's labor intensive and your budget quickly runs out. The common quip in testing is "Testing is never finished, only abandoned" which is what happens when the budget runs out. If you can automate the testing then your budget ought to stretch further (except that automation doesn't come free, as it carries some overhead) and you can do more testing within your budget and time constraints.

Which gets me to dancing buttons.

Today buttons and other GUI components don't dance, they're rather still instead. But we can see the writing on the wall. CPU's are getting faster, memory more abundant, and more importantly graphics acceleration becoming relatively common. In a relatively short time it will be fairly common for GUI's to no longer be flat 2D still-lifes. Instead they'll be 2.5D dancing animations, and maybe they'll sing too with sound output.

I can just see the marketeers salivating over that coolness.

Anyway ... I'm wondering how we're going to test that capability when it becomes ubiquitous enough to worry about it.

With the flat 2D still-life style components we have today it's hard to automate visual verification. You can take a screen capture of the component and compare it against a known-good image. But there's a lot of caveats and overhead so in practice it's not that simple. But with dancing (and maybe singing) GUI components the complexity is amplified even more. It's not a still-life, but it's moving, and when you make the screen capture it's like trying to shoot a moving target.

What are we going to do when GUI's are dancing and singing away? How will we be able to automatically verify it's dancing the right way and not stumbling around?

That's what has been in the back of my mind since I saw Scoble's video.

Safety is freedom?

Berin Loritsch riffs off one of James Goslings claims: James Gosling Asserts that Safety is Freedom and heads into type safety in strongly typed versus loosely typed programming language.

It seems the basis for Berin's argument (loosely typed languages are safer/easier) is to take an analogy from Ben Franklin that "Any society that would give up a little liberty to gain a little security will deserve neither and lose both." While old Ben was a wise man and there's a lot of wisdom in that statement, computer programs are not people nor are they societies.

What computer programs are depends on the specific type of program you're writing in the moment. In general software is highly rigid, containing millions of lines of rigidly specified precise instructions. Writing software is like telling a baby to walk by directly firing electrical impulses down the babies nervous system. You get one thing wrong and the baby flops on the floor and starts crying.

People and societies on the other hand are made of processing units that have a huge degree of self intelligence, awareness, and autonomy. You can tell someone to "go over there", and they'll figure out all the steps. But if you tell a computer to do it, you'll look as silly as Scotty talking into the mouse on a Mac in the Star Trek movie where they saved the whales.

Okay, so, he used a bad analogy.

He's right -- there's a kind of software where you want quick scripting language type of experience. But, I say, there's also a kind of software where you want high rigidness.

Strong type safety is useful in the latter case, while in the former you want loose typing. It's really that simple.

I've programmed in dozens of languages and I value both language approaches. Hence, Java has its place with its strong typing, and other languages have their place with their loose typing.

In Mustang we're starting to offer scripting support. I haven't tried it yet, but the general concept is a good idea. That allows a kind of hybrid approach where you have a scripting language with its loose rapid style, accessing modules built in Java and presumably those modules are the ones needing a stricter style.

Bangalore (the city) changing its name? ...now Bengaluru

I thought I was done writing about Bangalore, which I was doing with my recent trip there. I wanted to share some "color" about the Java team, and the place (Bangalore) where some of us live. At least, the Bangalore as seen with the eyes of a Westerner.

Today The Register has an article describing how Bangalore's city government has decided to rename the town to Bengaluru. I suppose this follows in the footsteps of Bombay renaming itself as Mumbai, and Madras renaming itself to be Chennai.

The origin story of Bangalore, as told by The Register, is in line with what the tourist guides said. Namely, the original name in old Karnataka was "Benda Kaal Ooru" or "boiled beans town" and derives from a King who was lost and some local person showed kindness by giving him some boiled beans to eat.

Since the locals don't speak old Karnataka any longer, the old pronunciation is just as foreign to them as it would be to me. It's going to be interesting to wrap my tongue around Bengaluru when it's so similar to the accustomed Bangalore.

I understand this as the native culture reasserting its true identity. The name "Bangalore" was an anglicization imposed upon them by the British rule. It strikes me how in India the native culture remained strongly enough entrenched that they can reassert the native identity like this.

In the U.S. the native culture was basically wiped off the face of the earth and the remnants have little power to do anything. If in todays time a country did to its native culture what the U.S. government did in the 1800's there'd be a worldwide hue and cry of genocide.

What time is it?

Since the Quality Team is so widespread (as I blogged before, Santa Clara CA, Beijing China, Bangalore India, Hyderabad India, St. Petersburg Russia and Dublin Ireland) we have a constant question of what time we need to hold meetings so the majority of us can participate. It's tough because for most purposes Santa Clara and the other teams are on opposite sides of the world and we're asleep when the others are awake, and vice versa.

On my recent trip to Bangalore I found the kworldclock tool to be immensly valuable. I wanted to know when to call home to talk with my sweetie. What kworldclock does is show on a world map where the sun's shadow is currently located. Similarly on British Airways flights during the trip I was able understand better what I was doing in relation to the sun, because BA has a similar display in the airplanes.

(ASIDE: the flight back was a veeery long day, beginning in Bangalore at 4:00 AM, flying to London arriving around 12:00 leaving around 2:00, flying to Seattle arriving around 5:00 PM and then to San Francisco arriving around 7:30 PM. I was chasing the sun across the planet, spent around 24 hours of my time returning home, and it didn't get dark until sometime during the flight from Seattle to San Francisco. That is, except for this brief period flying over Greenland where we were in the perma-dark area near/above the arctic circle)

Anyway .. back to the time zones and my concern about what time it is. The kworldclock has a nice feature that it lets you list selected time zones at the bottom of the screen. Unfortunately kworldclock doesn't exist for every platform, and I'd like to have similar functionality on other systems.

What I have here for todays blog entry is the underlying Java code which does the minimalist functionality I need. That is, given a set of time zones, print a table showing the current time in each.

The first thing to see is that the Date class doesn't have the required functionality. You can make a Date object and it gives you the current time, or you can initialize it for some other time. But, how do you know the time in some other timezone? Further it's getYear and other methods are deprecated in favor of the Calendar class anyway.

What you do is make a TimeZone object, and then initialize a Calendar (or GregorianCalendar) object using that TimeZone. With the Calendar object in hand you then use "get" to retrieve the various fields and print them out.

Let's first get the TimeZone objects:

    TimeZone tzsca = TimeZone.getTimeZone("America/Los_Angeles");
    TimeZone tziec = TimeZone.getTimeZone("Asia/Calcutta");    // same as Bangalore
    TimeZone tzmoscow = TimeZone.getTimeZone("Europe/Moscow"); // same as St. Petersburg
    TimeZone tzprague = TimeZone.getTimeZone("Europe/Prague");
    TimeZone tzdublin  = TimeZone.getTimeZone("Europe/Dublin");
    TimeZone tzlondon  = TimeZone.getTimeZone("Europe/London");
    TimeZone tztokyo  = TimeZone.getTimeZone("Asia/Tokyo");
    TimeZone tznsk  = TimeZone.getTimeZone("Asia/Novosibirsk");
    TimeZone tzhongkong  = TimeZone.getTimeZone("Asia/Hong_Kong");
    TimeZone tzshanghai  = TimeZone.getTimeZone("Asia/Shanghai");    // same as Beijing
    TimeZone tzseoul  = TimeZone.getTimeZone("Asia/Seoul");
    TimeZone tzbroomfield  = TimeZone.getTimeZone("America/Denver"); // same as Broomfield
    TimeZone tzboston  = TimeZone.getTimeZone("America/New_York");   // same as Boston

An important thing to note are the comments saying "same as NNN". I think its unfortunate that the names by which you retrieve TimeZone objects use specific cities, but they're often not the city I'm interested in. This is because each time zone covers a vertical area and the TimeZone names are based on some specific city in that vertical area. So rather than America/Boston we have America/New_York or instead of Asia/Bangalore we have Asia/Calcutta.

This is fine if you understand the time zones in the country you're interested in. In my case St. Petersburg Russia isn't a TimeZone name and I don't know enough about Russia to know which other city to use. Fortunately one of my coworkers is from Russia, and he happened to walk by just as I was wondering about this and I asked him. He says Moscow is the correct city to use. I also had a similar problem with Beijing (turns out Shanghai is probably the correct answer).

This code prints a list of TimeZone names

        String[] tzlist = TimeZone.getAvailableIDs();
        Arrays.sort(tzlist);
        for (String tz : tzlist) {
            System.out.println(tz);
        }

The next is, printing the time for each selected time zone.

    public static void printTimeZone(TimeZone tz) {
        GregorianCalendar cal = new GregorianCalendar(tz);
        System.out.println(tz.getDisplayName()+" ["+tz.getID()+"]"
            +": "+cal.get(Calendar.MONTH)
            +" "+cal.get(Calendar.DAY_OF_MONTH)
            +", "+cal.get(Calendar.YEAR)
            +" @ "+cal.get(Calendar.HOUR_OF_DAY)
            +":"+cal.get(Calendar.MINUTE)
            +":"+cal.get(Calendar.SECOND)
            +" "+cal.get(Calendar.ZONE_OFFSET)/(60*60*1000));
    }

That gives me some nice output as below. I could have made a Formatter, I suppose to straighten it out even better.

Pacific Standard Time [America/Los_Angeles]: 11 5, 2005 @ 14:21:5 -8
Mountain Standard Time [America/Denver]: 11 5, 2005 @ 15:21:5 -7
Eastern Standard Time [America/New_York]: 11 5, 2005 @ 17:21:5 -5
Greenwich Mean Time [Europe/Dublin]: 11 5, 2005 @ 22:21:5 0
Greenwich Mean Time [Europe/London]: 11 5, 2005 @ 22:21:5 0
Central European Time [Europe/Prague]: 11 5, 2005 @ 23:21:5 1
Moscow Standard Time [Europe/Moscow]: 11 6, 2005 @ 1:21:5 3
India Standard Time [Asia/Calcutta]: 11 6, 2005 @ 3:51:5 5
Novosibirsk Time [Asia/Novosibirsk]: 11 6, 2005 @ 4:21:5 6
Hong Kong Time [Asia/Hong_Kong]: 11 6, 2005 @ 6:21:5 8
China Standard Time [Asia/Shanghai]: 11 6, 2005 @ 6:21:5 8
Korea Standard Time [Asia/Seoul]: 11 6, 2005 @ 7:21:5 9
Japan Standard Time [Asia/Tokyo]: 11 6, 2005 @ 7:21:5 9

Now looking at this table I see a few problems.

First is the name of the month isn't printed, but instead the number. In Mustang we have added the getDisplayName method to get this automatically, and one also supplies a Locale. Without the getDisplayName method one can obviously make an array of month names, but what about localizing the month names to different languages?

The pre-getDisplayName implementation looks like this:

    static final String months[] = {
        "January", "February", "March", "April", "May",
        "June", "July", "August", "September", "October",
        "November", "December"
    };
...
            +": "+months[cal.get(Calendar.MONTH)]
...

Second, what about the three-letter code for the timezone? I don't see a method anywhere to retrieve it.

The last is the printing of the timezone offset isn't quite right.

So, with a little bit of work one can get pretty close to a simple tool to solve the problem I started with. We don't have a fancy-schmancy worldclock style display, but at least it lets me know what time it is.

UPDATE: I found out how to get the three-letter timezone code:

...
   +" ["+tz.getDisplayName(tz.inDaylightTime(cal.getTime()), TimeZone.SHORT, lc)+"]"
...

The quality team's test execution load and scheduling

The other day I wrote about a patent some of us in the quality team received over a test execution scheduling tool we developed. While the patent and the software is interesting, the thing that's really interesting is what that software enables us to do.

Namely ... because of DTF we are able to schedule execution of a tremendous amount of testing on a wide set of platform combinations. Without DTF we would get lost with the test execution schedule the java quality team faces. I thought it would be good to outline just what that is.

First thing is the number of tests. I don't know where to find the total, but I am sure it's over 100,000 tests. This isn't just the JCK but a whole slew of "functional" tests the quality team writes. The JCK tests have a strictly defined charter of measuring conformance with the standard, and there's a whole lotta testing done beyond that for functional behavior.

Obviously we don't run all the tests everytime we do testing. I understand it takes 2-4 weeks (or more?) to go through all the tests on all the platforms we test on. We therefore shift and juggle what gets tested each week based on what changes were made, and just to rotate through the tests and platforms.

The next thing to consider is the number of platforms on which we test Java. I can't publish the exact platform list, but obviously it's going to be a lot more than "Windows", "Linux" and "Solaris". The platform matrix considers many more factors such as the operating system version, the CPU, graphics cards, locales, web browsers and more. For example an i18n test needs to run in every locale, while a plugin test needs to run on every web browser. Graphics tests need to run on every graphics card.

Clearly if we did a straight permutation of every operating system version, on every CPU, on every locale, on every web browser, on every graphics card ... that is the direction we go with the platform matrix, but nobody could afford to test every combination. Hence we make a platform matrix that includes several combinations of those attributes, and we base it on judgement and experience.

What this gives us is each week is a group of tests to execute, and a group of platforms on which to execute those tests.

Here's where DTF comes in. In case you didn't read my posting the other day, the capsule summary of what DTF does is this: Using JINI technology it dynamically constructs a network of systems on which we can schedule test executions. It has a "controller" which matches test execution job requests with test systems on which to execute the tests. It uses JINI requests to tell systems what tests to run when, and it collects results and sends out notifications.

So then, handling the test execution load is moderately simple ... one just creates a set of "job files" describing the tests to execute and the test platforms, and hands those job files to DTF. DTF does all the scheduling working at it all night long etc.

A patent for the Java Quality organization

Okay, go http://patft.uspto.gov/netahtml/srchnum.htm to the patent lookup service and enter this number: 6,961,937

Scan a few names down the list and you'll see yours truly. I'm speechless, astonished, and more. I kinda don't like patents, but there it is, my name on a patent. This means I'm going to have to hunt up my resume and add a new section on Patents Awarded. My gosh.

I think that I or Constantin might have mentioned the software for which this patent was awarded. It's a pretty cool package that is (?unfortunately?) available only inside Sun. The patent tells the story but you have to read quite a long ways past an impressive list of claims before you get to it.

The Distributed Test Framework (DTF) is the software in question. The purpose is to schedule test execution jobs across a range of computer platforms and collect results. Before we designed and built DTF the Quality Team was running around the test lab like madmen, frantically executing test suite executions and trying to remember what was running and collecting the results afterwards. Human beings can only scale this kind of activity so far before they go nuts.

To test Java we select a couple dozen platform combinations. On each platform combination we run tens of thousands of tests (perhaps over 100 thousand?). And we are striving to do this on a regular basis, such as every night or at the very least every week. You might think "Oh, Java is cross platform, so why test on every platform"? After all we've had marketeers telling you guys for years to not quip "write once test everywhere" so why are we testing everywhere? It's simple, a lot of the Java implementation runs on top of native code that has varying implementation across platforms. AWT obviously is different for every platform, and also obviously are the Server and Client compilers in the VM. And there's more. That means any time there is underlying native code that's unique to a platform, then Java is likely to show platform specific bugs. Hence, to catch platform specific bugs we have to test on a wide variety of platform combinations.

Like I said, a human trying to schedule all that testing by hand, or with a spreadsheet, would quickly go nuts.

Hence, the management came to some of us in the quality team and said "do something about that", and we did. The names you see in the patent were involved with designing the solution. We of course started with a review of the available software (this was 1998-1999) and came up empty handed in terms of job-execution-scheduling packages that could dispatch jobs, including ones needing to put GUI's onto the local display, to Solaris, Linux and Windows.

What we did was turn to Jini, which was newly released at the time. The hype around Jini at the time was about embedded devices ... the popular vision being you'd have printers with Jini services built in, the printer sits on the network, whatever device you're carrying around could query the network with Jini for devices that offer a "PRINT" service, and that would solve world hunger, free the slaves, and so on. But that's not quite what we did with Jini because I saw it could be used for our purposes.

In our case we wanted to build a system that: supported a self-organizing set of test systems ... allowed us to submit a job based on a description of test system characteristics ... by matching a job by test system characteristics, the job could be dispatched to any matching test computer ... hence it would let us "scale" the test execution capabilities of our test network simply by adding new computers ... since it's a self organizing set of test systems, we can add and remove test systems at will and the system will adjust itself automatically.

Jini offered the foundational building blocks to let us easily build that system. Each test system has a "machine service" which advertises the existance of the test system. What it advertises are a set of properties describing the characteristics (e.g. operating system, os version, CPU, graphics card, etc). A Jini lookup service collects the advertisements of test systems. And the other leg is a controller which collects jobs to run, finds matching test systems by looking in the lookup service, and dispatches the jobs to test systems. It dispatches the job by sending a request to an agent on the test system, whose job is to handle test execution, it runs the specific test harness for the suite, handles collecting the results, and sending out notifications.

We've been using DTF successfully for several years now and gotten a lot of mileage out of it.

Still, I'm astonished at having received a patent. First, I still don't think this was patentable since we were drawing on a long history of job scheduling systems. I, for one, was reminiscing while designing the controller about punched cards fed into an IBM 370 and how I used to name my jobs "mine" so that I could point at the job status monitor and exclaim "that job's mine". Okay, so I'm a little strange.

But it dawned on me this evening ... if one were to say things aren't patentable because you simply used or built on existing tools ... then nothing would ever get patented. On the other hand, every new tool that comes out, Jini for example, allows for new combinations of ideas to click together and the new tool allows people to design new systems with features not available before because the new tool enables those new features. Surely it means something about patentability that I used the word "new" five times in that sentence.

This might not be the end of the patent story here. We filed for four patents, so there's a decent chance of more to be approved.

Hopefully the patents are narrowly enough defined so it doesn't stop too many of y'all from implementing your own systems.

The Java quality team in Bangalore

I've written a couple posts about my trip here to Bangalore. I've been talking about the life in India, as seen through my western eyes. While there's such a wide variety of stuff like that I could talk about ... I also want to talk about the Java Quality Team here in Bangalore. It's these people I've spent the last two weeks with.

We have perhaps 30 people in Bangalore, and one more in Hyderabad. Unfortunately I could not meet him, unless I were to have traveled to Hyderabad. So far as I know the Quality team has been here for around 5 years.

The bulk of the team members are in SQE, that is, Software Quality Engineering. We have a distinction between QE and QA, where QA is more focused on executing tests while the QE is more focused on creating tests. Our product is Java, and the user interface to that product is the Java API's, hence the test suites we write start from the API's, and hence the QE team is largely equivalent to software developers. The difference is the software we develop is tests meant to exercise and test the Java platform.

The QE organization is divided by the functional area of Java. So we have a team to cover Swing, another AWT, another 2D, another the libraries, another the tools, another the VM, another for JDBC, and so on.

The SQE teams represented in the Bangalore office are Swing, AWT, 2D, DragAndDrop (DnD), libraries, tools and networking.

There are two other teams here to discuss besides SQE.

The first is the Update Release team. You may or may not know this, both 1.4.2 and 1.5 are in maintanence mode. In Sun's product life cycle model, they are in the "sustain" phase where active development has finished, and there is a sustaining engineering team working to fix bugs.

Both 1.4.2 and 1.5 have an update release every 2 months. I see that 1.4.2 is at update 10 (a.k.a. 1.4.2_10) while 1.5 is at update 5 (a.k.a. 1.5.0_05).

The fixes that go into the update releases are developed by the Sustaining Engineering team. Sustaining is not part of the quality organization, and in any case there are some Sustaining Engineering team members here in Bangalore.

The Update Release quality team works with Sustaining Engineering, and they test the update releases before being shipped. To do so they use the test suites QE created for earlier releases.

The last team to discuss is the team I am a part of. That is, the Infrastructure Team, and the Tools team. Together we create and maintain the infrastructure used by the rest of the quality team, and our charter is creating efficiency and effectiveness in the rest of the organization.

These are the people in Bangalore. We have team members in other cities around the world including Beijing, St. Petersburg and Dublin Ireland.

More on Bangalore

I'm still visiting the Quality team in Bangalore, and want to share some more observations about life here. A significant portion of the Java SE quality team, sustaining engineering and even development engineering teams are located here in Bangalore. It seems to me worthwhile to talk about life here so that you who use Java can see a little about where those of us who create Java live.

Yesterday I bought a guidebook that covers tourist spots in Karnataka. Karnataka is the Indian State of which Bangalore is the capital. The guidebook is a hoot, because it describes Bangalore as one of the "prettiest and cleanest cities in India", it's popularly called the "Garden City", etc. I can't imagine those epithets applied to the Bangalore of today. The people here complain endlessly about the traffic, and often simply don't go out on weekends to avoid the traffic. The main strategy to get through traffic is to ride a two-wheeler (scooter or motorcycle), which allows the rider to slip through cracks in the traffic cars cannot navigate.

Bangalore is clearly a melting pot, as it has drawn people from all over India and it regularly sees foreign visitors, such as myself, here on business trips. This city may be more cosmopolitan than other Indian cities, but I don't have much to go by.

There is a lot of "IT" corporations located here, and a massive boom going on in all sectors. Back when the guidebook was written perhaps those epithets were true, but Banglore (according to an article in todays newspaper) has over 7.5 million people and is on track to house 10+ million people. Another indicator is the traffic at the airport. Bangalore is in the process of designing and building a new International Airport. According to a newspaper article the other day, the original airport traffic projections were for 5 million visitors by 2010, but today they're seeing 4.5 million visitors per year. Clearly the growth rate is outstripping their projections.

Most of the roads are in bad repair, large parts of the city appear to be in a constant state of crumble, which looks to me to be simple overwhelm. But at the same time this isn't true for the whole city, as there are parts that are in good conditions. Further the city was affected by recent heavy rains, with flooding that badly impacted the roads.

Other infrastructure issues are the power system and communications system.

Power flickerings are a constant issue. At the office or at my hotel (it doesn't matter where) the power will regularly flick off. The amazing thing, to me, is for the people to not break stride, they just keep going at whatever they're doing. These power flickers must be so common they don't notice, and sure enough a couple minutes later the power flickers back on and they keep going at what they're doing. At the office the power flickers only for part of the systems, as the power for their desktop computers is somehow able to keep going.

In the communications arena, I'm told that it's very difficult to get wired communications going in ones home. The minimum wait to get a DSL line is 1 month (or longer), and I've heard stories about people who had just moved to Bangalore and the government telephone agency refused to give them a phone line. As a result mobile phones are very popular. But what of broadband Internet service?

Clearly if the future is information technology work, especially here in Bangalore which is the outsourcing poster-child city of the world, then broadband Internet service is a must. Even more of a "must" is traffic, because if you're going to give people jobs, they need to be able to get to work with relative ease. Ready access to broadband Internet service can go a ways to reducing traffic loads, as us IT workers can often work from home.

The final thing I want to discuss is culture.

Indian culture is very complex, being a hodgepodge of traditions dating back thousands of years. These people are not westerners, and "we" should not expect them to act as westerners do. Which means for interesting culture mashups.

For example there's the issue of the Urban Cattle. They really do have cows and bulls wandering around city streets, and it's just part of life here. So what happens when a cow decides to cross the street? The drivers stop and let it cross. Or what if it's sleeping in the median? The drivers navigate around the cow. What happens when a driver hits a cow? I was told that the onlookers will immediately pummel the driver to death. Hmmm... Sacred cows indeed!

But this goes way beyond cows. India has hundreds of languages, making for interesting communication between coworkers. In college my geography course instructor was a Jamaican woman whose country was also greatly affected by British Colonialism. She had nothing nice to say about the British Empire but for one thing. They gave English to the people of India, meaning that they all have English as a second language, and now the people of India have a way to talk with each other.

Some practical usage to findbugs

A few months ago I posted some initial impressions on Find Bugs. Since then my backburner task has been to study how to use the tool with the tools developed by the Java Quality Team.

The Java Quality team has several tools (harnesses, test frameworks, etc) to aid the work we do. I've discussed bits about them before, so I won't go into that again. Suffice to say the managers were impressed by Bill Pugh's demonstration when he visited us last summer and they're interested in seeing it or similar tools used in developing those tools. We are of course wanting to have our tools be robust etc so that the team can focus on their real job, finding bugs in Java, rather than fighting with flaky tools.

The simplest use is to simply run findbugs on your current build, and use the GUI to browse through the results. The GUI is fairly nice and that's easy enough to do.

In addition to the GUI there are several shell scripts in the distribution. Unfortunately they aren't well documented, with the only clue to the function often being the USAGE message and the source code.

Suppose you want to see how you're doing over time?

For example in Bill Pugh's demo, what probably caught the eye of the managers was his graph of infinite loops in the JDK. He's been tracking the bugs in the JDK over a long period of time. The chart he showed was the number of infinite loops in each JDK build, which was rather eye-opening to see. The problem is that the findbugs distribution doesn't include a method to generate those charts.

I just worked out a method to generate the chart.

First, you run the findbugs tool on several builds of your application. For each, you save the bugs found into a file. The file format is XML of a somewhat obvious format, which isn't important at the moment.

Next you run the "countBugs" script once for each set of bugs you saved previously. This script counts the number of bugs for each classification, giving you a simple text output.

Last you manually construct a spreadsheet. I found it easiest to use rows for the bug classification, and columns for the build number. I used the Star Office graphing facility and made some pretty nice graphs. To graph the counts for a specific bug type I'd select the row for the bug type, and select the first row which has the build names as column headers. Then select the INSERT/CHART menu, and click through the wizard, make sure to select labels from both the first row and first column.

There are more scripts but it isn't entirely clear how to make good use of them.

For example "aliasedWarnings" seems to find duplicate bugs, but it isn't clear what the matching criteria is.

And "updateBugs" appears to merge a set of results gathered over time, but it isn't clear what you can do with the resulting file.

Visiting the Quality team in Bangalore

Sun is a global company, and we have engineering centers all over the world. One of the biggest of these is in Bangalore, and the Java SE team has a lot of people here. I'm visiting here this week and next to meet with and work with some of my colleagues here.

In some ways Bangalore couldn't be more inconveniently located for collaboration with a team based in California. India is basically 12 timezones away from California. Hence, when we have conference call meetings half of the meeting is getting ready to sleep, and the other half of the meeting just woke up. The ones who are just getting ready to sleep have their evenings stolen from them, the job intruding on their personal time, while the ones who just woke up are having to change their morning ritual to accomodate the job. At least that's my experience.

But at least this gives me the chance to traval to India and learn more about this wonderful country.

In many ways the offices here are very modern. They have the same posters hanging in the walls of the Bangalore office that we have hanging on the walls in Santa Clara. They work in cubicles, have the same advanced equipment we have, and so on.

There's some interesting things to the offices here ... such as the food vendors that come to the lunchroom offering Indian food for prices that I find ridiculously low. Lunch today was 25 rupees, about 75 cents, for a full meal.

This building houses not just Sun Microsystems, but also Ernst and Young and several smaller companies. Next door Cisco has the whole building. Around town you have a who's-who of the tech industry each having engineering centers here. e.g. the building I saw the other day housing AMD in Bangalore. AMD's main headquarters is about 2 miles from Sun's Santa Clara campus where I work. The people explain to me that Bangalore used to be a quiet place, a place where retirees would come to live in peace and calm. But then the IT Industry came to town, drawn by the moderate climate, and set off an explosion of growth here. Now the streets are jammed, the infrastructure overloaded, and Bangalore seems like it's totally overwhelmed by the people living here.

The Java team has engineers here from all branches of the organization. This includes not just the Quality team, but development and sustaining. My team, the Quality team, has something like 20-30 engineers here. We share test development and execution tasks between the team in Santa Clara, and the team here in Bangalore. The Quality team also has engineers in Saint Petersburg Russia, Beijing China, and Dublin Ireland. In the past we had engineers in Novosibirsk Russia, and in Tokyo Japan, but reorgs and other business manglings dispersed those connections. All this makes for interesting juggling of our daily schedules, such as I described earlier.

An interesting development is the return of Indian expats to India. Among the Quality Team members here in Bangalore, 5 or more used to work for the team in Santa Clara CA, and now work for us here in Bangalore. In some cases immigration or visa issues forced their return, in other cases they were drawn back by the boom atmosphere in Bangalore, and in other cases they wanted to raise their children in India rather than the U.S.

January 2006

Search this blog:

Archives

January 2006
December 2005
November 2005
October 2005
September 2005
August 2005
July 2005

Recent Entries

Mustang Beta approaching - we want to know about your bugs and regressions

A useful testing technique to find deadlocks related to invokeAndWait

Developer scenario testing

This work is licensed under a Creative Commons License.