GCJ: What were the big stories or the big issues in grid computing in 2006, in your eyes?

Claunch: I think the biggest issue pertained to the private sector's use of grid as it became clear that their adoption characteristics were different from those of early-adopter universities and national laboratories. In the private sector, this was a year of anguish over standardization. The question was, should we enforce policies upon grid, or should we tolerate multiple islands of ad hoc grid implementations, even though that may entail some redundancy and conflict?

GCJ: And in your opinion, was there any resolution to that?

Claunch: Well, I think the anguish is understandable. Although standardization is desirable insofar as it reduces complexity, improves consistency and adds value, it must be implemented at the right time. And it's still a bit early for that. The result is, people are adopting a kind of compromise, which I'll call templates. That is, they find a particularly successful implementation out of the several in their firm and use that as a model or a template for new grid users. At the same time, most adopters are practical enough to realize that there may be legitimate reasons to tolerate diversity, whether it's because of the nature of the application or something related to functionality.

GCJ: And what standards do you think would help?

Claunch: It's not an issue of issuing formal standards so much as simply reducing choice. For example, a particular firm may have middleware from two or three different companies. They may have both open source and commercial tools to manage the grid. Or they might have different designs, some of which have dedicated interconnects, or dedicated networking, and others that are not. They might have some dedicated resources, and some shared. Given this diversity, it is obviously in the interest of the firm to standardize to one kind of software, or one kind of approach, or one kind of design, in order to reduce confusion and achieve better repeatability. Unfortunately, there are still legitimate reasons to retain all these choices.

GCJ: Greg Nawrocki, the president of the Globus Consortium, has said that 2006 is a make or break year for grid, in his opinion. Do you agree?

Claunch: I don't agree, because I think grid is already well established as a valuable tool that's used by any number of firms in financial services, manufacturing, life sciences, and on and on, for a growing number of applications. So I don't see this as a decisive step one way or the other.

In fact, even as adoption broadens in those early-adopting industries, we're gaining traction at new adoption points in the market. For example, in the last year and a half, we've started to see the insurance companies looking at grid as they are suddenly forced to tackle problems that are an order of magnitude or bigger than what they've handled before. There is also mounting interest in using grid to enable extreme business intelligence. So I believe grid is losing its niche status as it continues to gain applicability. I just don't see any evidence that grid is lacking some critical mass that could make it a make-or-break year.

There may be some residual frustration amongst groups that have gotten together to either promote the use of grid or to try to establish standards. We've seen a lot of turbulence with, for example, Enterprise Grid Alliance and GGF merging together. So maybe the question is can consortia possibly do anything to move this along faster at this stage in the game to avoid a more gradual adoption and standardization process.

GCJ: Do you think the persistent confusion regarding grid, SOA, virtualization, grid 2.0, and how they might all fit together in a grid environment is part of the problem?

Claunch: I think SOA first arose as an answer to an objection that was frequently raised with respect to adoption. The original objection was that most of the work that people were running in the current environment were big, chunky, monolithic programs, that wouldn't effectively leverage small slices of resource that might be scattered throughout a grid. It was likewise argued that these tasks didn't have the potential to be accelerated by running them in parallel, because they're just one big chunk by design. And that was certainly true in the commercial space.

So services-oriented architecture is an approach based upon applications that are, by their very nature, much more granular, and easily decomposed into small services that are easily distributed. SOA thereby eliminated what seemed like a conceptual barrier to widespread use of grid for commercial processing. At least that's my analysis of why SOA seems to be joined at the hip with grid: it evolved as a two-step project. Once we're in SOA and everything's granular, the thinking goes, then we can easily exploit that to grid.

Why are there certain industries which use grid very actively, for tasks like high-performance computing workloads? It's precisely because those are the areas that have benefited from decades of work in HPC focused on moving toward parallel execution and away from very large traditional vector supercomputers in order to take advantage of clusters and scalable systems. When the applications, the algorithms, the community, and indeed, the entire model, was oriented around parallelism, it was fairly easy to throw it all on a grid and accelerate it.

Now, it's going to take some time for SOA to really replace all of the non-SOA predecessor versions of the applications, so even if that were to completely solve the applicability issue it would still take some time to see the results for grid. So I don't really see them as necessarily connected.

GCJ: And virtualization?

Claunch: With respect to virtualization, there are basically two strategies to better match outstanding work to the available IT resources. One of them is to move the work to where there are free resources, and grid is one of the mechanisms that does that. The alternative is to move resources to where more is needed, in which case you might use a tool such as VMware.

But I should step back here and say that while that can sound confusing, the reason to adopt grid is the overwhelming business advantage afforded by scaling up processing to a level that was previously unreachable. That increase in capacity is more empowering, I think, than the efficiency grid lends to preexisting solutions. The case for more efficiency, while it's important to people, has not impelled spending anywhere near like the promise of competitive advantage has.

And so in almost all instances of early grid adoption, it's been because in financial services someone believes they make more money by using grid to get better answers while they're trading. Or if it's in life sciences, they believe that there's potentially millions of dollars of profit to be made by getting to market quicker, so from the minute some medical journal publishes new information about diseased protein interactions, they get as quickly as possible to a list of candidates to go test in a laboratory. In the manufacturing sector, reducing rework and identifying clashes between the work of different design teams is regarded as highly valuable, not just for efficiency's sake, but because of the outcome, a faster time to market.

So all of the private sector cases of grid deployment I've seen have been driven by a competitive advantage that was previously unavailable. And because the advantage is huge, it's the business considerations, and not necessarily the IT, that warrants putting up with a lot of immaturity or pain. When you contrast that rationale to the alternative objectives of improving efficiency or utilization of traditional work, ROI analysis evidently just hasn't been as persuasive.

GCJ: What about grid 2.0?

Claunch: Again, there's a delusion that if we can establish a really huge pool, then the economics change. Fundamentally, I don't believe IT typically enables economies of scale the same way other kinds of utility activities can. For example, if you are generating electrical power, a bigger generating station is usually going to have a lower cost per kilowatt-hour than a smaller one such that there is an advantage in centralizing. Yet when we look at IT, the economies of scale result from people driving from the big vector supercomputers or from big mainframes down to high volume, smaller scale machines. But these industry-standard, Intel- and AMD-based small machines are the most efficient point for IT, and we're going to parallelism exactly because the economies of scale are inverted in IT.

So I think that one of the premises underlying the predictions that giant utilities will centralize IT resources, is actually at odds with the reality of IT economics.

GCJ: From what you've seen, are most deployments today compute grids, data grids, or application grids?

Claunch: It's mostly used for high-performance, compute-intensive tasks, because adopters are trying to use scale to do something they couldn't do in traditional ways. There are only a few cases in which someone needs to host a pool of data that's so massive, that no one entity on its own can support that. The data challenges facing people today are largely matters of federating access to multiple, distinct groups of data, rather than one single, enormous database. So outside of the national digital mammography archive and a few others, mostly it's on the compute side rather than data side.

GCJ: What is your view and opinion of the Globus Toolkit?

Claunch: So I think the Globus Toolkit experienced a self-induced slowdown in adoption because of the decision to convert to the Web services-based version 3, rather than piggyback on the wide use of version 2 that was out there in the public sector. That change introduced a dichotomy and a conversion process.

I think the second thing that remains fascinating is how different the public and private sectors are. In the private sector, there has been amazingly small take-up of open source for grid functionality. It really is most surprising, because the very same organizations and groups that are slow on grid are the ones that have also embraced Linux, embraced Maui and other open source things for clusters. So they're very open source-friendly, but they have chosen, for reasons that it's difficult for them to articulate, to go with commercial products. And even the emergence of Univa as a support organization has had little effect on that trend. And, of the few data points in open source that I'm seeing in the private sector, I hear Condor more often than anything else, so I think that's still an outstanding question about the private market.

That picture contrasts starkly with he public sector, where we see very wide standardized application of grid functionality across entities, and even among groups and entities that cooperate together. So the behavior is really different in the public sector.

GCJ: What are your hopes for 2007?

Claunch: I hope to see more and more applications of grid benefiting our clients in both the public and private sectors. I also hope 2007 witnesses some breakthroughs in applications, so that grid moves beyond the mostly computation-intensive work that we see today. And, finally, I hope to see these solutions realized in increasingly standard ways, so that the private sector can enjoy the benefits of open standards and open source in much the same way that the public sector has.

close window