GCJ: What, in your opinion, have been the big stories about grid in 2006?

Fellows: From our point of view - when it comes to grid computing in the enterprise - the lack of progress with standards and data management is a big story. We anticipated that there would be further enhancement of grid technologies beyond the compute grid. Instead, what we've seen is the emergence of a sort of a holy trinity of virtualization, orchestration, and provisioning, which is really driving grid at the moment. One could almost argue that what we used to call grid computing is being subsumed by a range of functions that we now call a "platform for shared services."

It has been disappointing. The early adopters we speak to here at The 451 Group are asking the same questions and running into the same problems that they did three years ago. And neither the industry bodies nor commercial products have yet to address them in a significant way.

GCJ: Can you cite any specific examples?

Fellows: At The 451 Group, we have been tracking more than 250 commercial early adopters of grid computing. In our conversations with them, we have found that data management and commercial software licensing remain the key technology challenges. Evidently, social and organizational issues overwhelm all other concerns. But when technology is involved, there will always be demand for a good way of implementing data management for grid computing. This need encompasses everything from the capacity of storage systems to handle multiple incoming requests, to finding connections over wide area networks with suitable latency and throughput, to the ways in which distributed enterprise caches are integrated with existing middleware and data management strategies.

Essentially, there's a huge range of companies that think they've either got the answer, or at least part of it. In reality, no one approach, no group of vendors, nor any single vendor, has achieved any kind of leading position. Early adopters are forced either to choose point products and hand-stitch them together, or to create an entirely in-house solution. It's a bit of a hairball, and it's one of the more important challenges beyond the realm of compute grid that needs to be solved.

GCJ: Greg Nawrocki, the president of the Globus Consortium, has said that calendar year 2006 is a make-or-break year for grid. Do you agree or disagree, and why?

Fellows: Well, I think it's a make-or-break year for organizations and companies that have hung their hats on grid! It'll be interesting to find out what happens to GridWorld next year, and to see if OGF is able to finally deliver some of the things that users have been requesting for such a long time-for instance, improvements to the EGA reference model. It's a fact that companies that have branded their products with the term "grid" are now re-branding, as many observers have noted. Grid is not a funding concept at this point.

Indeed, I think most of the ISVs and middleware companies in grid are now seeking to break into other opportunities. And I think that's really to be expected, as 2006 drove home the realization that grid is a means to an end, not an end in and of itself.

In other words, the way early adopters are moving grid activities forward is by creating either shared internal utilities, or a platform on which to deploy shared services. In more limited deployments, they are either sticking with just HPC grids, or they've opted to outsource some of this capacity. I think the world of grid is certainly moving here.

GCJ: We hear a lot about how grid is up against SOA, virtualization, and Web 2.0. How is that contest perceived today and what will be the outcome?

Fellows: We see grid and all the technologies mentioned with grid attributes - virtualization, resource reallocation, self-management, etc. - providing a technology infrastructure, while SOA and related technologies, effectively provide an application infrastructure.

Certainly a majority of the early adopters we've spoken with equate SOA with web services, which is not necessarily the way other areas of the industry see it. But that's definitely what real users tell us: as far as they're concerned, SOA means web services.

Now, how do we move forward? We think combining the two kinds of technology delivers a more dynamic infrastructure, and frees users and applications from those underlying limitations. It also delivers what vendors are calling service-oriented infrastructure (SOI). The terminology does not matter an awful lot, but the goal of using infrastructure to dynamically provision and use shared services, is definitely the driving force.

It will be interesting to see whether next year will be the end of grid. Within the new model for shared services, the challenge is to meet requirements for the services and applications on top, and to organize resources underneath supporting them below. From this perspective, grid is just another way of organizing resources. And an enterprise grid probably isn't what you would want for this purpose, a more generic shared services environment combining virtualization, orchestration, and provisioning, is key.

GCJ: Without standards, it is sometimes said, the adoption of grid in the data center will stall. Which standards are going to make a difference?

Fellows: Early adopters of grid computing have asked clearly and consistently for some pretty basic standards that would make their lives easier-things like having a common IO, a common way to get data in and out of a grid, and a common stack or interface that would allow their developers to use some common tools. We don't think it makes sense to really plumb the myriad and complex world of standardization, since most early adopters we speak with don't have any interest in it. They do not feel that their needs are being heard, and, consequently, they don't think any of it is practically useful.

We'd like to offer up a challenge to OGF, pick one or two of these key challenges and find a technical solution to these problems. If they don't, then how long before it becomes relevant or someone else does it instead?

GCJ: Which vertical markets are using grid most actively?

Fellows: When we started talking to early adopters in the financial services industry (over three years ago), about 70% of the top 20 investment banks were using grid in some way, shape, or form. Now, nearly 100% of them are using grid. In addition, the use of grid has trickled into other kinds of activities, beyond equities and risk-for instance, into retail banking, asset management, and insurance.

The same applications are emerging in other vertical markets as well. In the energy sector, for example, the big petrochemical companies are increasingly subjected to the real-time supply and demand challenges, as well as the pricing problems, that the financial guys have already handled. And they are now starting to deploy the same kinds of applications and techniques used by financial services in risk management. The appetite in the energy industry for utility computing and outsourcing for additional capacity is pretty keen.

On the other hand, the pharmaceutical industry is more conservative in adopting grid to support other kinds of non-HPC work. In other words, they're doing grid-enabled drug design, but there's little to no interest in doing anything else. The manufacturing industry is concerned with primarily linking their disparate compute resources at this point. And what we call grid, the digital imaging and film industries have been calling rendering for years now. Companies in the telecommunications sector are looking to reinvent themselves as the trusted supplier of IT, given that voice and related technologies is a dwindling business. All of them will be or are using grid and grid techniques moving forward.

Grid adoption is certainly moving ahead significantly in the areas mentioned. Clearly the demand for innovation is most promising in the financial sector and with some key early adopters in the other segments.

GCJ: And from a grid usage, is it mostly compute grids in that field, or are we looking at other type of grids being used?

Fellows: The 451 Group believes that there aren't any other types of grids. It doesn't make sense to aggregate one set of resources and not aggregate another if you are planning to use this technology in a commercial environment. So if the question is - what other kinds of application uses are there beyond conventional HPC? - then quite clearly we do see organizations starting to run more mixed workloads on their grids.

From this perspective, analytics have been the low-hanging fruit for the vendors. There's an opportunity to run the master data management activity on top of the grid, including everything from analytics to business intelligence, customer data integration, and ETL. The activity of large ISVs, like SAP, should make these kinds of applications more readily grid-enabled. We've seen also groups like SalesForce.com and WebEx move to grid models. But the really interesting area of development is at organizations like eBay, Amazon, and Google - where they are using grid models internally and offering grid-enabled, on-demand compute and storage services.

They have certainly caught the attention of the financial guys! What happens when eBay financial markets evolves and eventually cuts them out of the transaction with the customer, so that they are no longer then middlemen?

GCJ: What is your opinion of the Globus Toolkit? And how widespread is its use?

Fellows: The Toolkit is where it all began, But where does it go from here? Among early adopters, only a handful have used it or are considering using it - the jury's still out on whether this thing is going to make it. There have been great expectations placed on a lot of third-party products, but we haven't seen any evidence that they have panned out-or, more importantly, that anyone is using this stuff. Instead we have heard more about the use of products like Condor. To see the shift you only need to look at Univa, the company that was set up to be the Red-Hat of grid, and realize that it now has a slightly different focus. The Globus Consortium has needed to reinvent itself too.

GCJ: Any further predictions, hopes, or expectations on your side for grid in 2007?

Fellows: Increasingly, grid allows organizations to move from silos of applications or resources to horizontally integrated resources and a platform for running shared services with scalability. SOAs are coming, but it's going to take time to turn everything into a service. Clearly, there are immediate benefits from users being able to dodge platform wars and run web services faster, better, cheaper. Increasingly, grid attributes are going to dissolve into the fabric.

close window