One of the most impressive commercial level Grid efforts underway today is in the UK - in a project jointly funded by the UK Department of Trade and Industry and the British Broadcasting Corporation (BBC). In a recent interview with GCJ, Terry Harmer - Technical Director at the Belfast e-Science Centre - shares his thoughts on this impressive effort, and explains the role of the Globus Toolkit.

GCJ: Tell us a little bit about Belfast e-Science Center's Gridcast project with BBC.

Harmer: The nature of what the BBC does involves large file transfers - high definition files are about 280 gig per hour broadcast these days. That'll only start growing as you add extra services into the actual content themselves... things like multi-language layers, or stories with multiple stories built in. So naturally, the BBC is interested in Grid capabilities like resource sharing, resilience, flexibility and work flow.

But the BBC also wanted to move away from what was a very closed, specialized area, into what is now a very commodity-based IT infrastructure. From the BBC's perspective, they wished to track the way in which IT infrastructures were moving - and determine what the next generation of IT infrastructure for broadcasting might be. And they determined it was one where commodity PCs and commodity servers are used - and they saw Grid and Gridcast as a way to get to that next generation of IT infrastructure.

The Gridcast project is an idea I developed jointly with Stephen Craig of BBC Northern Ireland and owes much to the innovative thinking within BBC Nations and Regions, BBC New Media and BBC Research and Development - the BBC were thinking grid long before they started calling it grid!

GCJ: So they have roughly how many compute centers spread across what geographic region?

Harmer: Well, they have a compute center in most capitals in the world today - wherever they have journalists. So their longer term view is that a field reporter sitting in a field in Iraq can access all the resources [via satellite] that BBC has on the Grid, as if they were sitting in the London office. That was the vision they had.

GCJ: Are they running Linux in these environments typically?

Harmer: It really depends which of their computing centers you're talking about. Windows, of course, is probably the most widely used desktop OS. Linux has niche areas in terms of video rendering. And Macintosh is commonly used in the area of post-production. So it's a very mixed environment.

And in fact, the most appealing aspect of Grid for BBC was the notion of Grid as a greater integration fabric. BBC isn't really viewing Grid from the high performance cycle sharing standpoint. They will be using Grid for those purposes to some extent, but they're more interested in Grid from the standpoint of an integration framework that allows you to tie together the various platforms and deploy your software in an open way. For example, we have a number of trial deployment that are really tying together a number of different technologies including various content management systems that are based on Windows boxes.

GCJ: To what extent are the BBC and Belfast e-Science Centre using the Globus Toolkit?

Harmer: Everything in Gridcast is built using Globus Toolkit. We use it as a means by which we create, define, and deploy services. We have used all the major components that Globus provide. We are big users of GridFTP. All of our projects have used Globus as the middleware of choice. The use of an open, standards-based middleware is important to all of our industrial partners.

GCJ: What do you think about the new advancements in GT4? Do you have any criticisms of how the toolkit has progressed?

Harmer: In terms looking at GT4 compared to GT3 -- GT4 is significantly more robust and significantly more stable than GT3 was. We are very happy with the progress and the incorporation of web services.

My one criticism would be - and this is partly to do with the kind of niche area where Grid is today - in terms of selling commercial partners on the idea of using the Globus Toolkit, it would be useful to have a period of stability for a while, where there aren't new versions coming out. It is difficult going back to partners and saying, well, yes, we were using GT3, but now we need to move to GT4...

So we would like just a period of calm for a while. Our clients such as the BBC are really interested in using the Globus Toolkit, but they don't want to see it or have to keep thinking about new versions. The Toolkit is just plumbing to them, infrastructure. They want to see what it can do, but new iterations coming out all the time is a distraction to them.

GCJ: The Belfast eScience Centre is a publicly funded effort, correct?

Harmer: Yes. We are publicly funded under the UK e-science program. A lot of the funding came from the UK Department of Trade and Industry, which is probably the reason why most of the UK projects have a very industrial focus -- they're required to have 50% of the funding provided by industrial partners. So 50% of Gridcast is funded by BBC, and 50% is funded by the Department of Trade and Industry.

To give full credit for the UK s-Science programme you must look to the vision of John Taylor, previous head of the UK Office of Science and Technology, and proponents such as Tony Hey, the outgoing head of the UK e-Science programme, that realized the vision.

The future of Belfast e-Science will be as a self funding grid R&D entity with a number of commercial spin-out companies.

GCJ: Have you developed any homegrown resource management controllers, or have you guys built anything in-house that you're particularly proud of in a Grid context?

Harmer: Yes, lots of things. We have a very lightweight, very quick workflow engine that's very good for building simple workflows like scripting. A lightweight grid resource monitor we use to enable resource management. We have management tools -- we built a BBC project that actually allows you to do things like deploy, refresh, configuration, and deals with grouping services together and creating them as a pool and deploying them as a pool. We have a number of enhancements in the index service we have and the ways in which - or faster ways of searching collections of index services. Again, we're interested in pushing that back into Globus. So there's a whole range of things we've developed.

GCJ: How many people work on the Gridcast effort?

Harmer: We are a small organization, 20 people in Belfast e-Science with 4 working full time on Gridcast. The BBC have an equivalent number working on Gridcast. We believe we have some of the most dedicated, hardworking and knowledgeable grid developers around.

GCJ: Now, isn't a lot of the integration and building of Grids labor intensive? Does it take a lot of human touch to make these systems work?

Harmer: It is pretty labor-intensive. Some of the tools I've been talking about, like what we call the configuration management system, was built because deploying a Grid is hard work. Two years ago, when we first starting doing this for Gridcast, it was a lot of work because we had never built a Grid in a lab. All of our Grids were out on the Net. Gridcast has systems in two places in London, two places in Northern Ireland, and one in the middle of southern England. So we do have the geographical problem that we really can't go to all those sites all the time, in person. So we have to worry about remote configuration, remote management. We have a grasp on how to handle services, and we're very efficient with that. But management and deployment is going to be pretty labor intensive until more sophisticated tool-sets start to appear.

GCJ: At a recent GGF event, you shared some perspectives on GT3-GT4 migration. How did Belfast e-Science Centre handle its migration?

Harmer: Well, we created a script that automated the task for us -- we're big on automation. I wrote a script in Perl which simply takes the old GWSDL, which is the service description for GT3, and creates the Web services definition, all the deployment definitions -- and then we by hand changed the services. For the systems we have, migration to GT4 has not been a huge issue. Indeed, we found it surprisingly easy. There are issues to deal with in the way the model has changed with GT3, and that has forced us to redesign some of the way in which our services work. But actually the migration itself hasn't been that bad.

GCJ: Beyond the Gridcast effort, what are some other interesting projects that the Belfast e-Science Centre is working on?

Harmer: Belfast e-Science has a very industrial and commercial focus. Gridcast will move into a new phase with the development of a project called PRISM which addresses large-scale field deployed digital content archives, B2B and B2C services. There is an off-shoot of Gridcast that is looking at military media grid infrastructures with QinetiQ. We have a financial service project called RiskGrid which we intend to commercialize. It's looking at risk assessment in terms of real-time risk management for share portfolios and it is in collaboration with a financial services software company called First Derivatives. That's a more traditional Grid-based solution because it's really throwing lots of horsepower at a problem which is quite complex. We've got a real time data mining project called GEDDM with applications in the banking and security sector with a company called Datactics. And we've got a commercial bioinformatics project which is a like a virtual lab between ourselves and commercial biotech companies Fusion Antibodies and Amtec Medical. And all those projects are Globus toolkit based and have deployments in the field. A key part of our work has been to deploy grid in the field rather than in a lab which I think is important to demosntrate grid technology.

You can find a list of our current projects here.

close window