Grid computing,

Has any one set up anything like this??

As I mentioned in another post, I am going to be trying to set up a BOINC server, (although I may look at CONDOR as well).

Does any one have any other suggestions. My job is to look at distributed methods (like the two above), while another person is looking at grid/cluster computing like map-reduce, hadoop and gaingrid.

If any one has experience in distributive computing I would be very interested to hear about them.

The Idea is we will be using the distributive systems on 500+ nodes to batch run large repetitive Epidemiology models. I would really like to get a system up and running ASAP to demonstrate to people with test applications in the next few weeks.

Find more posts tagged with

Save $250 on 2025 certification boot camps from Infosec!

Book now with code EOY2025

Button

Comments

Fugazi1000

An interesting project.

If I was presented with this type of problem, I would consider something like EC2 from Amazon.

If you think 500 nodes would be enough the costs of AWS are likely to stack up. Having said that, if you have 500+ desktop clients sitting on desks not being used 70% of the time, then a BOINC type deployment might also work. Getting 'public' PCs to cooperate might not be the way to get a Proof of Concept off the ground.....

Keep this forum posted.....

DevilWAH

Hi there

sorry been playing with this for the last few days and got Boinc running with a demo app.

This would be a completely in house set up, clients will be our own internal desktop machines, many of which are only used for basic word processing and are multi core 1+gig of ram, so would make nice clients.

Pproof of concept is going to be on one BOINC server, attached to 10 managed clients in a protected network. As soon as we have some "real" applications running on it I will try to post some results of the speed gain.

As for using things like EC2, unfortunaly the company I work for will not let data be sent outside the company (we run under a government secure network) so although using EC2 would be great, the cost in terms of risk assessments and demonstrating it is secure just becomes a nightmare.

At the moment the models we are running may take one PC about 7 to 10 days to complete. So even if we only where to have 20 - 30 clients connected to a BOINC server, we could hopefully cut that down to less than a working day.

Like I say I will keep you posted as to how it goes.