[General] Any linux gurus?

Fri Dec 13 12:01:15 CST 2013

I just wonder if there is a better file system out there for what you
are doing.  Ext4 is nice but...

I just may look into it if you already have not.

Especially if you are crunching all this data how are you making sure
it is not getting corrupted on the disks?  ZFS is nice but I am not
sure it is suited exactly for fast block access and its linux
implementation...I do not know how fast it is now.

If you are not worried about silent data corruption or other
types...you could look into something else too.

I might also think about building fast data buffers into the software
or data storage solution.  You may be already doing that with the SSDs
but can you buffer anything into memory before the problem is solved
and have the GPU's start crunching before everything is there?  Is it
a total bits needed type of thing or can it start with a section and
give the fs to memory proccess time to catch up?  The way you where
talking that how long it takes it really not known as you do not know
when it will solve it.

I dont know...just some ideas.  Sounds really freaking cool.

Web.....

On Fri, Dec 13, 2013 at 7:13 AM, Stephan Henning <shenning at gmail.com> wrote:
> Method of Moment, Computational ElectroMagnertics.
>
> Program is called Vlox
>
>
> On Fri, Dec 13, 2013 at 10:47 AM, David <ainut at knology.net> wrote:
>>
>> MoM CEM vlox -- could you expand those acronyms, please?  Is this a
>> logistics planning tool?
>>
>>
>>
>> Stephan Henning wrote:
>>
>> -David
>>
>> Hmm, sounds interesting. The problem is distributed a little currently,
>> you can think of it kind of what is being done as a form of monte carlo, so
>> the same run will get repeated many times with light parameter adjustments.
>> Each of these can be distributed out to the compute nodes very easily,
>> currently this is being done with condor.
>>
>>
>> -James
>>
>> It's a MoM CEM tool called vlox.
>>
>>
>>
>> On Fri, Dec 13, 2013 at 5:43 AM, James Fluhler <j.fluhler at gmail.com>
>> wrote:
>>>
>>> I'm just curious what simulation program are you running? I've used a
>>> number in the past that also utilize the GPU's for processing.
>>>
>>> James F.
>>>
>>> On Dec 12, 2013, at 11:28 PM, David <ainut at knology.net> wrote:
>>>
>>> IIRC, the good thing about this cluster is the automagic load leveling.
>>> Your existing binary may not run at max optimization but if the task can be
>>> spread among processors, Beowulf does a nice job of it.  If each computer
>>> has it's own GPU(s), then all the better.
>>>
>>> You can test it right there without changing anything on the system's
>>> disks.  Just create and run all the cluster members off a CD.
>>>
>>> Then to test, pick the fastest one of them (maybe even your existing Xeon
>>> box), run your benchmark, record execution time, then boot all the other
>>> machines in the cluster and run it again.  There are only about two dozen
>>> steps to set it up.  One professor even put most of those, along with
>>> automatic cluster setup(!) as a downloadable you can boot off of.  That
>>> leaves half a dozen steps to tweak the cluster together, then you're good to
>>> go.  I have one of those CD's around here somewhere and I can get details if
>>> you're interested.  Something to play with.  I did it with only 4 pc's
>>> around the house with some code and even though the code was never designed
>>> for a cluster (just multiprocessing), I got about 40% decrease in execution
>>> time.  The code was almost completely linear execution so I'm surprised it
>>> got any improvement but it did.
>>>
>>> David
>>>
>>>
>>> Stephan Henning wrote:
>>>
>>> -WD
>>>
>>> I believe it's either ext3 or ext4, I'd have to ssh in and check when I
>>> get back on Monday.
>>>
>>> -David
>>>
>>> I'll check into the Beowulf and see what that would entail. I'll try and
>>> talk with the developer and see what their thoughts are on the feasibility
>>> of running it on a cluster. They may have already gone down this path and
>>> rejected it, but I'll check anyway.
>>>
>>>
>>> On Thu, Dec 12, 2013 at 6:16 PM, David <ainut at knology.net> wrote:
>>>>
>>>> Sounds like a perfect candidate for a Beowulf cluster to me.  There are
>>>> possibly some gotcha's but you'll have the same problems with just a single
>>>> computer.
>>>>
>>>> Velly intewesting.
>>>>
>>>> Stephan Henning wrote:
>>>>>
>>>>> -WD
>>>>>
>>>>> The GPUs are sent data in chunks that they then process and return. The
>>>>> time it takes a GPU to process a chunk can vary, so I assume the bottle
>>>>> necks we were seeing was when several of the GPU cores would finish at about
>>>>> the same time and request a new chunk and the chunk they needed wasn't
>>>>> already in RAM, so the drive array would take a heavy hit.
>>>>>
>>>>> Beyond that, I can't really give you a numerical value as to the amount
>>>>> of data they are dumping into the pcie bus.
>>>>>
>>>>>
>>>>> -David
>>>>>
>>>>> Ya, not sure an FPGA exists large enough for this, it would be
>>>>> interesting though.
>>>>>
>>>>> While the process isn't entirely sequential, data previously processed
>>>>> is reused in the processing of other data, so that has kept us away from
>>>>> trying a cluster approach.
>>>>>
>>>>> Depending on the problem, anywhere from minutes per iteration, to weeks
>>>>> per iteration. The weeks long problems are sitting at about 3TB I believe.
>>>>> We've only run benchmark problems on the SSDs up till now, so we haven't had
>>>>> the experience of seeing how they react once they start really getting full.
>>>>>
>>>>> Sadly, 2TB of RAM would not be enough. I looked into this Dell box
>>>>> (http://www8.hp.com/us/en/products/proliant-servers/product-detail.html?oid=4231377#!tab=features
>>>>> <http://www8.hp.com/us/en/products/proliant-servers/product-detail.html?oid=4231377#%21tab=features>)
>>>>> that would take 4TB, but the costs were insane and it can't support enough
>>>>> GPUs to actually do anything with the RAM...
>>>>>
>>>>>
>>>>>
>>>> <<<snip>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> General mailing list
>>>> General at lists.makerslocal.org
>>>> http://lists.makerslocal.org/mailman/listinfo/general
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> General mailing list
>>> General at lists.makerslocal.org
>>> http://lists.makerslocal.org/mailman/listinfo/general
>>>
>>>
>>> _______________________________________________
>>> General mailing list
>>> General at lists.makerslocal.org
>>> http://lists.makerslocal.org/mailman/listinfo/general
>>>
>>>
>>> _______________________________________________
>>> General mailing list
>>> General at lists.makerslocal.org
>>> http://lists.makerslocal.org/mailman/listinfo/general
>>
>>
>>
>>
>> _______________________________________________
>> General mailing list
>> General at lists.makerslocal.org
>> http://lists.makerslocal.org/mailman/listinfo/general
>>
>>
>>
>> _______________________________________________
>> General mailing list
>> General at lists.makerslocal.org
>> http://lists.makerslocal.org/mailman/listinfo/general
>
>
>
> _______________________________________________
> General mailing list
> General at lists.makerslocal.org
> http://lists.makerslocal.org/mailman/listinfo/general