<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>So is your concern with provisioning and setup or with actual job distribution?<br><br>~mc mobile</div><div><br>On Jan 22, 2015, at 17:15, Stephan Henning <<a href="mailto:shenning@gmail.com">shenning@gmail.com</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr">This is a side project for the office. Sadly, most of this type of work can't be farmed out to external clusters, otherwise we would use it for that. We do currently utilize AWS for some of this type work, but only for internal R&D.<div><br></div><div>This all started when the Intel Edison got released. Some of us were talking about it one day and realized that it <i>might</i> have <i>just enough</i> processing power and ram to handle some of our smaller problems. We've talked about it some more and the discussion has evolved to the point where I've been handed some hours and a small amount of funding to try and implement a 'cluster-in-a-box'. </div><div><br></div><div>The main idea being to rack a whole bunch of mini-itx boards on edge into a 4U chassis (yes, they will fit). Assuming a 2" board-board clearance across the width of the chassis and 1" spacing back-to-front down the depth of a box, I think I could fit 27 boards into a 36" deep chassis, with enough room for the power supplies and interconnects. </div><div><br></div><div>Utilizing embedded motherboards with Atom C2750 8-core CPU's and 16gb of ram per board, that should give me a pretty substantial cluster to play with. Obviously I am starting small, probably with two or three boards running Q2900 4-core cpus until I can get the software side worked out.</div><div><br></div><div>The software-infrastructure side is the part I'm having a hard time with. While there are options out there for how to do this, they are all relatively involved and there isn't an obvious 'best' choice to me right now. Currently our in-house HPC cluster utilizes HTCondor for it's backbone, so I would like to maintain some sort of connection to it. Otherwise, I'm seeing options in the Beowulf and Rocks areas that could be useful, I'm just not sure where to start in all honesty. </div><div><br></div><div>At the end of the day this needs to be relatively easy for us to manage (time spent working on the cluster is time spent not billing the customer) while being easy enough to add notes to, assuming this is a success and I get the OK to expand it to a full 42U racks worth. </div><div><br></div><div><br></div><div>Our current cluster is almost always fully utilized. Currently we've got about a 2 month backlog of jobs on it. </div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 22, 2015 at 4:55 PM, Brian Oborn <span dir="ltr"><<a href="mailto:linuxpunk@gmail.com" target="_blank">linuxpunk@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">If you can keep your utilization high, then your own hardware can be much more cost effective. However, if you end up paying depreciation and maintenance on a cluster that's doing nothing most of the time you'd be better off in the cloud.</div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 22, 2015 at 4:50 PM, Michael Carroll <span dir="ltr"><<a href="mailto:carroll.michael@gmail.com" target="_blank">carroll.michael@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Depending on what you are going to do, it seems like it would make more sense to use AWS or Digital Ocean these days, rather than standing up your own hardware. Maintaining your own hardware sucks.<div><br></div><div>That being said, if you are doing something that requires InfiniBand, then hardware is your only choice :)</div><span><font color="#888888"><div><br></div><div>~mc</div></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 22, 2015 at 4:43 PM, Joshua Pritt <span dir="ltr"><<a href="mailto:ramgarden@gmail.com" target="_blank">ramgarden@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">My friends and I installed a Beowulf cluster on a closet full of Pentium 75 Mhz machines we were donated just for fun many years ago back when Beowulf was just getting popular. We never figured out anything to do with it though...</div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 22, 2015 at 5:31 PM, Brian Oborn <span dir="ltr"><<a href="mailto:linuxpunk@gmail.com" target="_blank">linuxpunk@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">In my previous job I set up several production Beowulf clusters, mainly for particle physics simulations and this has been an area of intense interest for me. I would be excited to help you out and I think I could provide some good assistance.<div><br></div><div>Brian Oborn (aka bobbytables)<br><div><br></div><div class="gmail_extra"><br><div class="gmail_quote"><span>On Thu, Jan 22, 2015 at 4:25 PM, Stephan Henning <span dir="ltr"><<a href="mailto:shenning@gmail.com" target="_blank">shenning@gmail.com</a>></span> wrote:<br></span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span><div dir="ltr">Does anyone on the mailing list have any experience with setting up a cluster computation system? If so and you are willing to humor my questions, I'd greatly appreciate a few minutes of your time. <span><font color="#888888"><div><br></div><div>-stephan</div></font></span></div>
<br></span><span>_______________________________________________<br>
General mailing list<br>
<a href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a><br>
<a href="http://lists.makerslocal.org/mailman/listinfo/general" target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a><br></span></blockquote></div><br></div></div></div>
<br>_______________________________________________<br>
General mailing list<br>
<a href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a><br>
<a href="http://lists.makerslocal.org/mailman/listinfo/general" target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a><br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
General mailing list<br>
<a href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a><br>
<a href="http://lists.makerslocal.org/mailman/listinfo/general" target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a><br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
General mailing list<br>
<a href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a><br>
<a href="http://lists.makerslocal.org/mailman/listinfo/general" target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a><br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
General mailing list<br>
<a href="mailto:General@lists.makerslocal.org">General@lists.makerslocal.org</a><br>
<a href="http://lists.makerslocal.org/mailman/listinfo/general" target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a><br></blockquote></div><br></div>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>General mailing list</span><br><span><a href="mailto:General@lists.makerslocal.org">General@lists.makerslocal.org</a></span><br><span><a href="http://lists.makerslocal.org/mailman/listinfo/general">http://lists.makerslocal.org/mailman/listinfo/general</a></span></div></blockquote></body></html>