<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
You might write a separate program whose only function is to write
files of that size and do the timings for you.<br>
<br>
Btw, you are aware that there are only so many writes you can do to
SSD before they're used up, right? If this is mission critical
data, you may want to keep frequent backups on other media.<br>
<br>
Still, having huge SSD's, and several of them to boot, makes me
jealous! :)<br>
<br>
David M.<br>
<br>
<br>
<div class="moz-cite-prefix">Stephan Henning wrote:<br>
</div>
<blockquote
cite="mid:CACu1UD7o90nrzQL9cw2Y_9PQP46w=gt8_C95TNx1hmLFaUTZsw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>The program will write out a file of variable size,
it's based on the problem being run. Currently, it writes
out approximately 1.5TB for the benchmark problem, most of
that contained in a single file, much too large for a
ramdisk. Unfortunately, the problems have grown so large
that they can't be run in ram any more. This is a GPU
accelerated program so this file gets modified very
heavily during the process of a run. <br>
<br>
</div>
Current testing is being done on a Raid0 of 5c Crucial 960G
SSDs. This has proven to be significantly faster than the
old array, but I am trying to determine exactly how hard the
disks are being hammered so I can try and optimize the
hardware configuration. <br>
<br>
</div>
The program is compiled from source, but I'm not involved in
that process, I'd much rather try and piggyback something and
monitor the process than try and go in and have something
added to source. <br>
<br>
</div>
I'll add parted and gparted to my list of things to read up on,
thanks. <br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Thu, Dec 12, 2013 at 12:29 AM, David
<span dir="ltr"><<a moz-do-not-send="true"
href="mailto:ainut@knology.net" target="_blank">ainut@knology.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Excellent approach.
<div>
<div class="h5"><br>
<br>
<br>
<div>Arthur wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">How big are the files that you're
dealing with?
<div>If they're small you can just make a ramdisk
and try running everything in there.</div>
<div>It's not a final solution, but between that
and strace you should be able to see if that's
really the issue or not.</div>
<div><br>
</div>
<div>Are you compiling from source? If you are,
then there are a bunch of debugging tools you
can use as well as doing things like timing
individual commands, and seeing how many times
each line of code is run.</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Wed, Dec 11, 2013 at
10:48 PM, Stephan Henning <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:shenning@gmail.com"
target="_blank">shenning@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="ltr">This is a RedHat6 Enterprise
install.
<div><br>
</div>
<div>I don't think htop has the data I need,
but I'll check. I'm not familiar with ntop
and I didn't consider using trace for
this, I'll check that as well.</div>
<div><br>
</div>
<div>The goal is to record read/write rates
and block sizes. I'm pretty sure I am
bottlenecking against the drive array, I'm
hoping I can get some definitive answers
from this. </div>
</div>
<div>
<div>
<div class="gmail_extra"> <br>
<br>
<div class="gmail_quote">On Wed, Dec 11,
2013 at 6:01 PM, David <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:ainut@knology.net"
target="_blank">ainut@knology.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000"> ntop might do the
trick, but not available in
Fedora.
<div>
<div><br>
<br>
<br>
<div>David wrote:<br>
</div>
<blockquote type="cite"> Can
'htop' show open files?<br>
<br>
For intensive live net data,
look at WireShark for linux.<br>
<br>
<br>
<div>David wrote:<br>
</div>
<blockquote type="cite"> If
that's what you're looking
for, there are several
(free) programs you could
run from the command line
in a separate
window/screen while your
program is running that
give you all you're asking
about. Sort of an
equivalent to Winblows
"System Explorer." What
flavor or Linux are you
using? <br>
<br>
David M.<br>
<br>
<br>
<div>Devin Boyer wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Try
something like "strace
-T myapp" or "strace
-T -c myapp"; they'll
show the system calls
being made and the
amount of time spent
in each. It's
slightly different
information than
iostat, but it may be
useful in figuring out
what and where your
program is performing
io access.</div>
<div class="gmail_extra"><br>
<br>
<div
class="gmail_quote">On
Wed, Dec 11, 2013 at
3:37 PM, Stephan
Henning <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:shenning@gmail.com" target="_blank">shenning@gmail.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0
0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>No,
iostat will
normally just
dump to the
terminal
window, but
I'd like to
pipe it's
output to a
file so I can
parse it
later. <br>
<br>
</div>
My end goal
here is to be
able to
generate a log
of iostat
output while I
run this
program, I'm
trying to
determine
exactly how
hard this
program is
hitting my
harddrive and
at what points
during it's
run does it
access the
drive the most
frequently. <br>
<br>
</div>
I've done
something
similar in
bash before,
but it is
rather clunky.
<br>
<br>
</div>
I'll take a look
at exec and see
if I can use it.
<br>
</div>
<div>
<div>
<div
class="gmail_extra">
<br>
<br>
<div
class="gmail_quote">On
Wed, Dec 11,
2013 at 4:46
PM, David <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:ainut@knology.net" target="_blank">ainut@knology.net</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div
bgcolor="#FFFFFF"
text="#000000">
Do you need to
do anything
with the
results or
just need them
displayed?<br>
If you need to
manipulate the
results,
consider using
Perl, <br>
or if C or
C++,<br>
in your 'exec'
call, pipe the
output to a
file, then
just read that
file into your
program.<br>
Ain't UNIX
great?
<div><br>
<br>
David M.<br>
<br>
<br>
<div>Stephan
Henning wrote:<br>
</div>
</div>
<blockquote
type="cite">
<div>
<div dir="ltr">
<div>I'd like
to take some
metrics with
iostat while I
have a
specific
program
running, is
there a way to
wrap iostat
around another
program (it is
called from
the command
line) so that
iostat ends
when the
program
finishes
running? <br>
<br>
</div>
I know I can
do it with a
bash script,
but I'm hoping
for a more
elegant
solution. <br>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
<div>
<pre>_______________________________________________
General mailing list
<a moz-do-not-send="true" href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a>
<a moz-do-not-send="true" href="http://lists.makerslocal.org/mailman/listinfo/general" target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a></pre>
</div>
</blockquote>
<br>
</div>
<br>
_______________________________________________<br>
General
mailing list<br>
<a
moz-do-not-send="true"
href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a><br>
<a
moz-do-not-send="true"
href="http://lists.makerslocal.org/mailman/listinfo/general"
target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a><br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
General mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a><br>
<a
moz-do-not-send="true"
href="http://lists.makerslocal.org/mailman/listinfo/general"
target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a><br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
General mailing list
<a moz-do-not-send="true" href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a>
<a moz-do-not-send="true" href="http://lists.makerslocal.org/mailman/listinfo/general" target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a></pre>
</blockquote>
<br>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
General mailing list
<a moz-do-not-send="true" href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a>
<a moz-do-not-send="true" href="http://lists.makerslocal.org/mailman/listinfo/general" target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a></pre>
</blockquote>
<br>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
General mailing list
<a moz-do-not-send="true" href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a>
<a moz-do-not-send="true" href="http://lists.makerslocal.org/mailman/listinfo/general" target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a></pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
General mailing list<br>
<a moz-do-not-send="true"
href="mailto:General@lists.makerslocal.org"
target="_blank">General@lists.makerslocal.org</a><br>
<a moz-do-not-send="true"
href="http://lists.makerslocal.org/mailman/listinfo/general"
target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a><br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
General mailing list<br>
<a moz-do-not-send="true"
href="mailto:General@lists.makerslocal.org"
target="_blank">General@lists.makerslocal.org</a><br>
<a moz-do-not-send="true"
href="http://lists.makerslocal.org/mailman/listinfo/general"
target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a><br>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
Sincerely,<br>
Arthur Moore<br>
<a moz-do-not-send="true"
href="tel:%28256%29%20277-1001"
value="+12562771001" target="_blank">(256)
277-1001</a><br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
General mailing list
<a moz-do-not-send="true" href="mailto:General@lists.makerslocal.org" target="_blank">General@lists.makerslocal.org</a>
<a moz-do-not-send="true" href="http://lists.makerslocal.org/mailman/listinfo/general" target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a></pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
General mailing list<br>
<a moz-do-not-send="true"
href="mailto:General@lists.makerslocal.org">General@lists.makerslocal.org</a><br>
<a moz-do-not-send="true"
href="http://lists.makerslocal.org/mailman/listinfo/general"
target="_blank">http://lists.makerslocal.org/mailman/listinfo/general</a><br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
General mailing list
<a class="moz-txt-link-abbreviated" href="mailto:General@lists.makerslocal.org">General@lists.makerslocal.org</a>
<a class="moz-txt-link-freetext" href="http://lists.makerslocal.org/mailman/listinfo/general">http://lists.makerslocal.org/mailman/listinfo/general</a></pre>
</blockquote>
<br>
</body>
</html>