[Git][NTPsec/ntpsec][master] ntpviz: 30% speedup. Better to write plot to tmp file than pipe it.

[Gary E. Miller]

Gary E. Miller pushed to branch master at NTPsec / ntpsec


Commits:
74d433d1 by Gary E. Miller at 2016-10-19T12:52:12-07:00
ntpviz: 30% speedup.  Better to write plot to tmp file than pipe it.

My guess is that the buffering on pipes and stdin is the problem that
got avoided.

Bonus, we have a copy of the plot file for debug puproses.

- - - - -


1 changed file:

- ntpstats/ntpviz


Changes:

=====================================
ntpstats/ntpviz
=====================================
--- a/ntpstats/ntpviz
+++ b/ntpstats/ntpviz
@@ -26,7 +26,9 @@ from __future__ import print_function, division
 import argparse
 import csv
 import re
-import atexit, binascii, collections, os, socket, sys, time
+import atexit, binascii, collections, os, socket, sys
+import time
+import tempfile
 from ntp.statfiles import *
 
 # overload ArgumentParser
@@ -265,22 +267,28 @@ def gnuplot(template, outfile=None):
         out = None
     else:
         out = open(outfile, "w")
-    # shell=True is a security hazard
-    # would be great to capture stderr, but the Python doc says
-    # that can lead to deadlock on large stderr output.  gnuplot
-    # can output a lot to stderr...
-    # bigger buffers make it slower, use the default
-    proc = subprocess.Popen("gnuplot", shell=False,
-                            stdin=subprocess.PIPE, stdout=out)
-    v = sys.version_info
-    if 3 <= v[0]:
-        # next line works in Python 3, not Python 2
-        proc.stdin.write(template.encode('ascii', 'ignore'))
+    ##
+
+    # can be 30% faster to write to a tmp file than to pipe to gnuplot
+    # bonus, we can keep the plot file for debug.
+    tmp_file, tmp_filename = tempfile.mkstemp( suffix='.plt')
+    # note that tmp_file is a file handle, it is not a file object
+    os.write( tmp_file, template)
+    os.close(tmp_file)
+
+    # shell=True is a security hazard, do not use
+    rcode = subprocess.call( ['gnuplot', tmp_filename], stdout=out)
+
+    if 0 != rcode:
+        sys.stderr.write("ntpviz: WARNING: plot returned %s\n" % rcode)
+        sys.stderr.write("ntpviz: WARNING: plot file %s\n" % tmp_filename)
+    elif 2 <= args.debug_level:
+        sys.stderr.write("ntpviz: INFO: plot file %s\n" % tmp_filename)
     else:
-        # next line works in Python 2, not Python 3
-        proc.stdin.write(template)
-    proc.stdin.close()
-    return proc.wait()
+        # remove tmp file
+        os.remove(tmp_filename)
+        
+    return rcode
 
 class NTPViz(NTPStats):
     "Class for visualizing statistics from a single server."



View it on GitLab: https://gitlab.com/NTPsec/ntpsec/commit/74d433d11a934db9002137d4a26a1be479ae23d7
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ntpsec.org/pipermail/vc/attachments/20161019/aec74581/attachment.html>