[Git][NTPsec/ntpsec][master] 2 commits: Cache the results of peersplit(). 20% speed increase.

[Gary E. Miller]

Gary E. Miller pushed to branch master at NTPsec / ntpsec


Commits:
d569f9a5 by Gary E. Miller at 2016-09-09T20:11:22-07:00
Cache the results of peersplit().  20% speed increase.

Splitting the same 1MB file over and over again the same way is
very inefficient.  So cache it.

It might be good to just peersplit once when peerstats loaded.

- - - - -
7715082c by Gary E. Miller at 2016-09-09T20:12:18-07:00
Sort the peerlist.keys() so the index.html order is logical.

- - - - -


2 changed files:

- ntpstats/ntpviz
- pylib/statfiles.py


Changes:

=====================================
ntpstats/ntpviz
=====================================
--- a/ntpstats/ntpviz
+++ b/ntpstats/ntpviz
@@ -1030,12 +1030,15 @@ ntpviz</a>, part of the <a href="https://www.ntpsec.org/">NTPsec project</a>
             ("local-temps", stats.local_temps_gnuplot()),
             ("peer-offsets", stats.peer_offsets_gnuplot()),
             ]
-        for key in stats.peersplit().keys():
+
+        peerlist = stats.peersplit().keys()
+        peerlist.sort() # sort for output order stability
+        for key in peerlist:
             imagepairs.append(("peer-offset-" + key,
                            stats.peer_offsets_gnuplot([key])))
         imagepairs.append(("peer-jitters",
                            stats.peer_jitters_gnuplot()))
-        for key in stats.peersplit().keys():
+        for key in peerlist:
             plot = stats.peer_jitters_gnuplot([key])
             if len( plot ):
                 imagepairs.append(("peer-jitter-" + key, plot))


=====================================
pylib/statfiles.py
=====================================
--- a/pylib/statfiles.py
+++ b/pylib/statfiles.py
@@ -13,6 +13,7 @@ class NTPStats:
     "Gather statistics for a specified NTP site"
     SecondsInWeek = 24*60*60
     DefaultPeriod = 7*24*60*60
+    peermap = {}    # cached result of peersplit()
     @staticmethod
     def unixize(line, starttime, endtime):
         "Extract first two fields, MJD and seconds past midnight."
@@ -133,13 +134,16 @@ class NTPStats:
         return ret
     def peersplit(self):
         "Return a dictionary mapping peerstats IPs to entry subsets."
-        peermap = {}
+        "This is very expensive, so cache the result"
+        if len( self.peermap):
+            return self.peermap
+
         for line in self.peerstats:
             ip = line.split()[1]
-            if ip not in peermap:
-                peermap[ip] = []
-            peermap[ip].append(line)
-        return peermap
+            if ip not in self.peermap:
+                self.peermap[ip] = []
+            self.peermap[ip].append(line)
+        return self.peermap
     def tempssplit(self):
         "Return a dictionary mapping temperature sources to entry subsets."
         tempsmap = {}



View it on GitLab: https://gitlab.com/NTPsec/ntpsec/compare/b2e9677d958b9489e2f21fdc0792184efa7edcbe...7715082cd5abd64032c7f79b5ee5a6721b87e89d
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ntpsec.org/pipermail/vc/attachments/20160910/13f0b084/attachment.html>