[Git][NTPsec/ntpsec][21C08-stat-duality] 13 commits: CI: shuffle asciidoc classic and py3 jobs to where they will run right ...
James Browning (@jamesb_fe80)
gitlab at mg.gitlab.com
Wed Mar 31 16:21:44 UTC 2021
James Browning pushed to branch 21C08-stat-duality at NTPsec / ntpsec
Commits:
ea8b172e by James Browning at 2021-03-02T11:48:57-08:00
CI: shuffle asciidoc classic and py3 jobs to where they will run right ....
The job that was used to run the asciidoc py3 test got stuck installing
tex of all things and the jobs that were used to run asciidoc classic
tests had silently shifted to py3.
Move the classic jobs to Debian oldstable where the will run for a
while correctly and move the py3 job to AlpineLinux where it does
not have to worry about tex. Also, exploit our docker images for a
small speed boost.
- - - - -
cce9e9a3 by James Browning at 2021-03-23T17:31:39+00:00
ntpq: -c rv should return associd only when variable(s) not passed.
- - - - -
e47ea601 by James Browning at 2021-03-23T11:09:13-07:00
Remove redundant Asciidoc classic job which migrated to Asciidoc py3
- - - - -
4490b839 by James Browning at 2021-03-23T11:11:40-07:00
Ignore scheduled failing 'shell not found' job on Tumbleweed.
- - - - -
f673320a by Richard Laager at 2021-03-24T01:03:51-05:00
Fix a comment
This function clearly runs both python and non-python tests, so
"non-python" is incorrect.
- - - - -
b9a12069 by Matt Selsky at 2021-03-26T02:32:43-04:00
Remove CI targets for development versions of Ubuntu and openSUSE
The development versions of the distributions fail frequently (by design), and
we want to focus our energy on supporting stable distribution releases.
See NTPsec/ntpsec#698
- - - - -
01c29ffa by Richard Laager at 2021-03-28T00:14:42+00:00
jigs.py: Remove unnecessary shebang
This file is not designed to be directly executed.
Refs #670
- - - - -
b566e38a by Richard Laager at 2021-03-28T00:14:42+00:00
Remove shebangs from test/pylib/*
These are not executed directly, but are instead run explicitly under
the target Python.
Refs #670
- - - - -
9b2e1ce2 by Richard Laager at 2021-03-28T00:14:42+00:00
bin_test: Refactor version
This is a pure refactoring step to simplify the upcoming commit.
Refs #670
- - - - -
caa4aba1 by Richard Laager at 2021-03-28T00:14:42+00:00
bin_test: Run Python tests under the target Python
In a Python cross scenario, the tests need to run under the target
Python, not the waf Python in order to be able to load the module (which
was compiled for the target Python).
Fixes #670
- - - - -
1f185940 by Matt Selsky at 2021-03-30T22:16:59+00:00
Switch macOS CI targets to open-beta images
gitlab-com/macos-buildcloud-runners-beta#2
- - - - -
466cb753 by James Browning at 2021-03-31T09:19:57-07:00
WIP DRAFT ntpq: If sysstat duality failks drop to singleton.
- - - - -
e95dec8c by James Browning at 2021-03-31T09:19:58-07:00
WIP DRAFT ntpq: If sysstat duality fails do not drop to singleton ...
Instead print out annoying '???' and crawl forward on stumps.
- - - - -
14 changed files:
- − .dockerfiles/opensuse-tumbleweed
- − .dockerfiles/ubuntu-devel
- .gitlab-ci-docker-images.yml
- .gitlab-ci.yml
- .gitlab-opttest-ci.yml
- ntpclients/ntpq.py
- tests/pylib/jigs.py
- tests/pylib/test_agentx.py
- tests/pylib/test_agentx_packet.py
- tests/pylib/test_ntpc.py
- tests/pylib/test_packet.py
- tests/pylib/test_statfiles.py
- tests/pylib/test_util.py
- wafhelpers/bin_test.py
Changes:
=====================================
.dockerfiles/opensuse-tumbleweed deleted
=====================================
@@ -1,3 +0,0 @@
-FROM opensuse/tumbleweed
-
-RUN zypper -n install bison gcc libopenssl-devel libcap-devel pps-tools-devel python3-devel python3-curses libbsd-devel
=====================================
.dockerfiles/ubuntu-devel deleted
=====================================
@@ -1,4 +0,0 @@
-FROM ubuntu:devel
-
-RUN apt-get update
-RUN apt-get install -y netbase bison gcc libssl-dev libcap-dev pps-tools python3-dev libbsd-dev
=====================================
.gitlab-ci-docker-images.yml
=====================================
@@ -74,9 +74,6 @@ build-gentoo-hardened:
build-opensuse-leap:
<<: *build_definition
-build-opensuse-tumbleweed:
- <<: *build_definition
-
build-python3-rc:
<<: *build_definition
@@ -85,6 +82,3 @@ build-ubuntu-latest:
build-ubuntu-rolling:
<<: *build_definition
-
-build-ubuntu-devel:
- <<: *build_definition
=====================================
.gitlab-ci.yml
=====================================
@@ -76,18 +76,11 @@ AsciiDoc-3-Fedora:
- asciidoc3/asciidoc3 --version || echo -n
- PATH="$PWD/asciidoc3:$PATH" python3 ./waf configure --enable-doc --enable-manpage build
-AsciiDoc-classic2-Alpine:
+AsciiDoc-py3-Alpine:
<<: *job_definition
- image: alpine
- script:
- - apk update && apk add gcc bison musl-dev openssl-dev libcap-dev python2-dev asciidoc
- - python ./waf configure --enable-doc --enable-manpage build
-
-AsciiDoc-classic3-Alpine:
- <<: *job_definition
- image: alpine
+ image: $CI_REGISTRY/ntpsec/ntpsec/alpine
script:
- - apk update && apk add gcc bison musl-dev openssl-dev libcap-dev python3-dev asciidoc
+ - apk update && apk add python3-dev asciidoc
- python3 ./waf configure --enable-doc --enable-manpage build
AsciiDoc-no:
@@ -97,13 +90,21 @@ AsciiDoc-no:
- apk update && apk add gcc bison musl-dev openssl-dev libcap-dev python2-dev
- python ./waf configure --disable-doc --disable-manpage build
-AsciiDoc-py3-Ubuntu:
+AsciiDoc-classic3-Debian-Old:
<<: *job_definition
- image: ubuntu:devel
+ image: $CI_REGISTRY/ntpsec/ntpsec/debian-oldstable
script:
- apt-get update
- - DEBIAN_FRONTEND="noninteractive" apt-get install -y netbase bison gcc libssl-dev libcap-dev pps-tools python3-dev asciidoc-base
- - python3 ./waf configure --enable-doc --enable-manpage build
+ - DEBIAN_FRONTEND="noninteractive" apt-get install -y python3-dev asciidoc-base
+ - python3 ./waf configure --enable-doc --enable-manpage build --disable-nts
+
+AsciiDoc-classic2-Debian-Old:
+ <<: *job_definition
+ image: $CI_REGISTRY/ntpsec/ntpsec/debian-oldstable
+ script:
+ - apt-get update
+ - DEBIAN_FRONTEND="noninteractive" apt-get install -y python2.7-dev asciidoc-base
+ - python2 ./waf configure --enable-doc --enable-manpage build --disable-nts
debian-oldstable-basic:
<<: *job_definition
@@ -189,20 +190,6 @@ ubuntu-rolling-refclocks:
script:
- python3 ./waf configure --disable-doc --disable-manpage --refclock=all build
-ubuntu-devel-basic:
- <<: *job_definition_allow_failure
- image: $CI_REGISTRY/ntpsec/ntpsec/ubuntu-devel
- script:
- - python3 ./waf configure --disable-doc --disable-manpage build --pyshebang "/usr/bin/env python3"
- - python3 ./waf install
- - python3 ./waf uninstall
-
-ubuntu-devel-refclocks:
- <<: *job_definition_allow_failure
- image: $CI_REGISTRY/ntpsec/ntpsec/ubuntu-devel
- script:
- - python3 ./waf configure --disable-doc --disable-manpage --refclock=all build
-
freebsd-11-basic:
<<: *job_definition
script:
@@ -317,20 +304,6 @@ openSUSE-leap-refclocks:
script:
- python3 ./waf configure --disable-doc --disable-manpage --disable-nts --refclock=all build
-openSUSE-tumbleweed-basic:
- <<: *job_definition_allow_failure
- image: $CI_REGISTRY/ntpsec/ntpsec/opensuse-tumbleweed
- script:
- - python3 ./waf configure --disable-doc --disable-manpage build --pyshebang "/usr/bin/env python3"
- - python3 ./waf install
- - python3 ./waf uninstall
-
-openSUSE-tumbleweed-refclocks:
- <<: *job_definition_allow_failure
- image: $CI_REGISTRY/ntpsec/ntpsec/opensuse-tumbleweed
- script:
- - python3 ./waf configure --disable-doc --disable-manpage --refclock=all build
-
debian-stable-basic-32bit:
<<: *job_definition
image: $CI_REGISTRY/ntpsec/ntpsec/debian-stable-i386
@@ -416,6 +389,7 @@ coverity-scan:
macos-basic:
<<: *job_definition
+ image: macos-11-xcode-12
script:
- PKG_CONFIG_PATH=/usr/local/opt/openssl/lib/pkgconfig python ./waf configure --disable-doc --disable-manpage build --pyshebang "/usr/bin/env python3"
- DESTDIR=build/prison python ./waf install
@@ -425,16 +399,17 @@ macos-basic:
- python ./waf uninstall
- rm -rvf build/prison ./prison ../prison
tags:
- - macos
+ - shared-macos-amd64
rules:
- if: '$CI_COMMIT_BRANCH && $CI_PROJECT_PATH == "NTPsec/ntpsec"'
macos-refclocks:
<<: *job_definition
+ image: macos-11-xcode-12
script:
- PKG_CONFIG_PATH=/usr/local/opt/openssl/lib/pkgconfig python ./waf configure --disable-doc --disable-manpage --refclock=all build
tags:
- - macos
+ - shared-macos-amd64
rules:
- if: '$CI_COMMIT_BRANCH && $CI_PROJECT_PATH == "NTPsec/ntpsec"'
=====================================
.gitlab-opttest-ci.yml
=====================================
@@ -39,12 +39,6 @@ opensuse-leap-options-tester:
script:
- zypper -n install bison gcc libopenssl-devel libcap-devel pps-tools-devel python-devel python-curses libseccomp-devel python3-devel python3-curses bash
-opensuse-tumbleweed-options-tester:
- <<: *option_tester
- image: opensuse/tumbleweed
- script:
- - zypper -n install bison gcc libopenssl-devel libcap-devel pps-tools-devel python-devel python-curses libseccomp-devel python3-devel python3-curses bash
-
gentoo-options-tester:
<<: *option_tester
image: gentoo/stage3-amd64
@@ -62,11 +56,12 @@ gentoo-hardened-options-tester:
macos-options-tester:
stage: test
needs: []
+ image: macos-11-xcode-12
script:
- PKG_CONFIG_PATH=/usr/local/opt/openssl/lib/pkgconfig PYTHON=python2.7 tests/option-tester.sh||true
- PKG_CONFIG_PATH=/usr/local/opt/openssl/lib/pkgconfig PYTHON=python3 tests/option-tester.sh
tags:
- - macos
+ - shared-macos-amd64
rules:
- if: '$CI_PIPELINE_SOURCE == "schedule" && $CI_COMMIT_BRANCH && $CI_PROJECT_PATH == "NTPsec/ntpsec"'
@@ -142,10 +137,6 @@ debian-unstable-options-tester:
image: debian:unstable-slim
<<: *deb_option_tester
-ubuntu-devel-options-tester:
- image: ubuntu:devel
- <<: *deb_option_tester
-
ubuntu-latest-options-tester:
image: ubuntu:latest
<<: *deb_option_tester
=====================================
ntpclients/ntpq.py
=====================================
@@ -415,7 +415,7 @@ usage: timeout [ msec ]
items.append((name, (value, rawvalue)))
except ntp.packet.ControlException as e:
if ntp.control.CERR_UNKNOWNVAR == e.errorcode:
- items.append((var, "???"))
+ items.append((var, ("???", None)))
continue
raise e
queried = ntp.util.OrderedDict(items)
@@ -437,17 +437,22 @@ usage: timeout [ msec ]
value2 = queried[name + '_r'][0]
rawvalue2 = queried[name + '_r'][1]
if fmt in (NTP_UINT, NTP_INT, NTP_FLOAT):
- if self.showunits:
- displayvalue = ntp.util.unitifyvar(rawvalue, name)
- displayvalue2 = ntp.util.unitifyvar(rawvalue2, name)
+ if self.showunits and isinstance(rawvalue, (int, float)):
+ display = ntp.util.unitifyvar(rawvalue, name)
else:
- displayvalue = value
- displayvalue2 = value2
- self.say("%13s \t%9d\t%9d\n" %
- (legend, displayvalue, displayvalue2))
+ display = value
+ if self.showunits and isinstance(rawvalue2, (int, float)):
+ display2 = ntp.util.unitifyvar(rawvalue2, name)
+ else:
+ display2 = value2
+ self.say("%13s \t%9s\t%9s\n" %
+ (legend, display, display2))
elif fmt == NTP_UPTIME:
- self.say("%13s %s\t%s\n" % (legend, ntp.util.prettyuptime(
- value), ntp.util.prettyuptime(value2)))
+ self.say("%13s %s\t%s\n" % (legend,
+ value if not isinstance(value, int) else
+ ntp.util.prettyuptime(value),
+ value2 if not isinstance(value2, int) else
+ ntp.util.prettyuptime(value2)))
else:
self.warn("unexpected vc type %s for %s, value %s"
% (fmt, name, value, value2))
@@ -533,6 +538,9 @@ usage: timeout [ msec ]
self.say("%s %s\n" % (legend, modes[value]))
except IndexError:
self.say("%s %s%d\n" % (legend, "mode#", value))
+ elif fmt == NTP_UPTIME:
+ self.say("%13s %s\n" % (legend, ntp.util.prettyuptime(
+ value)))
else:
self.warn("unexpected vc type %s for %s, value %s"
% (fmt, name, value))
@@ -981,9 +989,10 @@ usage: writelist [ assocID ]
associd = self.__assoc_valid(line)
if associd >= 0:
qtype = ntp.ntpc.TYPE_SYS if associd == 0 else ntp.ntpc.TYPE_PEER
+ silence = bool(len(line.split()) >= 2)
# Some scripts written for C ntpq need associd printed here
self.__dolist(line.split()[1:], associd,
- ntp.control.CTL_OP_READVAR, qtype)
+ ntp.control.CTL_OP_READVAR, qtype, quiet=silence)
def help_readvar(self):
self.say("""\
=====================================
tests/pylib/jigs.py
=====================================
@@ -1,4 +1,3 @@
-#! /usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function, division
=====================================
tests/pylib/test_agentx.py
=====================================
@@ -1,4 +1,3 @@
-#! /usr/bin/env python
# -*- coding: utf-8 -*-
import ntp.agentx as AX
=====================================
tests/pylib/test_agentx_packet.py
=====================================
@@ -1,4 +1,3 @@
-#! /usr/bin/env python
# -*- coding: utf-8 -*-
import unittest
=====================================
tests/pylib/test_ntpc.py
=====================================
@@ -1,4 +1,3 @@
-#! /usr/bin/env python
# -*- coding: utf-8 -*-
# SPDX-License-Identifier: BSD-2-Clause
import unittest
=====================================
tests/pylib/test_packet.py
=====================================
@@ -1,4 +1,3 @@
-#! /usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function, division
=====================================
tests/pylib/test_statfiles.py
=====================================
@@ -1,4 +1,3 @@
-#! /usr/bin/env python
# -*- coding: utf-8 -*-
import unittest
=====================================
tests/pylib/test_util.py
=====================================
@@ -1,4 +1,3 @@
-#! /usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function
=====================================
wafhelpers/bin_test.py
=====================================
@@ -6,36 +6,32 @@ import sys
import waflib.Context
import waflib.Logs
import waflib.Utils
-sys.path.insert(0, "%s/main/tests/pylib" % waflib.Context.out_dir)
-import ntp.util
-version = ntp.util.stdversion()
Popen = waflib.Utils.subprocess.Popen
cmd_map = {
- ("main/ntpclients/ntpleapfetch", "--version"): "ntpleapfetch %s\n"
- % version,
- ("main/ntpd/ntpd", "--version"): "ntpd %s\n" % version,
- ("main/ntpfrob/ntpfrob", "-V"): "ntpfrob %s\n" % version,
- ("main/ntptime/ntptime", "-V"): "ntptime %s\n" % version
+ ("main/ntpclients/ntpleapfetch", "--version"): "ntpleapfetch %s\n",
+ ("main/ntpd/ntpd", "--version"): "ntpd %s\n",
+ ("main/ntpfrob/ntpfrob", "-V"): "ntpfrob %s\n",
+ ("main/ntptime/ntptime", "-V"): "ntptime %s\n"
}
cmd_map_python = {
- ("main/ntpclients/ntpdig", "--version"): "ntpdig %s\n" % version,
- ("main/ntpclients/ntpkeygen", "--version"): "ntpkeygen %s\n" % version,
- ("main/ntpclients/ntpq", "--version"): "ntpq %s\n" % version,
- ("main/ntpclients/ntpsnmpd", "--version"): "ntpsnmpd %s\n" % version,
- ("main/ntpclients/ntpsweep", "--version"): "ntpsweep %s\n" % version,
- ("main/ntpclients/ntptrace", "--version"): "ntptrace %s\n" % version,
- ("main/ntpclients/ntpwait", "--version"): "ntpwait %s\n" % version
+ ("main/ntpclients/ntpdig", "--version"): "ntpdig %s\n",
+ ("main/ntpclients/ntpkeygen", "--version"): "ntpkeygen %s\n",
+ ("main/ntpclients/ntpq", "--version"): "ntpq %s\n",
+ ("main/ntpclients/ntpsnmpd", "--version"): "ntpsnmpd %s\n",
+ ("main/ntpclients/ntpsweep", "--version"): "ntpsweep %s\n",
+ ("main/ntpclients/ntptrace", "--version"): "ntptrace %s\n",
+ ("main/ntpclients/ntpwait", "--version"): "ntpwait %s\n"
}
# Need argparse
cmd_map_python_argparse = {
- ("main/ntpclients/ntplogtemp", "--version"): "ntplogtemp %s\n" % version,
- ("main/ntpclients/ntpviz", "--version"): "ntpviz %s\n" % version,
+ ("main/ntpclients/ntplogtemp", "--version"): "ntplogtemp %s\n",
+ ("main/ntpclients/ntpviz", "--version"): "ntpviz %s\n",
}
# Need python curses
cmd_map_python_curses = {
- ("main/ntpclients/ntpmon", "--version"): "ntpmon %s\n" % version,
+ ("main/ntpclients/ntpmon", "--version"): "ntpmon %s\n",
}
test_logs = []
@@ -52,16 +48,17 @@ def bin_test_summary(ctx):
waflib.Logs.pprint(i[0], i[1])
-def run(cmd, expected, pythonic):
- """Run an individual non-python test."""
+def run(cmd, expected, python=None):
+ """Run an individual test."""
+
prefix = "running: " + " ".join(cmd)
if not os.path.exists("%s/%s" % (waflib.Context.out_dir, cmd[0])):
addLog("YELLOW", prefix + " SKIPPING (does not exist)")
return False
- if pythonic:
- cmd = [sys.executable] + list(cmd)
+ if python:
+ cmd = python + list(cmd)
p = Popen(cmd, env={'PYTHONPATH': '%s/main/tests/pylib' %
waflib.Context.out_dir},
universal_newlines=True,
@@ -88,6 +85,18 @@ def cmd_bin_test(ctx):
"""Run a suite of binary tests."""
fails = 0
+ cmd = ctx.env['PYTHON'] + ['-c',
+ 'from __future__ import print_function;'
+ 'import ntp.util;'
+ 'print(ntp.util.stdversion())']
+ p = waflib.Utils.subprocess.Popen(cmd, env={'PYTHONPATH': '%s/main/tests/pylib' %
+ waflib.Context.out_dir},
+ universal_newlines=True,
+ stdin=waflib.Utils.subprocess.PIPE,
+ stdout=waflib.Utils.subprocess.PIPE,
+ stderr=waflib.Utils.subprocess.PIPE, cwd=waflib.Context.out_dir)
+ version = p.communicate()[0].rstrip()
+
if ctx.env['PYTHON_ARGPARSE']:
cmd_map_python.update(cmd_map_python_argparse)
@@ -95,11 +104,11 @@ def cmd_bin_test(ctx):
cmd_map_python.update(cmd_map_python_curses)
for cmd in sorted(cmd_map):
- if not run(cmd, cmd_map[cmd], False):
+ if not run(cmd, cmd_map[cmd] % version):
fails += 1
for cmd in sorted(cmd_map_python):
- if not run(cmd, cmd_map_python[cmd], True):
+ if not run(cmd, cmd_map_python[cmd] % version, ctx.env['PYTHON']):
fails += 1
if 1 == fails:
View it on GitLab: https://gitlab.com/NTPsec/ntpsec/-/compare/b215f493f6d120ab1338a2edae8a3422b701e625...e95dec8c2f00f6432ca8bea5f7f8c212d8ccb448
--
View it on GitLab: https://gitlab.com/NTPsec/ntpsec/-/compare/b215f493f6d120ab1338a2edae8a3422b701e625...e95dec8c2f00f6432ca8bea5f7f8c212d8ccb448
You're receiving this email because of your account on gitlab.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ntpsec.org/pipermail/vc/attachments/20210331/787dda21/attachment-0001.htm>
More information about the vc
mailing list