Working on https://github.com/lathama/Adynaton I have setup some testing that runs every time and I want to talk about it. I have worked on software in industries that could benefit from testing, coverage and style checks but the organizational impact causes testing to be a pretty harsh hurdle. Some amazing work has gone into bypassing testing in many industries. Here in this post I am discussing some testing, coverage and style checks that lower the bar and make it easy to use. The testing does not stop the operation or development. The code coverage report is useful. The style check is more of an indicator than anything else. This is a discussion and a process. Enable everyone from support to operations with that warm fuzzy feeling that testing is the new norm. Show the developers that testing is not a blocker.
Using the package initialization I am running testing, code coverage and even style checks on every load of the library.
Housekeeping stuff, let us get a directory to use. Other logic in creating and managing the directory are beyond the scope.
if os.geteuid(): ADYNATON_PATH = os.path.expanduser('~/srv/adynaton/') else: ADYNATON_PATH = '/srv/adynaton/'
First we need to load the code coverage tool called 'coverage' and get it started up.
import coverage if checkcodecoverage: codecoverage = coverage.Coverage(omit="*unittests/*") # Unittests skew results codecoverage.start()
Next we will startup unittesting with some output redirection so that the unittesting does not stop the run. It is much easier to debug things when the unittesting is not the road block so I setup the testing to keep from stopping with exit().
unittesting_log = open(ADYNATON_PATH + 'adynaton_unittesting.log', 'w') unittesting_log.write("Testing Adynaton - " + str(datetime.datetime.now().isoformat(' ')) + "\n") suite = unittest.TestLoader().discover('adynaton.unittests') unittest.TextTestRunner(stream=unittesting_log, descriptions=True, verbosity=6).run(suite) unittesting_log.close() if verbose: unittestinglog = open(ADYNATON_PATH + 'adynaton_unittesting.log', 'r') print(unittestinglog.read())
After the unittesting is done we can stop the code coverage tool and save the report.
codecoverage.stop() codecoverage.save() codecoverage.html_report(directory=ADYNATON_PATH + 'adynaton_coverage')
Lastly we have a style check that will compare the code style to PEP8 the Python code style guide. I am not a huge style enforcer but have found a great side effect in the past. Running a style check can find copy and paste code. Copy and pasting code is not the worst thing out there but can be an indicator that the code needs some review and or security checks.
import pycodestyle checker = pycodestyle.StyleGuide(exclude=['libraries'], quiet=False, ignore=['E501', 'E402']) result = checker.check_files('.') if verbose: print(result)
This testing is an example and I am working to create a better framework example at https://github.com/lathama/python-testing-example but am not there yet. Some points I want to leave you with are:
- Test everything all the time. Lower the barrier and make it normal for all.
- Code coverage is great, make sure to exclude unittesting code as some/many coverage tools will skew by marking unittesting as 100% covered.
- Don't exit on test failure. Enable your developers to work though problems with tests as useful tools not annoying roadblocks to turn off.
- Testing, coverage and style checks can be part of the project, not a requirement for CI/CD solutions.
- It is OK to ignore style items like line length and 100% test coverage is not realistic. Don't set the bar to high.
- Make it easy on everyone and let them grow to want more out of tests.
- From an Operations point of view, running the program on a server after some library upgrades and being able to see the tests pass is priceless.