Pytest. Between basic and advanced

Hello.

I will write down what I have learned about Pytest so far.
It is to help myself, hopefully others think it is useful in some way and I am also trying to fill in the gap between the often over simple basic examples and the advanced ones.
I miss the “intermediate” level.
Please have in mind, English is not my native language

The basic folder structure looks like this

src/
    basic_math.py

tests/
    __init__.py
    test_basic_math.py

The init file can be empty.
But it is needed for pytest to find files on import.

If you have a lot of files in src/ and want to organise the tests for them you can create subfolders in the tests folder.

src/
    basic_math.py
    advanced_math.py
    geometry.py

tests/
    # a bit of correction here.
    # the  __init__.py file has to be in every folder that have a test_something.py

    test_basic_math/
        __init__.py
        test_basic.py

    test_adv_math/
        __init__.py
        test_adv.py

    test_geometry/
        __init__.py
        test_geometry.py

This is one way to keep some kind of order in the project.

To be continued.

2 Likes

Is this intended to be a personal thread or can we all chip in?

You are welcome.
:+1:

If we help each other the bridge over the gap will be better

Cool. I’ve been using Pytest a bit more so captured some thoughts in this blog post but generally:

  • Running pytest with -v provides a good verbose summary of test execution results
  • Adding a decorator with @pytest.mark.some_keyword and running pytest with -k “some_keyword” will filter tests to be executed and is very dynamic
  • pytest.mark.parametrize(some data) is a great way to iterate over a test where multiple data scenarios are required.

I also had some issues so this little piece on how testing doesn’t always point to errors might be worth a read also.

2 Likes

Thanks for interesting reading @gpkesley.
I knew it was a good idea to let other contribute to this. :smiley:
From your blog I learned both a bit of Python and pytest.

I had plans to write short pieces of the topics you mentioned. I hope you dont mind?
I would be happy if you could have a look when I put up new text to see if there are something wrong or something can be improved or explained better.
This applies to all.
I do this to learn myself. When errors are corrected its useful for all.

Yeah of course! I only started a blog as I was learning so many useful little things from many sources and if I didn’t capture them I would forget.

The plan is to move the lot into my own Django blog once I finish building it. I just got the markdown to add really then i’ll have a specific section for Pytest as I really like it.

Sharing is caring IMO :slight_smile:

1 Like

One cool thing about Pytest is that you can choose wich tests to run. There are some different ways to do this.
In my case we need two imports

# file: test_basic.py

import pytest
from src import basic_math

@pytest.mark.simple_math
     def test_plus(a, b): # Edited. Forgot the "test_" before function name 
        return a + b

@pytest.mark.simple_math
     def test_minus(a, b):
        return a - b
 
@pytest.mark.math
     def test_div(a, b):
        return a / b

@pytest.mark.math
     def test_multiply(a, b):
        return a * b

From the same level as src/ and tests/
Now we can call the test like this:

pytest -v tests/test_basic_math.py

This give us just the four tests written in this file and not any tests in test_adv_math.py or test_geometry.py
The -v flag gives a little bit more details about the test.
One can also specify it a little bit further:

pytest -v tests/test_basic_math.py::test_multiply

Then we got one specific test in a specific test file

The next way to do is to use the -m flag and the pytest.mark.“tag”
“tag” is something you can choose yourself. All test with the same tag will run.

pytest -m ‘simple_math’

One can also choose more tests with:

pytest -m ‘simple_math or math’

pytest -m ‘math and not simple_math’ # This one is a bit redundant. same as:
pytest -m ‘math’

pytest -m ‘not math’

The third way to do is with the -k flag.
In this case you type in the name of the test function.

pytest -k ‘plus’

Same as above one can filter what to run with:

pytest -k ‘name1 or name2’

pytest -k ‘name1 and not name2’ # This one too is redundant. Same as:
pytest -k ‘name1’

pytest -k ‘not name1’

2 Likes

Another handy decorator if you have tests that you dont want to run at the moment is:

@pytest.mark.skip

tests/test_b/test_area.py::test_ekvation PASSED                          [ 20%]
tests/test_b/test_area.py::test_another_ekvation xfail                   [ 40%]
tests/test_b/test_pi.py::test_omkrets PASSED                             [ 60%]
tests/test_b/test_pi.py::test_cirkel_yta PASSED                          [ 80%]
tests/test_b/test_pi.py::test_volym SKIPPED                              [100%]

================ 3 passed, 1 skipped, 1 xfailed in 0.04 seconds ================

You will now see that one test was skipped

One can also add a reason for skipping the test:

@pytest.mark.skip(reason=‘Have to do more research about this first’)

If you then run Pytest with the -v and -rs flag:

pytest -v -rs

There is a nice summary of all tests that was run and at the bottom you see the reason if you forgot why you put the skip mark on this test.

ests/test_b/test_area.py::test_ekvation PASSED                          [ 20%]
tests/test_b/test_area.py::test_another_ekvation xfail                   [ 40%]
tests/test_b/test_pi.py::test_omkrets PASSED                             [ 60%]
tests/test_b/test_pi.py::test_cirkel_yta PASSED                          [ 80%]
tests/test_b/test_pi.py::test_volym SKIPPED                              [100%]
=========================== short test summary info ============================
SKIP [1] tests/test_b/test_pi.py:12: Have to do more research about this first
1 Like

Some tips how to create a nice structure in the tests folder.

Pytest look into all files and folders from where you are as long as the name of the folder starts or ends with “test”.
The test function has to start with “test” to run.

To have a clear structure if your tests start to grow you can make different folders for different kind of tests.

src/
tests/
    __init__.py
    test_parsing/
    test_math_functions/
    test_website/

So when it is time for testing its easy to run just those tests that you want at that moment.
If it is about parsing. Just specify that folder.

pytest -v tests/test_parsing/

Then you got those tests only.

#----------------------------------------------------------

Planning to put up something about fixtures and conftest.py soon.
Not advanced at all.
It will be fixtures for novices.
I welcome other to fill in what you have learned about the subject.

2 Likes

And parametrize please as it’s Uber cool for repetition :slight_smile:

That is also on the To Do list.
Coming soon on a cinema near you. :smile:

Not really. Just have to write down something that more than just me understand.

1 Like

Conftest time :sunny:

I have still the same structure as previous post.
Now I have put in a conftest.py.

In this file you can put pytest.fixtures. Fixtures can be put in a test_something.py file to be used of the test functions in that file.
From conftest.py the fixtures can be used in all test files in the same folder and below.


src/
tests/
    conftest.py
    __init__.py
    test_cube/

I made a class to do tests on:

class CubicShape(object):
    def __init__(self, length, width, depth):
        self.length = length
        self.width = width
        self.depth = depth

    def cube_volume(self):
        return self.width * self.length * self.depth

    def square_or_rektangle(self):
        if self.length == self.width:
            return "Shape is square"
        else:
            return "Shape is rectangular"

And a conftest.py


import pytest
from src import cubic_class

a = cubic_class.CubicShape(10, 10, 10)
# Instantiation in the same way as if you have imported a python module
# filename.ClassName

@pytest.fixture
def cube_volume():
    return a.cube_volume()
    # returning the first method

@pytest.fixture
def sqr_or_rekt():
    return a.square_or_rectangular()
    # returning next method

And the test file:

import pytest

def test_cube(cube_volume):
    assert cube_volume == 1000
    # as you can see here the function from conftest is put in as a parameter.
    # and then asserted against expected output

def test_square_or_rectangular(sqr_or_rekt):
    assert sqr_or_rekt == "Shape is rectangular"
    # this test will fail because the shape is not rectangular.
    # not all sides anyway.
3 Likes

And now some parametrizing.

import pytest
from src import cubic_class
# This class is a little bit extended.
# I put it at the end of this post

# The parameter "test_input" is: 1 * 100, 2 * 200 ...
# The parameter "expected_output" is the result" of that calculation (100, 400....)
# Note there is quotation marks around the parameters in here!
#  In the test functions there is not.
# After the two parametersinside "[ () ]" brackets the calculation takes place.  
@pytest.mark.parametrize("test_input, expected_output",
        [
            (1*100, 100),
            (2*200, 400),
            (4*400, 1600)
         ])

def test_addition(test_input, expected_output):
    assert test_input == expected_output
    # and here you test if 1*100 equals 100, 2*200  equals 400... and so on.


@pytest.mark.parametrize('start, middle, end',
            [('a', 'b', 'c'),
            ('A', 'B', 'C')
            ]
        )
def test_abc(start, middle, end):
    assert start.lower() + middle.lower() + end.lower() == 'abc'

orden = 'abcdef'
@pytest.mark.parametrize('w_input, result', [
            (orden[0], 'a'),
            (orden[1], 'b'),
            (orden[2], 'c'),
            (orden[3], 'd'),
            (orden[4], 'e'),
            (orden[5], 'f')
            ])

def test_words(w_input, result):
    assert w_input == result


# Here I succeded to parametrize the methods in a class
# for reference, see the file below

a = cubic_class.CubicShape(10, 10, 10)
@pytest.mark.parametrize('volume, result',[      
        (a.cube_volume(), 1000),
        (a.square_or_rektangle(), 'Shape is square'),
        (a.one_surface(), 100),
        (a.total_cube_surface(), 600)])

def test_volym(volume, result):
    assert volume == result
# cubic_class.py

class CubicShape(object):
    def __init__(self, length, width, depth):
        self.length = length
        self.width = width
        self.depth = depth

    def cube_volume(self):
        return self.width * self.length * self.depth

    def square_or_rektangle(self):
        if self.length == self.width:
            return "Shape is square"
        else:
            return "Shape is rectangular"

    def one_surface(self):
        return self.length * self.width

    def total_cube_surface(self):
        return self.one_surface() * 6

This is my way to learn these things better.
I welcome comments, questions and suggestions on improvement.

3 Likes
A free service run by Zed A. Shaw for learncodethehardway.org.