LPTHW_EX52: Testing app.py

Hey.

I’m trying to figure out the testing of app.py.

At the moment this is the way to see if another room has been entered:

data = {'action': 'tell a joke'}
    rv = web.post('/game', follow_redirects=True, data=data)
    assert rv.status_code == 200
    assert b"Laser Weapon Armory" in rv.data

(Using Pytest, so different testing syntax!)

Now this works just fine as the heading of the new room is found in rv.data.

First of all:

Do you think it makes sense to test this also on app.py? I know I am basically already testing if entering a new room works in the test_planisphere.py, but I was thinking that while maybe the planisphere could work just fine, but there could be some issue that fails to display the changes in the html… Does it make sense to test it also here or do you think it is redundant?

Second I would like to be able to find a solution where I don’t check wether the heading is somewhere in rv.data but instead directly check the room_name variable. However I am having trouble accessing it. I tried around using g. and get(). But I can’t seem to get it to return the room_name variable.

for instance I tried:

print(">>>> web.get(room_name)", web.get('room_name'))

I guess this shoots into nowhere because I only get:

>>>> client.get(room_name) <Response streamed [404 NOT FOUND]>

Is it possible to get the room_name variable?

The final reason why I want to access this variable is so that I can change it and test moving from one room (room_a) to another that is only accessible from this room_a. I was thinking that if I can change the room_name variable for the test I could just put in the room_a in there and test the options that are available for room_a rather then making my client move step by step through every room that is required to be passed until it reaches room_a. That would seem very inconvenient and inefficient. Any hints on how to do this?

I found the “Faking Resources and Context” section in the testing documentation for flask:

http://flask.pocoo.org/docs/1.0/testing/

and I think the appcontext_pushed function might be suitable for what I am trying to achieve! However I am getting a bit lost on how I should implement this method in my specific context and I think the first obstacle for me is what I mentioned above that I don’t even know yet how to access the room_name variable.
Any tips on how to implement this function in my context would be very much appreciated <3 !

Looking forward to hear back!

Hugs,

Leon

Hey, so first up, I actually prefer to do the inverse:

I tend to write all my web applications so that I just test everything through the UI, and then only add additional tests when I can’t get at something through the UI. Usually though, if I can’t get to it with the UI then it’s probably not something I need. The exception would be error handling code, which I test by going through the UI in tests, but using unittest.mock to cause the error I want.

Now, for getting at the room names: If you’re storing them in a variable in one of your modules, then you can just import that module into your test and then use it to drive the UI. However, if you’re thinking you’re going to do this:

  1. Pretend to be the browser and click on a link.
  2. In the middle of your handler, somehow dip inside of it and get the room_name local variable.
  3. Then do some kind of testing from there.

Then you’re not going to be able to do that, and it wouldn’t be smart. One day you’ll make a minor change to how your handler works and all your tests will need to change too. The advantage of using the UI to test the application is that you only need to change the tests when you change how the UI works.

What that means is, you should abandon this idea of getting at the guts of your handler (if I’m understanding what room_name is for you), and instead think about it as you work the UI to make the code go, and that’s it.

Finally, the key to avoiding repetition in your tests is to use loops and make the computer do it for you. If you have a complicated path that you need to follow then that’s just a list of actions. Simply write out your actions in a list, for loop through them, and test that you get a response.

Another more advanced trick is to randomize it. You create a list of “possible” things that you can do, and then your test just thrashes through all of them until it hits a bug. This is called fuzzing and it doesn’t make sure your intended code works as intended, but it does catch things you didn’t think about.

So, what you do is make a list of the correct path for someone, then write a fuzz test that just thrashes these possible steps at random for a few loops to see if you get a 500 or 404 error somewhere.

Now, if you can show me a tiny bit of code that explains what you’re doing with room_name I can continue with more, but you’re better off just thrashing the UI.

2 Likes

Hey!

Thank you very much for your elaborate answer! I see the point that you are making about testing the UI… I will try to implement using loops and also the fuzz tests sound like a good idea!

Cheers!

So now I do have one more question and that is that I keep running into trouble with the status of my sessions. I managed to restart the session within one module. For instance app.py in test_app.py by creating a new fixture for every test.

@pytest.fixture()
def new_client():

    app.config['TESTING'] = True
    client = app.test_client()

    yield client

Not sure if this is a good way to do it, but it does restart the session for app.py
However I do still run into problems because I run tests in my first test function that change values in another ‘gamestate.py’ module (Character Health or Inventory, etc…) and when I restart the session in my test_app.py it doesn’t reset the changes in the ‘gamestate.py’ module.

So all following tests that check gamestate.py and expect the values of a new, reseted session fail…

I read the ‘Testing Flask’ documentation but I couldn’t find a solution from there…

grateful for any input :slight_smile:

Hugs,

Leon

If you want to be able to tell your app “I’m running you in a test right now so do some extra stuff”, it’s better to use environment variables. In python do this:

import os

os.environ['TESTING'] = True

You can also do the same thing from the command line in bash (Linux or OSX):

TESTING=1 nosetests

Now, everywhere that you need to do something a little different for testing just do this:

if 'TESTING' in os.environ:
     # blah blah whatever hacks you need

An alternative is to assume you’re always testing, and that running in production is the exception. In that case, don’t set anything in your code but just do this:

if 'PROD' not in os.environ:
    # blah blah your testing hacks

Then you just do your tests like normal, knowing those area always there. When it comes time to run in production just run it with this:

PROD=1 ./bin/app.py

And it’ll get picked up because PROD=1 will make your if 'PROD' not in conditions be False so none of the testing hacks will be run.