Tuesday, February 4, 2014

Help required: Python importing (2.7 but could be convinced to change)

I am writing an application, which also incorporates the ability for users to run -- and test -- their own custom sub-project. These sub-projects are called "experiments". The structure on disk is as follows:

/tld
   -- application/
          __init__.py
          foo.py
          bar.py
          tests/
              test1.py
              test2.py
   -- data/
          experiments/
                first_experiment/
                     incoming_data/
                          foo.csv
                          bar.csv
                      scripts/
                          get_foo.py
                          make_bar.py
                      tests/
                          test_get_foo.py
                          test_get_bar.py
At the moment, test_get_foo.py includes sys.path.append('../scripts') in order to find the scripts which are being tested.

If I run py.test (or python -m unittest) from the data/experiments/tests/ directory, that's fine. If I run the application tests from the tld, that is fine. But I can't run the experiments tests from the tld, using e.g. py.test or python -m unittest discover. I'm in module importing hell, because '../scripts' is evaluated relative to the executing directory (tld) not relative to the test file directory.

What is the right thing to do here?

                   

2 comments:

Tennessee Leeuwenburg said...

So I've done some more reading and partially addressed this. By adding an __init__.py to each of the directories, I can now use a fully-qualified import inside the test, like

from data.experiments.sample_experiments.scripts import foo.py

L. Amber Wilcox-O'Hearn said...

You could try setting the PYTHONPATH before the python call:

PYTHONPATH=/tld/data/experiments/first_experiment/tests python -m unittest test_get_foo