TEST-NOTES

UnitTests
---------

Portage has some tests that use the unittest framework that ships with python (2.3-2.4ish)
Tests have a specific naming convention.

in lib/portage/tests/ there is a runTest script that invokes lib/portage/tests/__init__.py

This init looks at a hardcoded list of test dirs to search for tests.
If you add a new dir and don't see your new tests, make sure that the dir is in this list.

On the subject of adding more directories; the layout is basically 1 directory per portage
file at this point (we have few files, and even fewer large files).  Inside of the dir
you should have files of the form test_${function}.py.

So if I was to write a vercmp test, and vercmp is in portage_versions.

lib/portage/tests/portage_versions/test_vercmp.py

would be the filename.

The __init__.py file now does recursive tests, but you need to tell it so.  For example, if
you had cache tests the dir format would be something like...

lib/portage/tests/cache/flat_hash/test_foo.py

and you would put "cache/flat_hash" into the testDirs variable in __init__.py.


Skipping
--------

Please use the portage.tests.* classes as they support throwing a SkipException for
tests that are known to fail.  Normally one uses testing to do Test Driven Development
(TDD); however we do not do that here.  Therefore there are times when legitimate tests
exist but fail due to code in trunk.  We would still like the suite to pass in some instances
because the suite is built around two things, testing functionality in the current code as
well as poking holes in the current code (isvalidatom is an example).  So sometimes we desire
a test to point out that "this needs fixing" but it doesn't affect portage's overall
functionality.  You should raise portage.tests.SkipException in that case.

Playground metadata cache
-------------------------

Every ResolverPlayground runs egencache, which runs a depend phase for each
of its ebuilds.  To avoid repeating that work, the files that egencache
generates are cached, keyed by what they are generated from, and restored
into later playgrounds that write the same ebuilds.  A playground whose
files are all cached skips egencache altogether.

The key covers the ebuild and, for a Manifest, the package directory and the
distfiles.  It does not cover the profile or the rest of the playground's
configuration, so this assumes that the metadata of an ebuild does not
depend on them.  A test ebuild that expands a variable into its metadata
would break that assumption and has to be kept out of the cache.

A playground that defines eclasses, or that configures the auxdb module,
shares nothing: a depcachedir entry names the eclass directory that it was
generated from, and the auxdb module decides which entries egencache writes
at all.  A test that exercises metadata generation itself should pass
share_metadata=False, so that it keeps generating the metadata that it is
about rather than being handed a copy.

Setting PORTAGE_TEST_VERIFY_METADATA_CACHE checks the assumption: nothing is
restored, egencache generates every file, and each generated file is compared
against the entry that the cache would have restored in its place.  A mismatch
raises an AssertionError naming the file.  The suite then runs as slowly as it
does with no cache, and it runs egencache as often, so this is what CI uses to
keep exercising metadata generation.

	PORTAGE_TEST_VERIFY_METADATA_CACHE=1 pytest -n 20

The cache lives below pytest's basetemp, so it is shared by the xdist
workers of a run and discarded with the rest of the run's temporary files.


emerge
------

The emerge namespace currently has 0 tests