entropy_cache – Cache entropy values calculated from trained language models

entropy_cache.py

usage: entropy_cache.py [-h] [--model MODEL] [--val] [--reset_cache] [--debug]
                        [--verify]

-h, --help

show this help message and exit

--model <model>

Model to use for pretraining.

--val

Run with the validation dataset instead of the test.

--reset_cache

Whether to reset the cache before analysis.

--debug

Whether to run with single thread.

--verify

Whether to verify the entropy values line up with the dataset.