Overview

• Training from Scratch
  • Training Usage
  • Results evaluation
• Programming Style Considerations
  • Abstraction
  • Weak Spots/TODO
    • Configurations
    • Tokenization
    • Further Refactoring

Script Documentation

• train.py
• entropy_cache.py
• cached_experiments.py
• Experiment
  • Experiment.map()
  • Experiment.on_finish()
  • Experiment.plot()
  • Experiment.requirements()
• PerFieldEntropyExperiment
  • PerFieldEntropyExperiment.map()
  • PerFieldEntropyExperiment.on_finish()
  • PerFieldEntropyExperiment.plot()
  • PerFieldEntropyExperiment.requirements()
• gen.py
• GenerationExperiment
  • GenerationExperiment.eval_generation()
  • GenerationExperiment.examples_seen()
  • GenerationExperiment.min_size()
  • GenerationExperiment.on_finish()
  • GenerationExperiment.plot()
  • GenerationExperiment.stopping_criteria()
  • GenerationExperiment.window_size()
• NextActionExperiment
  • NextActionExperiment.eval_generation()
  • NextActionExperiment.examples_seen()
  • NextActionExperiment.on_finish()
  • NextActionExperiment.plot()
  • NextActionExperiment.stopping_criteria()
  • NextActionExperiment.window_size()
• ScoringExperiment
  • ScoringExperiment.eval_generation()
  • ScoringExperiment.examples_seen()
  • ScoringExperiment.on_finish()
  • ScoringExperiment.plot()
  • ScoringExperiment.stopping_criteria()
  • ScoringExperiment.window_size()
• entropy.py

Library Documentation

• EHRAuditDataset
  • EHRAuditDataset.load()
  • EHRAuditDataset.load_from_cache()
  • EHRAuditDataset.load_from_log()
• timestamp_space_calculation()
• EHRAuditGPT2
  • EHRAuditGPT2.forward()
• EHRAuditLlama
  • EHRAuditLlama.forward()
• EHRAuditRWKV
  • EHRAuditRWKV.forward()
• EHRAuditTransformerXL
  • EHRAuditTransformerXL.forward()
• TabularLoss
  • TabularLoss.forward()
• EHRAuditDataModule
  • EHRAuditDataModule.prepare_data()
  • EHRAuditDataModule.setup()
  • EHRAuditDataModule.test_dataloader()
  • EHRAuditDataModule.train_dataloader()
  • EHRAuditDataModule.val_dataloader()
• EHRAuditPretraining
  • EHRAuditPretraining.configure_optimizers()
  • EHRAuditPretraining.forward()
  • EHRAuditPretraining.predict_step()
  • EHRAuditPretraining.test_step()
  • EHRAuditPretraining.training_step()
  • EHRAuditPretraining.validation_step()
• collate_fn()
• worker_fn()
• EHRAuditLogitsProcessor
• EHRAuditTokenizer
  • EHRAuditTokenizer.batch_decode()
  • EHRAuditTokenizer.decode()
  • EHRAuditTokenizer.encode()
• EHRVocab
  • EHRVocab.field_names()
  • EHRVocab.field_to_token()
  • EHRVocab.global_to_token()
  • EHRVocab.globals_to_locals()
  • EHRVocab.globals_to_locals_torch()
  • EHRVocab.save()

Hugging Face Documentation

• gen_random_df()
• perform_action()
• publish_hf.py