podcast-sponsor-remove/split_dataset.py, branch master Attempt at identify sponsored segments in audio transcripts and removing them. try finetuning encoder-decoder model like t5gemma 2025-08-24T04:31:29+00:00 vin git@vineetk.net 2025-08-24T04:31:29+00:00 0f5fcee95648d706b792affee0658e2e652fd904 Unfortunately once it finishes the finetuning process, it doesn't actually output anything. With decoder-only models like small gemma3, it also doesn't output anything meaningful. I'm tempted to just classify each token as an ad or not, finetune an encoder like some long-context BERT, and then a post-processing script to convert back to tokens and their timestamps. May end up being more performant and efficient too... 
Unfortunately once it finishes the finetuning process, it doesn't
actually output anything. With decoder-only models like small gemma3, it
also doesn't output anything meaningful. I'm tempted to just classify
each token as an ad or not, finetune an encoder like some long-context
BERT, and then a post-processing script to convert back to tokens and
their timestamps. May end up being more performant and efficient too...
add helper script to split dataset.jsonl 2025-08-23T17:35:32+00:00 vin git@vineetk.net 2025-08-23T17:35:32+00:00 4ec10c33159f435963b73da5a9bd12f25f65cda9