Removed the attachment. I’ve made updates to the code, mainly learning to spell “Delimiter”. (Okay, actually there were more changes, including comment-based help, etc.)
Rather than keeping an attachment in this thread up-to-date, here’s a link to the file over in my github repository: Tokens.ps1
I’d still love to hear people’s input. As is, the script meets the requirements I pictured when I started writing it, but there may be other ideas on how to customize the behavior of a simple tokenizer that I didn’t consider.