I think I found the problem.
I use MS Word for part of my process and I do a lot of copy/paste and it seems ctrl-x from ms word might be adding the discretionary marks being interpreted as permanent for some reason.
I found out it was happening in my gmbinder as well which led me down a rabbit hole.