This is my issue as well but I resolved it by changing my email for Medium so at least now my personal inbox isn't flooded. I was missing seeing important emails and it bugged me no end. As an editor at another pub that has quite high traffic, I get every notification of new drafts which almost tipped me into pulling my hair out!

I still pick and choose who I read as there's just too many stories that come in from the dozen or so people I do subscribe to. But you are right that the first thing writers will think if you unsubscribe is that it's something personal about them.

Good luck with getting back in control!