These are the sort of questions I find I can't answer as asked.
Different cards, different marks, it all makes the issue more complex than the question requires.
A light pencil mark that originally left nearly no physical trace - I don't think removal is a bad thing.
IF
It's done in a way that doesn't damage the card. Some erasers are rougher on the paper surface than others.
A heavy pencil mark might be ok to remove, if for instance it's on the back and erasing it doesn't damage the printed portion. There will still be indentations from the writing, so I wouldn't expect the technical grade to change.
But I have a few T206s with erasures on the front. That's pretty bad, and I would rather they'd left it alone.
Spilled ink or stuff like that- Maybe, it's usually hard for an amateur to remove without doing more damage. Removed, lightened it should still be considered marked.
Some marks are more interesting. Like owners stamps and stuff like that. I would consider removing that sort of mark to be bad, as to me the stamp might be better than the card.
never an "it depends" as an answer........
Steve B