I have no interest in marginal HOFers either.
My tweaks on this form of collecting:
1. I collect certain players' cards and card-like ephemera, and for those guys I limit it to career-contemporary items.
2. I am simply not interested in the glut of modern cards issued after players retired, with the exception of the 1970s Negro League issues that often represented the player's first trading card.
2. For many players I try to get something that isn't necessarily their 'best' item or most valuable or rookie item, but that speaks to me the most. Like this, my favorite Ted Williams card:
Or this 1953 St. Louis Browns Satchel Paige signed postcard: