For a lot of the low-grade t206s I buy, the holder adds little to no value, and I find them generally clunky. I also keep my t206s in a binder, but I use one of the least efficient means possible. I use 4-pocket pages, with each raw card in its own penny sleeve and toploader in a slot on the page. As it happens, 4-pocket page pockets are big enough to accommodate a slabbed card as well.
Here's a fairly random page with 4 slabbed cards tucked away. (Sorry for the blur - my scanner hates scanning cards in slabs...)
Short form: I crack the card out when the holder doesn't add value and/or gets in the way of my ability to appreciate the card.