Personally I don't use a penny sleeve before placing the card in a semi-rigid top loader. However, let's make sure we're talking about the same kind of top loader.
I use these Ultra Pro that come 200 per box:
I do not use these top loaders that come 25 per pack:
I've been using the first ones for well over 20 years and never had a problem (they were another company before Ultra Pro). I tried using a penny sleeve with these, but the penny sleeve wants to bind up as I'm trying to slide the card in, so I just don't use one.