In 1979-81 I was aware of only two comic shops in Toronto (Comics Unlimited on Keewatin Avenue just east of Yonge Street and Queen's Comics & Collectibles at the Beach) and Dreamland Comics on James Street North in Hamilton that carried cards. I later learned that there was also a stamp/coin shop in Hamilton just north of the comic shop that dealt in cards so perhaps there might have been a couple of other such shops in Toronto of which I wasn't aware.
Dedicated card shows didn't make an appearance in Toronto (and perhaps anywhere in Canada) until about 1986 and they were then really low budget affairs held in less than first class halls/meeting rooms. I remember being excited to learn that another (my second) card show was going to take place in far away Niagara Falls in 1987(?). Shortly thereafter newspapers and other media sources started running stories about the prices fetched by the T206 Honus Wagner and 1952 Topps Mickey Mantle cards and card collecting absolutely exploded in popularity with the general public.
Those days were certainly different, albeit not necessarily better. The internet has certainly done wonders to put collectibles and collectors together.