I totally understand about the cost - I learned to dive when I was in college. Assuming that you already have mask/fins/snorkel, the next thing you should shop for is a good wetsuit. You’ll get a better fit that you will with a rental, always know how much weight to use and (best of all) you’ll be the only one who’s peed in it. Since you live up north, think about getting a core warmer and a beanie/hood when you need added warmth. If you want gloves for protection (not insulation), you can get fabric work gloves at the hardware store for about $5.
I waited to buy my octo/BC and computer until I began to dive more often and renting began to cost too much.