Now I want to play Persona even more. I feel like I'm missing something big
Ya I know the feeling. A few weeks ago I had no clue what persona was then I tried out P4G now i'm in love with the series. Bought P3F, P4G, and am currently trying out P4A. If you like RPGs then I think you should deffinitly give it a try.
You can get P3 FES for the PS3 on PSN, the second half of Persona 2 on PSN, and P4 Arena for the PS3 also.
Persona 3 FES is on PSN (But you could do Persona 3 Portable for vita but lacks as much content)
I know Perosna one and Persona 2 are on PSN.
And then there is Persona 4 arena which is the sequel to persona 4 with persona 3 characters in it on Ps3.