Regression Testing - would be done after a system has been tested once, the tester would go back through the system to make sure bugs have been fixed and the intended functionality has been maintained.
Shaky is given us a very clear picture, but still to put you in more ease i am coming with this discussion.
"performance Testing" - The ability of the system to withstand and cope with no. of hits/throughput.
i.e how system is able to with stand in its performance expectations like how it works/responds to the users at 100 users & its response at 1000 users etc., and where exactly the performance start degrading.
"Regression Testing" - Normally will go for this testing in case of any minor/major changes in the application/product & to check Bug fixings.
this is why bcause when ever Bug fixes we will again test the total application whether that particular 'fix' has caused any other new bugs or not is the first one.
And to test the application versions which are modified/enhanced according to User requirements.
Hope i am not confused.