What is Performance tuning? What is its importance? At what stage performance tuning is required?

In very layman words, Performance tuning means changing the system configurations, change in some settings, change in the application etc..., to improve the performance of the application.
But when this needs to be done? At what stage performance tuning is required? Before preparing the test plan? Or after writing the test plan or after running the tests (Load test etc...)? Many people would probably say they need to do performance tuning before or after the test plan is prepared. But in reality performance tuning is required after running a load test, benchmark test, stress test etc… It is because only after performing few tests you will compare the response times and other metrics and if you think any of these metrics are abnormal, then you can suggest for some changes in system configuration, settings etc… so this is the exact time that you need performance tuning.