Hi Raz,
This looks like weird. Typically you should be able to delete the application or registration anytime, but you will not be able to delete the connections if it is used by applications.
I would suggest you to create a new application configuration. You could try deleting the old app config later.
Regards, Midhun
SAP Technology RIG