Not even sure if 20" rims were invented when the E53 was developed? They were definately a "thing" when the E70 came out. So the suspension set-up on the E70 would definately have been designed with that size of wheel in mind, so shouldn't be any major issues.
Poor ride and handling is usually a by product of incorrectly sized rims and tyres being fitted to the vehicle in my experience. If you have wheels that were a factory option fitted with the correct tyres then that removes that possibility, which is a good start.
The E70s do seem to have a liking for going through tyres, especially rears, fairly quickly. I think this is in part down to the rear geometry causing the inner shoulders to wear, and the fact that it is easy to drive them like a sporty car as they go so well - with the result that things do wear out a bit quicker.