Only if its an over the top fake voice like those last two mentioned. The first two, I dont think would ruin a film, but I can understand why people would be upset by the character having an accent that does not pertain to the nationality (a Canadian character with an Aussie accent or an American actor with a Welsh accent would be somewhat annoying and go against the characters background).
That said, if they decided a character like Captain America or Thor, had voices that were neither powerful or booming, would not upset me at all.
The point you made was that you believed Rhodey should have a deeper voice is merely personal opinion, and somewhat cliched. There is absolutely nothing in the comic books that does not suggest that Rhodey does not sound like Howard. For all we know, that could be exactly the kind of voice the people at Marvel think he should have.
Theres a big difference between voice tonality, and an over the top voice/accent like some people use.
At the end of the day, there are people out there who probably think that Downey sounds nothing like Stark should. Or that Bridges sounds nothing like they imagined Stane. Point is, not one single "voice" in the Iron Man movie did anything to spoil the film.