Do Gödel Incompleteness theorems have any influence on the possibility of constructing / finding a "Theory of Everything" ?

We frequently see people arguing that Gödel Incompleteness theorems imply that no "Theory of Everything" could be found, because we would expect it to be complete, in the sense that it would adress (at least in principle) any question about the physical world, and consistent.
But the theorems are about mathematical theories and no existing physical theory is completely mathematical. I don't mean that their mathematical formalisms can't be made rigorous, what I mean is that they also have a conceptual background which is not included in the calculational scheme. They are not pure mathematics.
A related question that might shed some light in this issue is: there could be a complete correspondence between physical reality and mathematical objects? Notice, however, that even if this turns out to be true, the question is not settled down, since there is the possibility of our "Axiomatic Theory of Everything" not being expressive enough to formalize the natural numbers and in this case Gödel's hypothesis would not apply.
So, what do you think?