In 1931 Wittgenstein wrote: ‘the limit of language manifests itself in the impossibility of describing the fact that corresponds to (is the translation of) a sentence without simply repeating the sentence’. Here, Wittgenstein claims, ‘we are involved … with the Kantian solution of the problem of philosophy’. This paper shows how this remark fits with Wittgenstein's early account of the substance of the world, his account of logic, and ultimately his view of philosophy. By contrast to the currently influential resolute reading of the Tractatus, the paper argues that the early Wittgenstein did not aim at destroying the idea of a limit of language, but that the notion lies at the very heart of Wittgenstein's early view. In doing so, the paper employs and defends the Kantian interpretation of Wittgenstein's early philosophy.