This review paper summarizes the various brain modules that are involved in speech and language communication in addition to a left-dominant “core” language network that, for the present purpose, has been restricted to elementary formal-linguistic and more or less disembodied functions such as abstract phonology, syntax, and very basic lexical functions. This left-dominant perisylvian language network comprises parts of inferior frontal gyrus, premotor cortex, and upper temporal lobe, and a temporoparietal interface. After introducing this network, first, the various roles of neighboring and functionally connected brain regions are discussed. As a second approach, entire additional networks were considered rather than single regions, mainly motivated by resting-state studies indicating more or less stable connectivity patterns within these networks. Thirdly, some examples are provided for language tasks with functional demands exceeding the operating domain of the core language network. The rationale behind this approach is to present some outline of how the brain produces and perceives language, accounting, first, for a bulk of clinical studies showing typical forms of aphasia in case of left-hemispheric lesions in the core language network and second, for wide-spread activation patterns beyond this network in various experimental studies with language tasks. Roughly, the brain resources that complement the core language system in a task-specific way can be described as a number of brain structures and networks that are related to (1) motor representations, (2) sensory-related representations, (3) non-verbal memory structures, (4) affective/emotional processing, (5) social cognition and theory of mind, (6) meaning in context, and (7) cognitive control. After taking into account all these aspects, first, it seems clear that natural language communication cannot really work without additional systems. Second, it also becomes evident that during language acquisition the core language network has to be built up from outside, that is, from various neuronal activations that are related to sensory input, motor imitation, nursing, pre-linguistic sound communication, and pre-linguistic pragmatics. Furthermore, it might be worth considering that also in cases of aphasia the language network might be restored by being trained from outside.