Controlling the growth of material damage is an important engineering task with plenty of real world applications. In this paper we approach this topic from the mathematical point of view by investigating an optimal boundary control problem for a damage phase-field model for viscoelastic media. We consider non-homogeneous Neumann data for the displacement field which describe external boundary forces and act as control variable. The underlying hyberbolic-parabolic PDE system for the state variables exhibit highly nonlinear terms which emerge in context with damage processes. The cost functional is of tracking type, and constraints for the control variable are prescribed. Based on recent results from [M. H. Farshbaf-Shaker, C. Heinemann: A phase field approach for optimal boundary control of damage processes in two-dimensional viscoelastic media. Math. Models Methods Appl. Sci. 25 (2015), 2749--2793], where global-in-time well-posedness of strong solutions to the lower level problem and existence of optimal controls of the upper level problem have been established, we show in this contribution differentiability of the control-to-state mapping, well-posedness of the linearization and existence of solutions of the adjoint state system. Due to the highly nonlinear nature of the state system which has by our knowledge not been considered for optimal control problems in the literature, we present a very weak formulation and estimation techniques of the associated adjoint system. For mathematical reasons the analysis is restricted here to the two-dimensional case. We conclude our results with first-order necessary optimality conditions in terms of a variational inequality together with PDEs for the state and adjoint state system.