Big Text, i.e., large repositories of textual data, is a part of Big Data. In total, 80–85 % of Big Text comes in unstructured form, with significant contribution from social media. In this position paper, we discuss Big Text advantages and challenges in respect to text classification. We propose a new approach to performance evaluation of classification algorithms when they applied to Big Text, ... [Show full abstract] namely, using corpora comparison in the result evaluation. We also discuss a significant increase in texts with comprehensive information and challenges Big Text methods face in analysis of such texts.