The paper deals with the usefulness of model modification statistics, such as the modification index and the Wald statistic, in situations where equivalent models are involved. Two cases are considered. In the first case, a baseline model M_0 is fitted, and it is known that there are two equivalent models M_1 and M_2, both having only one parameter more than the baseline model. It is proven and
... [Show full abstract] illustrated that, except for the modification index, most modification statistics have a different value for each of the parameters that either lead to M_1 or M_2. In the second case, both equivalent models M_1 and M_2 are fitted themselves to the sample data. The Wald statistic, for example, is shown not to be invariant in this situation. The results indicate that in a process of model modification a number of well-known statistics should be interpreted very cautiously.