Density-based histogram of singular values for √ n ˆ L for the words sequential data in blue bars and the theoretical predictions associated with the improvement clustering with K = 200 as the red line. Not visible in this figure is that both empirical distributions have long tails. Still 9% of the singular values ofˆNofˆ ofˆN / √ n exceed 10 and 1% of the singular values of √ n ˆ L exceed 30.

Density-based histogram of singular values for √ n ˆ L for the words sequential data in blue bars and the theoretical predictions associated with the improvement clustering with K = 200 as the red line. Not visible in this figure is that both empirical distributions have long tails. Still 9% of the singular values ofˆNofˆ ofˆN / √ n exceed 10 and 1% of the singular values of √ n ˆ L exceed 30.

Source publication
Preprint
Full-text available
Motivated by theoretical advancements in dimensionality reduction techniques we use a recent model, called Block Markov Chains, to conduct a practical study of clustering in real-world sequential data. Clustering algorithms for Block Markov Chains possess theoretical optimality guarantees and can be deployed in sparse data regimes. Despite these fa...

Contexts in source publication

Context 1
... structure in the spectral noise is however not as one would expect in a BMC. Consider Figure 5 for a comparison of the empirical singular value distribution with the theoretical predictions. Observe that there is a good match forˆNforˆ forˆN but a discrepancy forˆLforˆ forˆL. ...
Context 2
... empirical LaplacianˆLLaplacianˆ LaplacianˆL removes this dominant effect after which it may be observed that the empirical distribution has a heavy tail which is not present in the BMC-based prediction. In Supplement 14.4 we demonstrate by a numerical example that the discrepancy which is observed in Figure 5 agrees precisely with the type of discrepancy which is observed for a heavy-tailed perturbation of the BMC. The fact that the entries of the matricesˆNmatricesˆ matricesˆN andˆLandˆ andˆL are heavy-tailed may also be verified by direct inspection. ...