Could someone help me decide the ideal no.of clusters from the pseudo t squared graph in SAS?

Question

In SAS, I ran ACECLUS and Cluster procedure to analyze a data set of 16 morphometric characters of a bird.

Ryan Simmons · Answer

It would be helpful if you provided your SAS code for PROC CLUSTER. For all we know, you made a mistake. See, for example, the examples in the SAS documentation for PROC CLUSTER.
Assuming your code is correct and there aren&#x27;t any problems with the data itself, these results suggest to me that there isn&#x27;t any clustering in your data. Due to the peak is psuedo t squared, you should graphically investigate the data using 5 (see below) clusters, but my intuition is that you won&#x27;t see clear separation. Let&#x27;s look at each statistic individually:
CCC is the cubic clustering criterion; the idea behind it is to compare the R squared you get with a specific number of clusters versus the R squared you would get by clustering a uniformly distributed set of points. That is, you interpret it similarly as you would R squared. You are getting STRICTLY negative values (and, in fact, they are decreasing with additional number of clusters before increasing again; I would interpret that increase as overfitting). This means that the model you are fitting to the data with X number of clusters fits worse than uniformly distributed points. This is evidence of a lack of clustering (or problems with the data).&#xA0;
In addition, according to SAS Technical Report A-108 (CCC was developed by SAS),&#xA0;&#x22;If all values of the CCC are negative and decreasing for two or more clusters, the distribution is probably unimodal or long-tailed.&#x22; He goes on to say that very negative values may be due to outliers. I would check your data and your code to make sure, but this heavily implies that the data is unimodal, and thus there is no clustering.
Pseudo F is the ratio of between-cluster variance to within-cluster variance. That is, it provides a measure of how separated the clusters are. Your plot shows that this ratio is essentially unchanged regardless of the number of clusters your define. This implies a lack of clustering behavior.&#xA0;
Pseudo T squared is an index that quantifies the difference in the ratio of between-cluster variance to within-cluster variance when clusters are merged at a given step (put another way, psuedo T squared is working &#x22;backwards&#x22;, from right to left on the plot). If there is a distinct jump in psuedo T squared with X number of clusters, then X+1 represents the optimal number of clusters. In your case, you see a jump in psuedo T squared with 4 clusters, so you should graphically investigate the data with 5 clusters to look for clear separation. However, given the CCC and psuedo F values, I would guess you don&#x27;t have any clustering and this psuedo T squared is just an artifact due to an outlier.&#xA0;

Zohara Rafi · Answer

Thank you very much Ryan for your descriptive explanation. These data are of two sister species and it SHOULD cluster. I&#x27;ve attached my SAS code.

Ryan Simmons · Answer

I don&#x27;t see any obvious errors. So, two notes:
1) Plot the data with scatterplots using the PROC CLUSTER output. See the example in the linked SAS help page. What do the data look like? Can you see distinct clusters in the data?&#xA0;
2) Again, you need to do some exploratory data analysis. I notice you are using the Ward minimum-variance method for determining cluster distance. Note that this method is extremely sensitive to outliers; check your data for skew and outliers that may be driving the results.
http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_cluster_sect027.htm

Zohara Rafi · Answer

Thank you for your continued support Ryan.... I plotted.... there seems to be clusters. What do you think?
In the aceclus procedure I have invoked p=0.03 to get rid of outliers.
Could you please tel me the codes for changing the 4 colors of the clusters into 4 shapes please?
the code used was
LEGEND1 FRAME CFRAME=LIGR CBORDER=BLACK
POSITION=CENTER VALUE=(JUSTIFY=CENTER);
AXIS1 LABEL=(ANGLE=90 ROTATE=0) MINOR=NONE ORDER=(-10 TO 20 BY 5);
AXIS2 MINOR=NONE ORDER=(-10 TO 20 BY 5);
PROC GPLOT DATA=ZOHARA.NEW;
PLOT CAN2*CAN1=CLUSTER/FRAME CFRAME=LIGR
LEGEND=LEGEND1 VAXIS=AXIS1 HAXIS=AXIS2;
RUN;

Could someone help me decide the ideal no.of clusters from the pseudo t squared graph in SAS?

Most recent answer

All Answers (4)

Similar questions and discussions

Related Publications