This letter presents hyperspectral image segmentation based on the phase-correlation measure and updating the segments using a post processing operation based on adaptive thresholding. Spectral signature of each pixel is subsampled to gain robustness against noise and spatial variability, and phase correlation is performed to measure spectral similarity. Similar and dissimilar pixels are decided according to the peak value of the phase correlation result to determine pixels that fall into the same segments. An adaptive threshold value that is determined for each segment considering in-segment similarity distribution is used to update the segment. Segmentation accuracy is increased compared to phase correlation based segmentation.