In this paper we present a novel image analysis methodology for automatically distinguishing low and high grades of breast cancer from digitized histopathology. A set of over 3,400 image features, including textural and nuclear architecture based features, are extracted from a database of 48 breast biopsy tissue studies (30 cancerous and 18 benign images). Spectral clustering is used to reduce the dimensionality of the feature set. A support vector machine (SVM) classifier is used (1) to distinguish between cancerous and non-cancerous images, and (2) to distinguish between images containing low and high grades of cancer. Classification is repeated using different subsets of features to compare their performance. The system achieves a 95.8% accuracy in distinguishing cancer from non-cancer using texture-based characteristics (Gabor filter features), and 93.3% accuracy in distinguishing high from low grades of cancer using architectural features. In addition, we investigate the underlying manifold structure on which the different grades of breast cancer lie as revealed through spectral clustering. The manifold shows a smooth spatial transition from low to high grade breast cancer.