Gleason Grading in Prostate Cancer Images: An Effective Segmentation Method and an Optimized Dense Convolutional Neural Network

Abstract

To qualitatively characterize the various tumor histology observed in the prostate, recognized as the most prevalent and the second most lethal type of cancer in men globally, pathologists employ a range of screening procedures. The GG (Gleason-grade) classification of PCs (prostate cancers), which is based on photographs of the illness acquired via transrectal ultrasound imaging, is a significant instrument that is utilized in risk assessment as well as in the process of planning for patients. Subsequently, cancer-affected areas are discerned using Compactness Fuzzy C-means (CFCM). This method incorporates an adaptive processing approach rooted in the Least Mean Square (LMS) technique to determine the clip limit for CLAHE (Contrast Limited Adaptive Histogram Equalization) during the denoising and segmentation processes. Classification of PCs based on GG utilizing histological images is crucial for risk assessment and therapy planning, and an optimal deep model finally clas-sified the segmented images. The Dense Convolutional Deep Neural Network (DCDNN) architecture is utilized for multi-task prediction that uses the Modified Dunnock Search algorithm (MDSA) for optimal hyperparameter tuning of the CDCNN model, improving classification performance. This model has achieved the highest possible accuracy on both epithelial cell recognition and Gleason grading at the same time.