Colorectal cancer is the second most common malignancy diagnosed globally. Critical gaps exist in diagnostic and surveillance imaging modalities for colorectal neoplasia. Although prior studies have demonstrated the capability of photoacoustic imaging techniques to differentiate normal from neoplastic tissue in the gastrointestinal tract, evaluation of deep tissue with a fast speed and a large field of view remains limited. To investigate the ability of photoacoustic technology to image deeper tissue, we conducted a pilot study using a real-time co-registered photoacoustic tomography (PAT) and ultrasound (US) system. A total of 23 ex vivo human colorectal tissue samples were imaged immediately after surgical resection. Co-registered photoacoustic images of malignancies showed significantly increased PAT signal compared to normal regions of the same sample. The quantitative relative total hemoglobin (rHbT) concentration computed from four optical wavelengths, the spectral features, such as the mean spectral slope, and 0.5-MHz intercept extracted from PAT and US spectral data, and image features, such as the first- and second-order statistics along with the standard deviation of the mean radon transform of PAT images, have shown statistical significance between untreated colorectal tumors and the normal tissue. Using either a logistic regression model or a support vector machine, the best set of parameters of rHbT and PAT intercept has achieved area-under-the-curve (AUC) values of 0.97 and 0.95 for both training and testing data sets, respectively, for prediction of histologically confirmed invasive carcinoma.