Constructing descriptors for key points in image matching is a crucial task in computer vision, visual tracking, and pattern recognition. Highly rated descriptors such as scale-invariant feature transform (SIFT) and speeded up robust features (SURF) utilize the grayscale gradient information of the square regions around the key points. Nevertheless, these famous descriptors fail to take advantage of the image color information. Furthermore, their performances will be severely degraded for distorted images and the error due to rotation of their square windows. To address the problems, we proposed Circular Coordinate Combining Shape–Color Descriptor Under Distortion Based SURF (CSCD-SURF) that achieves competitive performance for colorful images with distortion contamination and numerical similar structures. The proposed approach can generate scale-space by adaptive filter to rectify distortion and integrate the SURF descriptor and shape–color information into our own custom concentric circular coordinate with flying colors. Experiments with Institut National de Recherche en Infomatique et Automatique dataset and spherical radial distortion-SIFT distorted image datasets prove that the ability of distortion rectification as well as matching performance against rotation, scaling, viewpoints, and blurring is more competitive than the state-of-the-art methods and its computation cost is acceptable.