Underwater fish recognition is an important task in fish stock assessment and marine ecosystem studies. Machine learning techniques have been applied to train high-performance fish recognition models from underwater images. However, underwater images often contain extremely noisy backgrounds, hindering the training of accurate recognition models. Traditional methods exploit handcrafted features to train traditional classifiers. These methods often suffer from low recognition accuracy and limited scalability to large-scale datasets. While deep learning approaches have been proposed, the challenge of learning with noisy underwater images has not yet been fully addressed. We propose a discriminative feature learning (DFL) framework to train accurate fish recognition models on noisy underwater images. By leveraging the idea of contrastive learning, DFL encourages the model to learn more discriminative features for images in different classes and similar features for images in the same class. To better address the noisy background problem, DFL also utilizes a regularization technique called attention suppression to prevent the model from paying too much attention to the noisy background. Experimental results on three benchmark datasets validate the superior performance of DFL over the current state-of-the-art deep learning approaches.