Unsupervised embedding learning aims to learn highly discriminative features of images without using class labels. Existing instance-wise softmax embedding methods treat each instance as a distinct class and explore the underlying instance-to-instance visual similarity relationships. However, overfitting the instance features leads to insufficient discriminability and poor generalizability of networks. To tackle this issue, we introduce an instance-wise softmax embedding with cosine margin (SEwCM), which for the first time adds margin in the unsupervised instance softmax classification function from the cosine perspective. The cosine margin is used to separate the classification decision boundaries between instances. SEwCM explicitly optimizes the feature mapping of networks by maximizing the cosine similarity between instances, thus learning a highly discriminative model. Exhaustive experiments on three fine-grained image datasets demonstrate the effectiveness of our proposed method over existing methods.