Soil is one of the essential natural resources that is at risk from heavy metal pollution. The traditional sampling method for soil heavy metal monitoring and assessment cannot meet the requirements for large-scale areas. The purpose of this study was to estimate the soil heavy metal concentrations based on Gaofen 5 (GF5) satellite hyperspectral imagery for the assessment of the heavy metal pollution in the study area and to analyze the scale effect under different resolutions. A total of 96 topsoil samples were collected in this work, and these samples were analyzed for the arsenic (As), cadmium (Cd), chromium (Cr), copper (Cu), nickel (Ni), lead (Pb), and zinc (Zn) contents. To solve the problem of the insignificant features caused by the complex imaging conditions of spaceborne hyperspectral satellite imagery, the binary weight symbiotic organisms search algorithm (BWSOS) was developed. After feature selection based on the BWSOS method, the heavy metal contents are inverted by the use of support vector machine regression. The experimental results show that the BWSOS feature selection method shows a good performance, with the Rp2 values for As, Cd, Cr, Cu, Ni, Pb, and Zn being 0.67, 0.68, 0.73, 0.71, 0.66, 0.65, and 0.71, respectively. Based on the estimated heavy metal concentration maps, the geoaccumulation index (I_geo), the pollution index, and the potential ecological risk index were calculated to assess the heavy metal pollution status in the study area. The results showed that only As contamination is present at a significant level, but with a low level of potential risk for the whole study area. A comparison with the results obtained using HyMap airborne hyperspectral imagery showed that the GF5 satellite hyperspectral imagery can obtain consistent results for heavy metal pollution assessment. The airborne hyperspectral imagery can provide more fine details, whereas the spaceborne hyperspectral imagery is more suitable for large-scale pollution assessment at a low cost.