We develop a new processing algorithm for the analysis of high-speed quantitative polarized light microscopy measurements. The measurements are obtained using a high-speed rotating polarizing component and a camera, collecting images at several polarizer angles per rotation. The technique uses data from less than the full quarter-waveplate rotation and then performs an optimal fit of the measured data to the expected response curves. Thus, it allows to increase the effective frame rate of the alignment angle and retardation maps due to the reduction in required images. Due to the complexity of the intensity response curves, a particle swarm optimization method is applied. The algorithm addresses the motion error in high-speed polarization imaging while still using multiple polarization angles for the reconstruction. We apply the algorithm to two example cases: quasi-static loading of a tensile coupon and quality inspection of a polymer fiber during rapid motion. The results demonstrate that increasing the reconstructions per second (i.e., decreasing the number of polarization angles per reconstruction) does not significantly decrease the quality of the reconstructions until ∼2.5 times the increase in reconstructions per second is achieved. Therefore, the developed algorithm is an effective method to increase the effective polarization imaging rate without complex hardware modifications.