Polarization beam splitters (PBSs) have received increasing attention due to their unique ability to split or combine two orthogonal linear polarization modes. We propose a compact all-fiber photonic crystal fiber-based terahertz (THz) PBS with a twin butterfly core. The design and optimization of the device are performed using the finite difference time domain method in combination with perfected matching layer boundary conditions. The influences of the PBS’s structural parameters on its coupling length as well as coupling length ratio are examined. The findings reveal that at 1 THz, the maximum polarization extinction ratio is 78.5 and 52.5 dB for x- and y-polarization lights, respectively, and it has an operation bandwidth of 24 GHz. The length of the PBS is merely 25.1 mm. Due to its remarkable characteristics and outstanding compatibility with the current THz optical fiber information system, the device will be widely applied in future THz radar, remote sensing, environmental monitoring, and imaging systems.