Phosphorite exploitation poses environmental risks due to potentially hazardous elements (PHEs). This study investigates their distribution, fractionation, and origin in Algeria’s sedimentary deposits. Whole-rock (n = 16), grain size fractions (n = 15), and phosphate particles (n = 32) were analyzed using petrographic, SEM-EDS, ICP-MS, and LA-ICP-MS techniques. Robust Principal Component Analysis (Robust PCA) and environmental indices, including enrichment factors (EF) and potential ecological risk index (PERI), were applied. Samples contains carbonate-fluorapatite (CFA) with minor carbonate and silica, while SEM-EDS occasionally detects sulfide inclusions in P-particle pores. Chemical analysis reveals high CaO and P2O5 (up to 45 and 38 wt%, respectively), along with elevated Sr (516–2597 mg/kg), Zn (106–609 mg/kg), Cr (199–348 mg/kg), Cd (0.5–143 mg/kg), and U (11–126 mg/kg). According to Robust PCA, CFA hosts, through substitutions, U, Sr, Y, V, Cr, Pb, Cd, Cu, and Zn, while the matrix retains Nb, Rb, Ta, Hf, Zr, Li, Cs, Mo, As, and Th, via adsorption. Fine fractions and glauconite show enhanced PHEs, with peak enrichment at the Paleocene-Eocene transition. Environmental assessments reveal extreme ecological risks, particularly from Cd and U, necessitating selective mining to mitigate contamination in PHE-enriched layers, along with appropriate waste management.