Free space optical system is a hot topic, which has gaining more and more attention. But, when the signal transmitted in the channel, the performance could be severely degraded due to the atmosphere turbulent. The purpose of this paper is to find a most suitable modulation method for FSO system under FSO channel. The performance of power efficiencies, bandwidth efficiency, BER and SNR for the
... [Show full abstract] four modulation schemes have studied and compared in this paper include On-Off keying (OOK), Binary Phase Shift Keying (BPSK), Differential Phase Shift Keying (DPSK) and Quadrature Phase Shift Keying (QPSK) without atmospheric turbulence. Numerical experiments show that BPSK and QPSK schemes are better compared to other schemes in BER performance and power requirements. When take intensity scintillation under Gamma-Gamma turbulence channel into consideration and the average BER is derived with Meijer- G function, BER performances of BPSK and QPSK scheme approximate the same. Compared with BPSK, BER performance for QPSK is 3dB lower. From the simulation results, modulation for BPSK is robust resist turbulence. As a result, BPSK scheme is suitable for free-space optical communication system.