A Fourier transform spectrometer (FTS) has been used to observe solar activities due to its ultra-high spectral resolution. However, the FTS in-band spectra are usually distorted and some artifacts appear in out-of-band regions due to nonlinear effects. Therefore, the FTS nonlinear problem must be corrected. In this study, we proposed a novel method to correct the nonlinear effects using simulated annealing. We simulated several nonlinear spectra to evaluate the performance of our method. The calculated quadratic coefficients are extremely close to the given values, demonstrating that the method is effective and accurate. The proposed method is further used to correct the blackbody and solar spectra with nonlinearity obtained by Bruker IFS-125HR installed at the Huairou Solar Observing Station, which is a pathfinder for the accurate infrared magnetic field measurements of the Sun project. To the blackbody spectra, the nonlinearity in low- and high-frequency regions are corrected by 89.09% and 60.84%. The nonlinear correction of the solar spectra in the low- and high-frequency regions have reached 65.34% and 81.04%, respectively. These results prove that our method can correct the nonlinear problem to improve the data accuracy.