When configuring a multitone analysis, you have to be careful about setting the Initial Simulation Period to be long enough for the system to reach steady-state. A too short Initial Simulation Period can have a big impact on the correctness of the calculated transfer function.
You may also have to experiment with the Amplitude setting. Start with a small value and increase it gradually until you get a clean result. Both too small and too large Amplitude settings can produce noisy results.
Finally, keep in mind that the frequency resolution is given by the start of your frequency range. In other words, if you start at 100 Hz, then the data-points in the Bode plot will be spaced by 100 Hz (except for at higher frequencies, at which PLECS may thin out the data-points to accelerate the analysis). Lowering the starting frequency may increase the quality of the transfer function in the mid-frequency range, but will also increase the time the analysis takes to complete. Take a look at the PLECS documentation and review the other options with regards to setting the frequency range and frequency points.
