The current design process for designing with multi-gigabit serial links is not very robust for system level engineers. Basically it consists of creating loss budgets, performing transmission channel bandwidth simulations, and possibly doing some time domain simulation at typical corners in a spice simulator. There is not much opportunity in this process to really simulate and determine the sensitivity of the system to various things under the system designs control such as trace dimensions, AC coupling capacitor placement, via type and dimensions, board material, etc. The signal integrity design process at lower data rates is well developed and involves running hundreds of parametric sweep simulations across corners for impedance, trace length, chip process and temperature, different levels of crosstalk coupling, etc. Channel analysis tools are emerging from SiSoft (Quantum Channel Designer) and Cadence (Allegro SI GXL). These tools use advanced behavioral models instead of the spice models, and allow the same type of parametric sweep simulations that are useful in preroute simulation of the lower data rate signals to help determine the solution space. There isn't much model support yet from chip vendors for these tools, but the concept seems very promising.
There was an article in EDN magazine recently by someone from Vitesse describing a home grown channel analysis and simulation process [1]. The process is based on creating models of both the transmission channel and the channel drivers from measurements with a TDR and oscilloscope. The measurements are then used to construct simulations with arbitrary data patterns using Scilab mathematic software. They claim to have very good correlation with measurement with this process. The article does not have too many implementation details about the channel analysis simulation process, but it does give a high level overview of how it works. It might not make sense for everyone to implement this themselves, but the article gives a lot of insight into how the new channel analysis tools work, and I thought it was very interesting.
[1] Sweetman, Eric. July 10, 2008. "Measurement-based Simulation Simplifies Analysis of Lossy Backplanes and Cables." EDN Magazine.