This paper is devoted to the computation of the solution to fractional linear algebraic systems using a differential-based strategy to evaluate matrix-vector products $A^\alpha x$, with $\alpha \in \mathbb{R}^{*}_{+}$. More specifically, we propose ODE-based preconditioners for efficiently solving fractional linear systems in combination with traditional sparse linear system preconditioners. Different types of preconditioners are derived (Jacobi, Incomplete LU, Padé) and numerically compared. The extension to systems $f (A)x = b$ is finally considered.