We present a new sparse linear solver for GPUs. It is designed to work with structured sparse matrices where all the non-zeros are on a few diagonals. Several iterative algorithms are implemented, both on CPU and GPU. The GPU code is designed to be fast yet simple to read and understand. It aims to be as accurate as possible, even on chips that do not support double-precision floating-point arithmetic. Several benchmarks show that GPU algorithms are much faster than their CPU counterpart while their accuracy is satisfying.