Cognitive radio technology is a promising solution to the exponential growth in bandwidth demand sustained by increasing number of ubiquitous connected devices. The allocated spectrum is opened to the secondary users conditioned on limited interference on the primary owner of the band. A major bottleneck in cognitive radio systems is to find the best available channel quickly from a large accessible set of channels. This work formulates the channel exploration-exploitation dilemma as a multi-arm bandit problem. Existing theoretical solutions to a multi-arm bandit are adapted for cognitive radio and evaluated in an experimental test-bed. It is shown that a Thompson sampling based algorithm efficiently converges to the best channel faster than the existing algorithms and achieves higher asymptotic average throughput. We then propose a multihop extension together with an experimental proof of concept.