Biomedical Named Entity Recognition is a difficult task, aimed to identify all named entities in medical literature. The importance of the task becomes apparent as these entities are used to identify key features, enable better search results and can accelerate the process of reviewing related evidence to a medical case. This practice is known as Evidence-Based Medicine (EBM) and is globally used by medical practitioners who do not have the time to read all the latest developments in their respective fields. In this paper we propose a methodology which achieves state-of-the-art results in a plethora of Biomedical Named Entity Recognition datasets, with a lightweight approach that requires minimal training. Our model is end-to-end and capable of efficiently modeling significantly longer sequences than previous models, benefiting from inter-sentence dependencies.