Motion is a key feature for a wide class of computer vision approaches to recognize actions. In this article, we show how to define bio-inspired features for action recognition. To do so, we start from a well-established bio-inspired motion model of cortical areas \V1\ and MT. The primary visual cortex, designated as V1, is the first cortical area encountered in the visual stream processing and early responses of \V1\ cells consist in tiled sets of selective spatiotemporal filters. The second cortical area of interest in this article is area \MT\ where \MT\ cells pool incoming information from \V1\ according to the shape and characteristic of their receptive field. To go beyond the classical models and following the observations from Xiao et al. [61], we propose here to model different surround geometries for \MT\ cells receptive fields. Then, we define the so-called bio-inspired features associated to an input video, based on the average activity of \MT\ cells. Finally, we show how these features can be used in a standard classification method to perform action recognition. Results are given for the Weizmann and \KTH\ databases. Interestingly, we show that the diversity of motion representation at the \MT\ level (different surround geometries), is a major advantage for action recognition. On the Weizmann database, the inclusion of different \MT\ surround geometries improved the recognition rate from 63.01 Â± 2.07% up to 99.26 Â± 1.66% in the best case. Similarly, on the \KTH\ database, the recognition rate was significantly improved with the inclusion of \MT\ different surround geometries (from 47.82 Â± 2.71% up to 92.44 Â± 0.01% in the best case). We also discussed the limitations of the current approach which are closely related to the input video duration. These promising results encourage us to further develop bio-inspired models incorporating other brain mechanisms and cortical areas in order to deal with more complex videos.