A Hawkes process on $\mathbb{R}$ is a point process whose intensity function at time $t$ is a functional of its past activity before time $t$. It is defined by its activation function $\Phi$ and its memory function $h$. In this paper, the Hawkes property is expressed as an operator on the sub-space of non-negative sequences associated to distances between its points. By using the classical correspondence between a stationary point process and its Palm measure, we establish a characterization of the corresponding Palm measure as an invariant distribution of a Markovian kernel. We prove that if $\Phi$ is continuous and its growth rate is at most linear with a rate below some constant, then there exists a stationary Hawkes point process. The classical Lipschitz condition of the literature for an unbounded function $\Phi$ is relaxed. Our proofs rely on a combination of coupling methods, monotonicity properties of linear Hawkes processes and classical results on Palm distributions. An investigation of the Hawkes process starting from the null measure on $\mathbb{R}_-$, the empty state, plays also an important role. The linear case of Hawkes and Oakes is revisited at this occasion. If the memory function $h$ is an exponential function, under a weak condition it is shown that there exists a stationary Hawkes point process. In this case, its Palm measure is expressed in terms of the invariant distribution of a one-dimensional Harris ergodic Markov chain. When the activation function is a polynomial $\Phi$ with degree ${>}1$, there does not exist a stationary Hawkes process and if the Hawkes process starts from the empty state, a scaling result for the accumulation of its points is obtained.