Studying the human brain is of great significance to the development of AI. The analysis of fMRI brain signal of visual cortex is a research hotspot in this field. One of key issues in analyzing fMRI brain signals is how to establish effective connectivity model from brain signals. Traditional analysis methods regard voxel correlations as effective connectivity. However, the result obtained by these methods are not effective connectivity but functional connectivity. To achieve real effective connectivity, we need to analyze the causality relationship between voxels. Therefore, we propose a hierarchical causality network model (Hcausal-Net). The model stratifies the voxels of the visual cortex in fMRI brain signals, and analyzes the causality relationship between voxels by Granger Causality Analysis (GCA), and then infer the effective connectivity given by Hcausal-Net. The voxels sensitive to the stimuli are extracted, and the forward encoding process model of visual perception is established. Finally, in the experiment, we also improved the image restoration method based on machine learning and conduct comparative experiments, and the results are better than the previous work, which proves the effectiveness of the model proposed in this paper.