As a non-invasive brain imaging technology, functional magnetic resonance imaging (fMRI) provides a basic tool for brain functional network modeling and brain disease diagnosis. Problems, such as large number of parameters, low training efficiency, and poor interpretability, are encountered in mainstream models because of the high complexity of fMRI and brain networks. To solve these problems, a novel structure feature combined graph neural network (SFC-GNN) with a low number of parameters is proposed. In particular, SFC-GNN is composed of 1) the graph convolution layer of the brain region perception and 2) the node pooling layer of the graph structure feature (GSF). It also receives the sparse brain graph modeled by each subject’s fMRI as input. Especially, the GSF layer can select brain regions that are important for classification, thereby localizing all active regions related to brain disease. Moreover, a group network is constructed according to the correlation among subjects, and SFC-GNN can be extended further to a node classification model to achieve better diagnosis performance. The proposed method has been validated on the ABIDE and ADNI datasets, thereby showing the effectiveness of our proposed method in various experiments.