The HAPNet-CD, a new change detection method, is proposed in this paper to solve the problems of noise misalignment, object boundary ambiguity and low change detection rate of small targets in the processes of encoding and decoding with the existing methods. On the one hand, the encoder of HAPNet-CD adopts siamese branches, in which HRNetV2 is used as the backbone network, and the alignment-and-perturbation-aided difference module is embedded to extract the variation features and difference information. As a result, the high-resolution feature representation can always be maintained in the process of feature extraction, so that the obtained features are more accurate in space. On the other hand, the decoder of HAPNet-CD uses the change features and difference information to construct a hybrid decoder and a differential decoder for decoding. By designing a loss function based on label smoothing, the network pays more attention to the variations of object boundaries and small targets, so that the change detection accuracy of object boundaries and small targets can be improved. Tests were carried out on the public data sets DSIFN-CD and LEVIR-CD, and the experimental results are as follows. Compared with the other 9 mainstream methods, the HAPNet-CD has improved the metrics of Precision, Recall, F1, and IoU by 2.55%,4.58%, 3.59%, and 5.9%, respectively, on the DSIFN-CD dataset. On the LEVIR-CD dataset, the Precision metric is improved by 0.54%, while the metrics of Recall, F1, and IoU are all close to the most advanced level.