The prognosis of renal function decline in Autosomal Dominant Polycystic Kidney Disease (ADPKD) is vital for early intervention. Currently, the accepted biomarkers are height-adjusted total kidney volume (HtTKV) with estimated glomerular filtration rate (eGFR) and patient age. However, the HtTKV delineation is time-consuming and prone to observer variability. Furthermore, improvement in prognosis can be achieved by incorporating automatically generated features of MRI images of kidney volumes in addition to the conventional biomarkers. Hence, to improve the prognosis we develop two deep learning algorithms. At first, we create an automated kidney volume segmentation model that can accurately calculate HtTKV. Secondly, we use the segmented kidney volumes with the predicted HtTKV, age, and eGFR at the baseline visit. Here, we use a combination of convolutional neural network (CNN) and multi-layer perceptron (MLP) for the prognosis of CKD stages 3A, 3B, and a 30 % decline in eGFR after 8 years from the baseline visit. We obtain AUC scores of 0.96, 0.96, and 0.95 for CKD stages 3A, 3B, and 30% decline in eGFR, respectively. Furthermore, our algorithm achieves a Pearson correlation coefficient of 0.81 between predicted and measured eGFR decline.