The mechanisms underlying the establishment, development, and maintenance of the renal vasculature are poorly understood. Here, we propose that the transcription factor “recombination signal binding protein for immunoglobulin kappa J region” (RBP-J) plays a key role in the differentiation of the mural cells of the kidney arteries and arterioles, as well as the mesangial cells of the glomerulus. Deletion of RBP-J in renal stromal cells of the forkhead box D1 (FOXD1) lineage, which differentiate into all the mural cells of the kidney arterioles along with mesangial cells and pericytes, resulted in significant kidney abnormalities and mortality by day 30 postpartum (P30). In newborn mutant animals, we observed a decrease in the total number of arteries and arterioles, along with thinner vessel walls, and depletion of renin cells. Glomeruli displayed striking abnormalities, including a failure of FOXD1-descendent cells to populate the glomerulus, an absence of mesangial cells, and in some cases complete loss of glomerular interior structure and the development of aneurysms. By P30, the kidney malformations were accentuated by extensive secondary fibrosis and glomerulosclerosis. We conclude that RBP-J is essential for proper formation and maintenance of the kidney vasculature and glomeruli.