The interaction of acute myeloid leukemia (AML) and acute lymphoblastic leukemia (ALL) blasts with the bone marrow microenvironment regulates self-renewal, growth signaling, as well as chemotherapy resistance. The chemokine receptor, CXC receptor 4 (CXCR4), with its ligand chemokine ligand 12 (CXCL12), plays a key role in the survival and migration of normal and malignant stem cells to the bone marrow. High expression of CXCR4 on AML and ALL blasts has been shown to be a predictor of poor prognosis for these diseases. Several small molecule inhibitors, short peptides, antibodies, and antibody drug conjugates have been developed for the purposes of more effective targeting and killing of malignant cells expressing CXCR4. In this review we will discuss recent results and strategies in targeting CXCR4 with these agents in patients with AML or ALL.