Adipose tissue is composed of many functionally and developmentally distinct cell types, the metabolic core of which is the adipocyte. The classification of “adipocyte” encompasses three primary types – white, brown, and beige – with distinct origins, anatomic distributions, and homeostatic functions. The ability of adipocytes to store and release lipids, respond to insulin, and perform their endocrine functions (via secretion of adipokines) is heavily influenced by the immune system. Various cell populations of the innate and adaptive arms of the immune system can resist or exacerbate the development of the chronic, low-grade inflammation associated with obesity and metabolic dysfunction. Here, we discuss these interactions, with a focus on their consequences for adipocyte and adipose tissue function in the setting of chronic overnutrition. In addition, we will review the effects of diet composition on adipose tissue inflammation and recent evidence suggesting that diet-driven disruption of the gut microbiota can trigger pathologic inflammation of adipose tissue.