This paper is devoted to the analysis of the impact of changes in olive urban forests affected by Xylella fastidiosa on ecosystem services. The focus is on microclimate and thermal comfort evaluated by two indices: the temperature of equivalent perception (TEP) and the predicted mean vote (PMV), which take into account both microclimate parameters and personal factors (heat resistance of clothing and human activity). The work has been carried out through (i) a qualitative analysis of the potential ecosystem services changes caused by temporary transition from olive groves to uncultivated soil, (ii) a study of the potential change of land use from monumental olive groves to other types of use, and (iii) a quantitative analysis on microclimate impact due to the loss of ecosystem services in two selected neighborhoods located in the Apulia region and chosen due to their proximity to the urban context. The analysis revealed that (i) direct effects on ecosystem services are principally linked with regulation functions and cultural services, (ii) a critical loss of cultural value of monumental olive groves occurred in the two neighborhoods, (iii) such a loss may lead to an increase of TEP and PMV, indicating a decrease of thermal comfort in the whole neighborhoods. Thus, it is necessary to plan the replanting policies of the use of the areas affected by X. fastidiosa not only in terms of agricultural planning but also in terms of landscape, urban planning, and human well-being.