A mesoscale defect dynamics model has been developed by coupling dislocation dynamics (DD) and finite element modeling (FEM). In this concurrently coupled model, the DD framework tracks the dynamic evolution of dislocation microstructure, while the stress fields with the given plastic strain (or transformation strain) field from DD are calculated in the finite element model. The developed model has been validated to accurately compute elastic interactions between multiple dislocation segments and to successfully handle the rotation of material and crystal coordinate systems in finite deformation theory. It could also provide the capability to account for complex loading and boundary conditions as well as multi-physical phenomena with enhanced computation efficiency. In this talk, we will present a dislocation-based damage model using the developed mesoscale defect dynamics model. In contrast to continuum based phenomenological models, our model could provide a unique opportunity to investigate detailed dynamic evolution of microstructure. Especially, the collective motion of dislocations could be correlated with the macroscopic damage behavior of the materials in various loading conditions and extreme environments. Additionally, micromechanical characterization results of single crystalline show good agreement with observations in terms of defect microstructure.