This paper presents a novel algorithm for detection of multiple flaws in structures as an inverse process, where the forward problem is based on eXtended Finite Element Method (XFEM). The proposed algorithm can be applied to quantify any flaw with arbitrary shape and size (e.g., cracks, voids, or their combination) whose number is unknown beforehand, and is shown to be significantly more efficient than other methods proposed in the literature. The basic concept is to employ a two-scale optimization framework, where first a coarse flaw region is detected and then fine scale convergence is used to zoom in on the flaw. Both optimization steps rely on a forward problem in which an XFEM model with both circular and elliptical enrichments is used. The advantage of XFEM is in the alleviation of costly remeshing techniques when candidate flaws keep updating with the optimization process. The proposed hierarchical optimizers include both heuristic and gradient-based algorithms, such as the discrete artificial bee colony (DABC) algorithms and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method. In details, the first step employs a DABC optimization as a coarse-scale search where the optimizer is limited to specific solutions that correspond to locations and shapes of flaws, thus converting a continuous optimization problem into a discrete optimization with a small number of choices. The results of the first step provide local subdomains and rough identified flaw parameters, which can be considered as search space reduction and initial guess for a fine tuning optimization step. This solution zooming is carried out by the BFGS method and leads to a fast converging method as illustrated on two benchmark detection examples.