In organic photovoltaics (OPV), perylene diimide (PDI) acceptor materials are promising candidates to replace the commonly used, but more expensive fullerene derivatives. The use of alternative acceptor materials however implies new design guidelines for OPV devices. It is therefore important to understand the underlying photophysical processes, which either lead to charge generation or geminate recombination. In this contribution, we investigate radiative losses in a series of OPV materials based on two polymers, P3HT and PTB7, respectively, which were blended with different PDI derivatives. Our time-resolved photoluminescence measurements (TRPL) allow us to identify different loss mechanisms by the decay characteristics of several excitonic species. In particular, we find evidence for unfavorable morphologies in terms of large-scale pure domains, inhibited exciton transport and incomplete charge transfer. Furthermore, in one of the P3HT-blends, an interfacial emissive charge transfer (CT) state with strong trapping character is identified.