3D imaging is demanding technology required in fluorescence microscopy. Even though holography is a powerful technique, it could not be used easily in fluorescence microscopy because of low coherence of fluorescence light. Lately, several incoherent holographic methods such as scanning holography, Fresnel in coherent correlation holography (FINCH), and self-interference incoherent digital holography (SIDH) have been proposed. However, these methods have many problems to be overcome for practical applications. For example, DC term removal, twin image ambiguity, and phase unwrapping problems need to be resolved. Off-axis holography is a straightforward solution which can solve most of these problems. We built an off-axis SIDH system for fluorescence imaging, and investigated various conditions and requirements for practical holographic fluorescence microscopy. Our system is based on a modified Michelson interferometer with a flat mirror at one arm and a curved mirror at the other arm of the interferometer. We made a phantom 3D fluorescence object made of 2 single-mode fibers coupled to a single red LED source to mimic 2 fluorescence point sources distributed by a few tens of micrometers apart. A cooled EM-CCD was used to take holograms of these fiber ends which emit only around 180 nW power.