We achieved p-(Zn,Mg)O by doping with phosphorous and the conduction type was confirmed by capacitance-voltage properties of metal/insulator/p-(Zn,Mg)O:P diode structures as well as Hall measurements. The p-(Zn,Mg)O:P/n-ZnO junction was grown by pulsed laser deposition on bulk ZnO doped with Sn. Without post-growth annealing, the phosphorous-doped ZnMgO showed p-type conductivity (hole density ~1016 cm-3, mobility ~6 cm2V-1s-1) in the as-grown state. The metal contacts in top-to-bottom p-n junctions were made with Ni/Au as the p-ohmic and Ti/Au as the backside n-ohmic contact. The p-contacts showed improved characteristics after annealing up to 350 - 400 °C, but the n-contacts were ohmic as-deposited. The simple, low temperature growth (≤500 °C) and processing sequence (≤400 °C) shows the promise of ZnO for applications such as low-cost UV light emitters and transparent electronics.