An SW-banyan network can play an important role in an optical interconnection network. A smart and compact SW-banyan network with a 2×2 node switch is proposed that uses polarizing beamsplitters (PBSs), phase spatial light modulators (PSLMs), and mirrors. PBS features include an s-component (perpendicular to the incident plane) of the incident light beam that is reflected, and a p-component (parallel to the incident plane) that passes through it. According to switching logic, under the control of external electrical signals, a PSLM controls the routing paths of the signal beams, i.e., the polarization of each optical signal is rotated or not rotated 90 deg by a programmable PSLM. Since the proposed optical setup consists of only optical polarization elements, it is compact in structure and possesses a low energy loss, a high signal-to-noise ratio, and a large number of available optical channels. Finally, the discussions and experimental results show that the SW-banyan network proposed here may be used in optical communication and optical information processing.