Moving target defense (MTD) is an emerging defense principle that aims to dynamically change attack surface to confuse attackers. By dynamic reconfiguration, MTD intends to invalidate the attacker's intelligence or information collection during reconnaissance, resulting in wasted resources and high attack cost/complexity for the attacker. One of the key merits of MTD is its capability to offer 'affordable defense,' by working with legacy defense mechanisms, such as intrusion detection systems (IDS) or other cryptographic mechanisms. On the other hand, a well-known drawback of MTD is the additional overhead, such as reconfiguration cost and/or potential interruptions of service availability to normal users. In this work, we aim to develop a highly secure, resilient, and affordable MTD-based proactive defense mechanism, which achieves multiple objectives of minimizing system security vulnerabilities and defense cost while maximizing service availability. To this end, we propose a multi-agent Deep Reinforcement Learning (mDRL)-based network slicing technique that can help determine two key resource management decisions: (1) link bandwidth allocation to meet Quality-of-Service requirements and (2) the frequency of triggering IP shuffling as an MTD operation not to hinder service availability by maintaining normal system operations. Specifically, we apply this strategy in an in-vehicle network that uses software-defined networking (SDN) technology to deploy the IP shuffling-based MTD, which dynamically changes IP addresses assigned to electronic control unit (ECU) nodes to introduce uncertainty or confusion for attackers.