Which mode of IPSec encrypts both the payload and the header?

Tunnel Mode is the correct answer because it is designed to encapsulate the entire original IP packet, which includes both the payload (the data being transmitted) and the original header (which contains routing information). In Tunnel Mode, a new IP header is added as the packet is encapsulated, thus providing both encryption and authentication for the entire packet. This offers an added layer of security, especially for virtual private networks (VPNs), where the confidentiality of the data being transmitted across untrusted networks is crucial.

Tunnel Mode is particularly useful when establishing secure communications between two endpoints over potentially insecure networks, as it can effectively hide the original packet's source and destination from third parties. This contrasts with Transport Mode, which only encrypts the payload and leaves the original IP header intact, thereby not providing the same level of security for IP routing information. The other options, such as Encapsulation Mode and Layer 2 Mode, are not standard terminologies within the context of IPSec and do not accurately describe the functions of IPSec modes.