We demonstrate a dual-rail optical Raman memory inside a polarization interferometer; this enables us to store polarization-encoded information at GHz bandwidths in a room-temperature atomic ensemble. By performing full process tomography on the system we measure up to 97\pm1% process fidelity for the storage and retrieval process. At longer storage times, the process fidelity remains high, despite a loss of efficiency. The fidelity is 86\pm4% for 1.5 \mu s storage time, which is 5,000 times the pulse duration. Hence high fidelity is combined with a large time-bandwidth product. This high performance, with an experimentally simple setup, demonstrates the suitability of the Raman memory for integration into large-scale quantum networks.