We propose a neural network-based watermarking approach to defend against speaker identity manipulation attacks. Our method extracts the source speaker embedding from the carrier waveform and embeds it back into the waveform before transmission. After undergoing various channel transmissions and potential identity manipulation attacks, the receiver reconstructs the source speaker embedding from the extracted watermark and compares it with the embedding obtained from the received waveform to assess the likelihood of identity manipulation. Experimental results demonstrate the robustness of the proposed framework against multiple digital signal processing based transmissions and attacks. However, we observe that while neural codec algorithms have minimal impact on manipulating speaker identity, they significantly degrade watermark detection accuracy, leading to failures in detecting identity manipulation.