The Coast Guard launched an investigation after a swastika was found at its Cape May training center, prompting a sharp response from its commandant.