The answer is no. You cannot remove a paused container. The container has to be in the stopped state before it can be removed.
