Students may switch or drop courses at the beginning of the semester. If a student is no longer in your class, you can remove them from your roster.
FYI: If you are integrating with Google Classroom or Clever, you will need to make the changes in the integration you are using.
More here on updating Classroom roster