The phrase 'keep something at bay' means; that you try to stop a problematic situation from getting closer to you.
Example in use: "That kid is very problematic; I am doing my best to keep him at bay."
The phrase "at bay" is a very old one. It first appeared in 1300, at the time it implied the chorus of barks coming from a pack of chasing dogs close behind their prey.