Collaborative robots are increasingly used in domestic, public, and industrial settings. This study investigated the human user's safety perception and acceptance of robots that use different modalities to communicate with users. A survey study was conducted to compare potential robot users' perceived safety and acceptance of three robot-to-human communication modalities, including visual display, visual light, and auditory beep. The results showed that the perceived safety of a robot is associated with the user's acceptance of the robot, suggesting that if human users feel safer around a robot, they are more likely to use it. Among the three interface modalities, participants showed higher perceived safety and acceptance for the robots using visual modalities than the auditory modality. Furthermore, this study examined whether the perceived safety and acceptance of the three modalities differ when human users interact with robots in different contexts. Our results also showed that perceived safety and acceptance for visual display were lower for an industrial robot than a home service robot. The levels of robot autonomy and severity of risks involving a robot did not have significant impacts on the perceived safety and acceptance of the three modalities. Our findings suggest that the design of robot-to-human communication is vital as it is related to users' perceived safety and acceptance, and the users' perception and acceptance of a robot communication modality may vary when interactions with a robot change.