A security dilemma in international relations arises when one state's efforts to enhance its security, such as increasing military capabilities, inadvertently threaten other states. This situation can lead to an arms race, with countries feeling compelled to respond defensively to each other's enhancements, ultimately resulting in decreased overall security. The security dilemma illustrates the complexities of trust and cooperation in global politics.