The testing of error handling and dynamic security mechanisms often depends on reproducing specific conditions outside the realm of an application’s normal program state. We present RUGRAT, a novel technique to automatically generate tests for these challenging test situations. RUGRAT uses a dynamic compiler to add instructions to the program during execution, and thus dynamically generates tests to exercise code designed to handle uncommon situations during program execution. The RUGRAT testing approach is independent of the source language, requires no modification to the source or binary program under test and generates runtime tests automatically based on a simple test specification. We demonstrate RUGRAT’s capabilities by targeting two particular uncommon situations: handling errors from system and application calls, and testing compiler-inserted extra code changing the code under test. Other approaches require the tester to provide additional on function pointers. Both code coverage and failure detection results indicate that RUGRAT is a cost effective approach that reduces the number of required test inputs and need for vulnerable programs.