In terms of some of the more challenging things I thought while developing a keyword driven framework...
*Getting it to run unsupervised via command line so it can be turned into a nightly build/test run process. - The command line to compile RFT tests that uses reflection and 3rd party libraries was very tricky. Some things had to be compiled using Java, and others had to be done through RFT itself, and there isn't a simple ANT task RFT uses.
Proxy Development - Proxy development is somewhat a paint to get a good tutorial one, as not too many people are doing it. One of the biggest annoyances is unable to run the proxy in a debugger since it's loaded into RFT in Java, while the proxy was done in C#. Debugging a proxy was annoying series of correlating log messages.
Implementing a keyword driver - This was somewhat on the more challenging side as with Java, you can't do a simple Eval() call like a scripting language. To do this, you have to use the Reflection API, which can every call can generate around 6-8 different types of exceptions that need to be caught and handled correctly.