This evening I encountered this question on twitter:
The answers were clear and so was mine: don't try to mimic XML or something using an Agent. Use a Custom REST Service instead.
The principle is very simple: create an XPage, place a REST service component on it and point it to a custom service bean which you have to create as a Java class. This Java class has to extend the CustomServiceBean class and to incorporate the renderService method which should generate an output. This output nowadays should be a JSON response.
I set up a simple example NSF to demonstrate how this works here