Testing REST APIs with Postman

Due to my recent work at FinancialForce.com‘s Product Innovation Lab, I have been creating REST APIs, and those APIs needed testing. Initially, I tested them manually, using tools like SOAP UI for POST requests and the browser for GET requests. But soon I needed a way of performing multiple tests, with different sets of data, in short periods of time. And for that, Postman became the ideal tool. To illustrate this, I have created a simple server with an even simpler REST API in Node.js. It is available in this GitHub repo.

 

The example app

The provided example app implements a server that can be started from a terminal with NPM and Node.js installed. After cloning it, simply go to its folder and call this from the command line:

 

[code language=”bash”]
> npm install
> PORT=8080 npm start
[/code]

 

It exposes the following REST API methods:

  • GET /api/v1.0/math/add/<value1>/<value2>
  • GET /api/v1.0/math/subtract/<value1>/<value2>
  • GET /api/v1.0/math/square/<value>
  • POST /api/v1.0/math/accumulate
    • It expects a JSON body with this format:

[code language=”javascript”]
{
"data": <number to accumulate>
}
[/code]

All of them return a JSON object with this format:

[code language=”javascript”]
{
"input": <input data>,
"result": <result value>;
}
[/code]

The REST API methods are defined in src/server.js. If you want more information about Node.js, NPM and how to create servers in Node, these links can be useful:

 

Now let’s start talking about Postman.

 

A powerful UI for testing APIs

Postman screenshot

There is a native app for all major desktop operation systems. And it performs really well, at least on Mac OS X. This app allows to:

  • Define collections of requests
  • Organise them in folders
  • Export collections to JSON files
  • Parameterise requests
  • Define tests for each request
  • Run collections of tests along with their tests in sequence

The last three points are what make Postman so interesting for our purposes. Almost everything in a request can be parameterised, allowing us to reuse requests with variable data. For example, if we want to take the port from a variable (let’s call it LocalhostPort), we just need to put {{LocalHostPort}} in the URL instead of the port. The actual value will be taken from one of these places:

  1. Global Variables. A set of pairs <key, value> defining values for variables, that are available to all requests at every time.
  2. Environment. A ser of pairs <key, value> defining values for variables. One environment, or none, can be selected each time. If a variable with the same name is defined on both the Global Variables and the selected Environment, the latter takes precedence.
  3. Iteration Data. Apart from the Global Variables and the Environment, it is possible to specify Iteration Data when running requests and tests in Postman. Iteration Data contains a list of sets of pairs <key, value>, defining multiple sets of values for the same variables. It is meant to be used in different runs, passing different data for each run. If a variable with the same name is defined on either the Global Variables or the selected Environment, the value from the Iteration Data takes precedence.

All of them can be exported to JSON files. More information about variables can be found here: Setting up an environment with variables.

A full Postman project (if you want to call it that) has been included in the example app, under the postman folder. Simply import the files by clicking Import at the top left => Choose File => select the files and click Open. You can import the collection (with all the requests and their associated tests), the environment (which defines a default value for some parameters to be used in the requests, in case of calling them independently), and the global variables (which defines the port as 8080). The inputData.json file includes data to iterate through when running the requests from the Runner.

 

The Runner

PostmanRunner Screenshot

By clicking the Runner button at the top left of the app, a separate window focused on running collections is opened. There, we can choose one of the collections (or folders inside a collection), an Environment, and optionally, iteration data, to run all of the requests and their tests in sequence. If you have imported everything after the previous section, then you should be able to select the Nodejs-Postman collection and the Nodejs-Postman environment. The global variables will be implicitly applied. Choose the inputData.json file from Data – Select File and click Start Run. If the server is running (if not, you can start it by following the explanation above), the tests should work fine.

 

The tests

A test in Postman is coded in Javascript and can look like this:

[code language=”javascript”]
var resultObject;

try
{
resultObject = JSON.parse(responseBody);
}
catch (ex)
{
console.log("Unexpected response body: " + ex);
console.log(responseBody);

return;
}

function defined(value) { return value !== null && value !== undefined; }

var input1 = Number(defined(data.Input1) ? data.Input1 : environment.Input1),
input2 = Number(defined(data.Input2) ? data.Input2 : environment.Input2);

tests["Request succeeded"] = responseCode.code === 200;
tests["Input included"] = !!resultObject.input;
tests["Input is an object"] = resultObject.input.constructor === Object;
tests["Input 1 is correct"] = resultObject.input.input1 === input1;
tests["Input 2 is correct"] = resultObject.input.input2 === input2;
tests["Correct result"] = resultObject.result === (input1 + input2);
[/code]

This code goes in the Tests section under a request in Postman. Postman provides a simple API that allows us, among other things, to:

  • Get the request’s response data
  • Get Global, Environment and Iteration Data variables
  • Update Environment variables
  • Define tests

All actual tests are defined by adding an entry to the tests object. The key is the test description, and the value is a boolean variable (true if the test succeeded, false otherwise). If nothing crashes, the whole code snippet is run; Postman will raise error information for each test that failed after the script finishes.

More info about Postman tests here: Testing Examples. Other useful links:

 

Newman: a CLI for Postman

It would be great if we could leverage Postman’s testing features in an automated testing workflow and integrate it in a continuous release system. It is perfectly possible thanks to the availability of Newman, a command-line interface for Postman. You can install it through NPM.

 

Newman is compatible with all files Postman can export, and provides nicely formatted results, which ultimately allows us to integrate these tests into any automated workflow or script. An example of a command line running exactly the same tests as above could be:

[code language=”bash”]
> newman run postman/Nodejs-Postman.postman_collection.json –iteration-data postman/inputData.json –environment postman/Nodejs-Postman.postman_environment.json –globals postman/globals.postman_globals.json
[/code]

 

The line above has been included in the postman.sh script, which can be run with:

[code language=”bash”]
> sh postman.sh
[/code]

… which, in turn, thanks to the scripts property in package.json, can be run with:

[code language=”bash”]
> npm test
[/code]

… providing a very straightforward and standard way of running the tests for this app.

 

Authentication

The provided example is very simple and does not require any authentication. A more realistic case will probably need some kind of authentication, typically OAuth. Postman does provide functionality for authentication, but it is also possible to provide the session token, or access token, depending on the authentication system, retrieved by other means. This token can be provided via header. The way of getting it can differ depending on the platform the API we are testing is hosted on. To get the most of this, an automatic way of getting the token and putting it into the header must be sorted out.

 

Conclusions

Postman, and its CLI Newman, have been extremely useful for me while implementing APIs, thanks to their usability, power and performance. I strongly recommend give them a go in case of working with APIs.

Remote Actions: Leveraging class hierarchies to safely pass data

Sometimes, we need our @RemoteAction to be very flexible. This can lead to the necessity of allowing the caller to pass in some arbitrary information. But by doing so, we could be opening a security breach. In this post, I will show an example of this, as well as a possible solution to the security problem. Some knowledge about VisualForce components and remote actions will be needed to follow this.

Say, we want to implement a VF component that takes some data from the org and shows it in a custom picklist. We’d like this component to retrieve data very quickly as the user interacts with it, so we will use a remote action for this.

 
[code language=”xml”]
<apex:component controller="CustomPicklistController">
<apex:attribute name="objectName" description="Object to query from" type="string" required="true"/>

<script type="text/javascript">
function getRemoteData() {
Visualforce.remoting.Manager.invokeAction(
‘{!$RemoteAction.CustomPicklistController.getData}’,
‘{!objectName}’,
function(result, event){

},
{escape: true}
);
}
</script>

</apex:component>
[/code]

 
This is the controller:
[code language=”java”]
public with sharing class CustomPicklistController {

@RemoteAction
public static CustomPicklistData getData(String objectName) {
List<SObject> records = retrieveData(objectName);
CustomPicklistData data = convertData(records);
return data;
}

private static List<SObject> retrieveData(
String objectName
) {
String safeObjectName = getSafeSOQLString(objectName);
String dbQuery = ‘SELECT Id, Name FROM ‘ + safeObjectName;
return Database.query(dbQuery);
}
}
[/code]

 
… where getSafeSOQLString() makes sure no malicious string breaks our query by injecting SOQL code. In the example, there is no filter for the query. We are retrieving all records for the given object.

We want this component to be flexible enough to allow us to set custom specific filters wherever we instantiate it. A first approach would consist of passing the filter text as attribute to the component. This is the definition:

[code language=”xml”]
<apex:attribute name="filterText" type="String" required="false"
description="Filter to append to the query"/>
[/code]

 
The controller would be updated as follows:

[code language=”java”]
public with sharing class CustomPicklistController {

@RemoteAction
public static CustomPicklistData getData(String objectName, String filterText) {
List<SObject> records = retrieveData(objectName, filterText);
CustomPicklistData data = convertData(records);
return data;
}

private static List<SObject> retrieveData(
String objectName,
String filterText
) {
String safeObjectName = getSafeSOQLString(objectName);
String whereClause = String.isBlank(filterText) ? ” : ‘ WHERE ‘ + filterText;
String dbQuery = ‘SELECT Id, Name FROM ‘ + safeObjectName + whereClause;
return Database.query(dbQuery);
}
}
[/code]

 
An example of instantiation for this component can look like:

[code language=”xml”]
<CustomPicklistController objectName="Account" filterText="{!queryCondition}"/>
[/code]

 
… where queryCondition would be something like:

[code language=”java”]
public String getQueryCondition() {
return ‘Name LIKE \’A%\”;
}
[/code]

 
Unfortunately, we cannot apply getSafeSOQLString to filterText, as it is a piece of SOQL code. If we escaped it in order to prevent SOQL injection, it would stop working as well. We control front-end too in this case, so we could think that taking care of making front-end secure (so the user is not able to provide a fully free input for this attribute) is enough. But it isn’t: data is passed to back-end as a JSON string, which is not secure. We need an alternative way of passing the filter information from the front-end to the back-end.

 

Using precompiled, back-end data

 
So we want a component that gives us flexibility to pass whatever filter we need, but also, we don’t want to pass that filter in plain SOQL. We could think of creating a class which instances store the filter somehow codified, and then allows us to retrieve the SOQL string for that filter. A specific instance of that class, with the correct information, would be created from the VF page’s controller and passed to the component via attribute, which, in turn, would then pass the info to the @RemoteAction method. But in the end, it would not be secure either as the filter information is travelling from front-end to back-end, one way or another.

Instead, the flexibility will be moved to the back-end, while the front-end will be restricted to the options implemented in back-end. Let me explain this.

The component will now expect an instance of a class that implements this interface:

[code language=”java”]
public interface ICondition {
String getCondition();
}
[/code]

 
Such instance will not contain any data. It will only give us the information about which specific class implements the condition, so we can call the getCondition() method and get the information we need. This way the query condition is not passed through front end.

For each specific filter, a custom class will be implemented, defining the condition for each component instance:

[code language=”java”]
public class SpecificCondition implements ICondition {
public String getCondition() {
return ‘Name LIKE \’A%\”;
}
}
[/code]

 
The component will now receive the name of the class that implements the condition:

[code language=”xml”]
<apex:component controller="CustomPicklistController">
<apex:attribute name="objectName" type="String" required="true" description="Object to query from"/>
<apex:attribute name="conditionClass" type="String" required="false" default="CustomPicklistController.NoCondition" description="Class implementing ICondition"/>

<script type="text/javascript">
function getRemoteData() {
Visualforce.remoting.Manager.invokeAction(
‘{!$RemoteAction.CustomPicklistController.getData}’,
‘{!objectName}’,
{‘apexType’: ‘c.’+'{!conditionClass}’},
function(result, event){

},
{escape: true}
);
}
</script>

</apex:component>
[/code]

 
What happened here? We are basically applying what this article from Salesforce’s official documentation says about how to pass interface instances to a @RemoteAction method: the instance itself is not passed. What we pass, is a string with the class name (and nothing else in our example given the instance contains no data). The system will then, automatically, take care of instantiating the class.

As you can see, the default c namespace is being concatenated here:

[code language=”xml”]
<script type="javascript">

{‘apexType’: ‘c.’+'{!conditionClass}’}

</script>
[/code]

 
This is to simplify the caller so it does not need to know about the namespace thing. Of course, if we wanted to make this component available from outside our package, we would need to tweak this.

Another interesting detail is the default value given to the attribute: CustomPicklistController.NoCondition. This references a class that we will include with the component’s controller:

[code language=”java”]
public class NoCondition implements ICondition {
public String getCondition() {
return ”;
}
}
[/code]

 
The reason why we need this, is that we cannot pass null or inexistent classes to a @RemoteAction, so if we don’t want to apply any filter, we need a default class that actually implements no filter.

With all of this settled, the @RemoteAction method now looks like this:

[code language=”java”]
public with sharing class CustomPicklistController {

@RemoteAction
public static CustomPicklistData getData(String objectName, ICondition condition) {
List<SObject> records = retrieveData(objectName, condition);
CustomPicklistData data = convertData(records);
return data;
}

private static List<SObject> retrieveData(
String objectName,
ICondition condition
) {
String safeObjectName = getSafeSOQLString(objectName);
String filter = condition.getCondition();
String whereClause = filter.isEmpty() ? ” : ‘ WHERE ‘ + filter;
String dbQuery = ‘SELECT Id, Name FROM ‘ + safeObjectName + whereClause;
return Database.query(dbQuery);
}
}
[/code]

 
So, the Visualforce page instantiates the component this way:

[code language=”xml”]
<CustomPicklistController objectName="Account" conditionClass="{!conditionClass}"/>
[/code]

 
… and the condition class is returned from the VF page’s controller:

[code language=”java”]
public String getConditionClass() {
return ‘WhateverPageController.SpecificCondition’;
}
[/code]

 
(… where WhateverPageController is the VF page’s controller, and SpecificCondition is the specific condition class we saw before, nested within the controller class).

 

Conclusions

With the proposed solution, we have avoided having to open a door to potential SOQL injection attacks by not allowing to pass any data, but only information to access the code that actually generates the data, and all of this while leveraging @RemoteAction’s speed and keeping a great flexibility for the component’s user.

Thanks to my teammates at FinancialForce Ana Cristina López, Abel Martos and Shaun Doyle for their great contribution while looking for a solution to the problem.