Testing REST APIs with Postman

Due to my recent work at FinancialForce.com‘s Product Innovation Lab, I have been creating REST APIs, and those APIs needed testing. Initially, I tested them manually, using tools like SOAP UI for POST requests and the browser for GET requests. But soon I needed a way of performing multiple tests, with different sets of data, in short periods of time. And for that, Postman became the ideal tool. To illustrate this, I have created a simple server with an even simpler REST API in Node.js. It is available in this GitHub repo.

 

The example app

The provided example app implements a server that can be started from a terminal with NPM and Node.js installed. After cloning it, simply go to its folder and call this from the command line:

 

> npm install
> PORT=8080 npm start

 

It exposes the following REST API methods:

  • GET /api/v1.0/math/add/<value1>/<value2>
  • GET /api/v1.0/math/subtract/<value1>/<value2>
  • GET /api/v1.0/math/square/<value>
  • POST /api/v1.0/math/accumulate
    • It expects a JSON body with this format:
{
	"data": <number to accumulate>
}

All of them return a JSON object with this format:

{
  "input": <input data>,
  "result": <result value>;
}

The REST API methods are defined in src/server.js. If you want more information about Node.js, NPM and how to create servers in Node, these links can be useful:

 

Now let’s start talking about Postman.

 

A powerful UI for testing APIs

Postman screenshot

There is a native app for all major desktop operation systems. And it performs really well, at least on Mac OS X. This app allows to:

  • Define collections of requests
  • Organise them in folders
  • Export collections to JSON files
  • Parameterise requests
  • Define tests for each request
  • Run collections of tests along with their tests in sequence

The last three points are what make Postman so interesting for our purposes. Almost everything in a request can be parameterised, allowing us to reuse requests with variable data. For example, if we want to take the port from a variable (let’s call it LocalhostPort), we just need to put {{LocalHostPort}} in the URL instead of the port. The actual value will be taken from one of these places:

  1. Global Variables. A set of pairs <key, value> defining values for variables, that are available to all requests at every time.
  2. Environment. A ser of pairs <key, value> defining values for variables. One environment, or none, can be selected each time. If a variable with the same name is defined on both the Global Variables and the selected Environment, the latter takes precedence.
  3. Iteration Data. Apart from the Global Variables and the Environment, it is possible to specify Iteration Data when running requests and tests in Postman. Iteration Data contains a list of sets of pairs <key, value>, defining multiple sets of values for the same variables. It is meant to be used in different runs, passing different data for each run. If a variable with the same name is defined on either the Global Variables or the selected Environment, the value from the Iteration Data takes precedence.

All of them can be exported to JSON files. More information about variables can be found here: Setting up an environment with variables.

A full Postman project (if you want to call it that) has been included in the example app, under the postman folder. Simply import the files by clicking Import at the top left => Choose File => select the files and click Open. You can import the collection (with all the requests and their associated tests), the environment (which defines a default value for some parameters to be used in the requests, in case of calling them independently), and the global variables (which defines the port as 8080). The inputData.json file includes data to iterate through when running the requests from the Runner.

 

The Runner

PostmanRunner Screenshot

By clicking the Runner button at the top left of the app, a separate window focused on running collections is opened. There, we can choose one of the collections (or folders inside a collection), an Environment, and optionally, iteration data, to run all of the requests and their tests in sequence. If you have imported everything after the previous section, then you should be able to select the Nodejs-Postman collection and the Nodejs-Postman environment. The global variables will be implicitly applied. Choose the inputData.json file from Data – Select File and click Start Run. If the server is running (if not, you can start it by following the explanation above), the tests should work fine.

 

The tests

A test in Postman is coded in Javascript and can look like this:

var resultObject;

try
{
    resultObject = JSON.parse(responseBody);
}
catch (ex)
{
    console.log("Unexpected response body: " + ex);
    console.log(responseBody);
    
    return;
}

function defined(value) { return value !== null && value !== undefined; }

var input1 = Number(defined(data.Input1) ? data.Input1 : environment.Input1),
    input2 = Number(defined(data.Input2) ? data.Input2 : environment.Input2);

tests["Request succeeded"] = responseCode.code === 200;
tests["Input included"] = !!resultObject.input;
tests["Input is an object"] = resultObject.input.constructor === Object;
tests["Input 1 is correct"] = resultObject.input.input1 === input1;
tests["Input 2 is correct"] = resultObject.input.input2 === input2;
tests["Correct result"] = resultObject.result === (input1 + input2);

This code goes in the Tests section under a request in Postman. Postman provides a simple API that allows us, among other things, to:

  • Get the request’s response data
  • Get Global, Environment and Iteration Data variables
  • Update Environment variables
  • Define tests

All actual tests are defined by adding an entry to the tests object. The key is the test description, and the value is a boolean variable (true if the test succeeded, false otherwise). If nothing crashes, the whole code snippet is run; Postman will raise error information for each test that failed after the script finishes.

More info about Postman tests here: Testing Examples. Other useful links:

 

Newman: a CLI for Postman

It would be great if we could leverage Postman’s testing features in an automated testing workflow and integrate it in a continuous release system. It is perfectly possible thanks to the availability of Newman, a command-line interface for Postman. You can install it through NPM.

 

Newman is compatible with all files Postman can export, and provides nicely formatted results, which ultimately allows us to integrate these tests into any automated workflow or script. An example of a command line running exactly the same tests as above could be:

> newman run postman/Nodejs-Postman.postman_collection.json --iteration-data postman/inputData.json --environment postman/Nodejs-Postman.postman_environment.json --globals postman/globals.postman_globals.json

 

The line above has been included in the postman.sh script, which can be run with:

> sh postman.sh

… which, in turn, thanks to the scripts property in package.json, can be run with:

> npm test

… providing a very straightforward and standard way of running the tests for this app.

 

Authentication

The provided example is very simple and does not require any authentication. A more realistic case will probably need some kind of authentication, typically OAuth. Postman does provide functionality for authentication, but it is also possible to provide the session token, or access token, depending on the authentication system, retrieved by other means. This token can be provided via header. The way of getting it can differ depending on the platform the API we are testing is hosted on. To get the most of this, an automatic way of getting the token and putting it into the header must be sorted out.

 

Conclusions

Postman, and its CLI Newman, have been extremely useful for me while implementing APIs, thanks to their usability, power and performance. I strongly recommend give them a go in case of working with APIs.

Advertisements

Remote Actions: Leveraging class hierarchies to safely pass data

Sometimes, we need our @RemoteAction to be very flexible. This can lead to the necessity of allowing the caller to pass in some arbitrary information. But by doing so, we could be opening a security breach. In this post, I will show an example of this, as well as a possible solution to the security problem. Some knowledge about VisualForce components and remote actions will be needed to follow this.

Say, we want to implement a VF component that takes some data from the org and shows it in a custom picklist. We’d like this component to retrieve data very quickly as the user interacts with it, so we will use a remote action for this.

 

<apex:component controller="CustomPicklistController">
<apex:attribute name="objectName" description="Object to query from" type="string" required="true"/>

    <script type="text/javascript">
        function getRemoteData() {
            Visualforce.remoting.Manager.invokeAction(
                '{!$RemoteAction.CustomPicklistController.getData}',
                '{!objectName}',
                function(result, event){
                    ...
                },
                {escape: true}
            );
        }
    </script>
    
</apex:component>

 
This is the controller:

public with sharing class CustomPicklistController {

    @RemoteAction
    public static CustomPicklistData getData(String objectName) {
        List<SObject> records = retrieveData(objectName);
        CustomPicklistData data = convertData(records);
        return data;
    }

    private static List<SObject> retrieveData(
        String objectName
    ) {
        String safeObjectName = getSafeSOQLString(objectName);
        String dbQuery = 'SELECT Id, Name FROM ' + safeObjectName;
        return Database.query(dbQuery);
    }
}

 
… where getSafeSOQLString() makes sure no malicious string breaks our query by injecting SOQL code. In the example, there is no filter for the query. We are retrieving all records for the given object.

We want this component to be flexible enough to allow us to set custom specific filters wherever we instantiate it. A first approach would consist of passing the filter text as attribute to the component. This is the definition:

<apex:attribute name="filterText" type="String" required="false"
    description="Filter to append to the query"/>

 
The controller would be updated as follows:

public with sharing class CustomPicklistController {

    @RemoteAction
    public static CustomPicklistData getData(String objectName, String filterText) {
        List<SObject> records = retrieveData(objectName, filterText);
        CustomPicklistData data = convertData(records);
        return data;
    }

    private static List<SObject> retrieveData(
        String objectName,
        String filterText
    ) {
        String safeObjectName = getSafeSOQLString(objectName);
        String whereClause = String.isBlank(filterText) ? '' : ' WHERE ' + filterText;
        String dbQuery = 'SELECT Id, Name FROM ' + safeObjectName + whereClause;
        return Database.query(dbQuery);
    }
}

 
An example of instantiation for this component can look like:

<CustomPicklistController objectName="Account" filterText="{!queryCondition}"/>

 
… where queryCondition would be something like:

public String getQueryCondition() {
    return 'Name LIKE \'A%\'';
}

 
Unfortunately, we cannot apply getSafeSOQLString to filterText, as it is a piece of SOQL code. If we escaped it in order to prevent SOQL injection, it would stop working as well. We control front-end too in this case, so we could think that taking care of making front-end secure (so the user is not able to provide a fully free input for this attribute) is enough. But it isn’t: data is passed to back-end as a JSON string, which is not secure. We need an alternative way of passing the filter information from the front-end to the back-end.

 

Using precompiled, back-end data

 
So we want a component that gives us flexibility to pass whatever filter we need, but also, we don’t want to pass that filter in plain SOQL. We could think of creating a class which instances store the filter somehow codified, and then allows us to retrieve the SOQL string for that filter. A specific instance of that class, with the correct information, would be created from the VF page’s controller and passed to the component via attribute, which, in turn, would then pass the info to the @RemoteAction method. But in the end, it would not be secure either as the filter information is travelling from front-end to back-end, one way or another.

Instead, the flexibility will be moved to the back-end, while the front-end will be restricted to the options implemented in back-end. Let me explain this.

The component will now expect an instance of a class that implements this interface:

public interface ICondition {
    String getCondition();
}

 
Such instance will not contain any data. It will only give us the information about which specific class implements the condition, so we can call the getCondition() method and get the information we need. This way the query condition is not passed through front end.

For each specific filter, a custom class will be implemented, defining the condition for each component instance:

public class SpecificCondition implements ICondition {
    public String getCondition() {
        return 'Name LIKE \'A%\'';
    }
}

 
The component will now receive the name of the class that implements the condition:

<apex:component controller="CustomPicklistController">
<apex:attribute name="objectName" type="String" required="true" description="Object to query from"/>
<apex:attribute name="conditionClass" type="String" required="false" default="CustomPicklistController.NoCondition" description="Class implementing ICondition"/>
...
    <script type="text/javascript">
        function getRemoteData() {
            Visualforce.remoting.Manager.invokeAction(
                '{!$RemoteAction.CustomPicklistController.getData}',
                '{!objectName}',
                {'apexType': 'c.'+'{!conditionClass}'},
                function(result, event){
                    ...
                },
                {escape: true}
            );
        }
    </script>
...
</apex:component>

 
What happened here? We are basically applying what this article from Salesforce’s official documentation says about how to pass interface instances to a @RemoteAction method: the instance itself is not passed. What we pass, is a string with the class name (and nothing else in our example given the instance contains no data). The system will then, automatically, take care of instantiating the class.

As you can see, the default c namespace is being concatenated here:

    <script type="javascript">
        ...
        {'apexType': 'c.'+'{!conditionClass}'}
        ...
    </script>

 
This is to simplify the caller so it does not need to know about the namespace thing. Of course, if we wanted to make this component available from outside our package, we would need to tweak this.

Another interesting detail is the default value given to the attribute: CustomPicklistController.NoCondition. This references a class that we will include with the component’s controller:

public class NoCondition implements ICondition {
    public String getCondition() {
        return '';
    }
}

 
The reason why we need this, is that we cannot pass null or inexistent classes to a @RemoteAction, so if we don’t want to apply any filter, we need a default class that actually implements no filter.

With all of this settled, the @RemoteAction method now looks like this:

public with sharing class CustomPicklistController {

    @RemoteAction
    public static CustomPicklistData getData(String objectName, ICondition condition) {
        List<SObject> records = retrieveData(objectName, condition);
        CustomPicklistData data = convertData(records);
        return data;
    }

    private static List<SObject> retrieveData(
        String objectName,
        ICondition condition
    ) {
        String safeObjectName = getSafeSOQLString(objectName);
        String filter = condition.getCondition();
        String whereClause = filter.isEmpty() ? '' : ' WHERE ' + filter;
        String dbQuery = 'SELECT Id, Name FROM ' + safeObjectName + whereClause;
        return Database.query(dbQuery);
    }
}

 
So, the Visualforce page instantiates the component this way:

<CustomPicklistController objectName="Account" conditionClass="{!conditionClass}"/>

 
… and the condition class is returned from the VF page’s controller:

public String getConditionClass() {
    return 'WhateverPageController.SpecificCondition';
}

 
(… where WhateverPageController is the VF page’s controller, and SpecificCondition is the specific condition class we saw before, nested within the controller class).

 

Conclusions

With the proposed solution, we have avoided having to open a door to potential SOQL injection attacks by not allowing to pass any data, but only information to access the code that actually generates the data, and all of this while leveraging @RemoteAction’s speed and keeping a great flexibility for the component’s user.

Thanks to my teammates at FinancialForce Ana Cristina López, Abel Martos and Shaun Doyle for their great contribution while looking for a solution to the problem.

Constructors and global classes in Apex

It is well known that special care must be taken when creating global classes in Apex, because global classes, methods and member variables that are released in a package, cannot be removed in further versions of the same package. Some of the restrictions are well explained in official Salesforce docs like this one and this one, but there are some issues for which it is difficult to be prepared until we face the problem or test it ourselves. This is the case of the thrilling world of constructors in global classes.

 

How do constructors behave in global classes?

Pretty much the same as in non-global ones. But there are some important nuances to be considered.

Let’s say… we are creating a class with no “particular” way of instantiation. We don’t need to pass any argument, there’s no needed initialisation, and it can be freely instantiated. We can then omit the constructor:

global class InstantiateMeIfYouCan {
    global void foo() {
        System.assert(false, 'OK, so you instantiated me. Now what?');
    }
}

Salesforce automatically generates an implicit, global constructor with no parameters. When releasing this class in a package, it can be instantiated and used from the customer’s org:

public class ISwearICanInstantiateYou {
    public static void foo() {
        new VSTest.InstantiateMeIfYouCan().foo();
    }
} 

(… where VSTest is the package’s namespace).

So far, so good.

 

Updating our global class with a new constructor

It turns out that after releasing our class, now for some reason we want it to be instantiable only with a parameter specified in the constructor. So we add a new constructor:

global class InstantiateMeIfYouCan {
    global InstantiateMeIfYouCan(Boolean coolModeActivated) {
        
    }

    global void foo() {
        System.assert(false, 'OK, so you instantiated me. Now what?');
    }
}

As with any other class, global or not, Apex automatically removes the implicit, no-param constructor. This common behaviour is detailed in the official documentation:

If you write a constructor that takes arguments, you can then use that constructor to create an object using those arguments. If you create a constructor that takes arguments, and you still want to use a no-argument constructor, you must include one in your code. Once you create a constructor for a class, you no longer have access to the default, no-argument public constructor. You must create your own.

And Salesforce allows us to package this new version. One could think that, implicit or not, the previously existing constructor with no parameters was global and had already been packaged, so the system would forbid us to perform this action. But the truth is, it doesn’t. When we release this new version, it is possible for the customer to upgrade the package in their org, despite of the fact that the new version breaks their existing code. When trying to run their code with the new version, an error similar to this arises:

Dependent class is invalid and needs recompilation: ISwearICanInstantiateYou: line 3, column 16: Constructor not defined: [VSTest.InstantiateMeIfYouCan]()

OK, so now we know this is a breaking change. Let’s try to fix this and release a new version.

 

Recovering the no-param constructor

We have already released our 1-param constructor, and it’s global, so there’s no way to return to the original status. Now the only thing we can do, is adding an explicit, no-param constructor:

global class InstantiateMeIfYouCan {
    global InstantiateMeIfYouCan() {
        
    }

    global InstantiateMeIfYouCan(Boolean coolModeActivated) {
        
    }

    global void foo() {
        System.assert(false, 'OK, so you instantiated me. Now what?');
    }
}

Again, the system allows us to package the new version and release it. And again, the customer can install the new version with no issues. But when running their code…

Dependent class is invalid and needs recompilation: ISwearICanInstantiateYou: line 3, column 16: Package Visibility: Constructor is not visible: [VSTest.InstantiateMeIfYouCan].<Constructor>()

Weird, isn’t it? So we have explicitly added the constructor and set it as global. We know it is global. But the platform does not get it correctly, even though when entering the class in the customer’s org, it informs about the availability of those methods:

InstantiateMeIfYouCan

Actually, not so weird. The existing class ISwearICanInstantiateYou is linked to a previous version of our package. It’s a matter of changing the version it is using by editing the class and going to Version Settings. If we create another, new class in the customer’s org that makes use of the mysterious no-param constructor:

public class SoYouThoughtICouldNotInstantiateYou {
    public static void foo() {
        new VSTest.InstantiateMeIfYouCan().foo();
    }
} 

It works directly because it is taking the latest version by default.

So, the fact that we didn’t explicitly declare the no-param constructor in the first version, caused issues in the end (our updates are throwing compile errors in customer’s org). This makes it highly advisable to always declare an explicit, global no-param constructor in our global classes in order to prevent potential issues. But it is not always possible: sometimes it does not make sense to allow customers to instantiate our class.

What if we create an explicit, private no-param constructor?

global class InstantiateMeIfYouCan {
    private InstantiateMeIfYouCan() {}

    global void foo() {
        System.assert(false, 'OK, so you instantiated me. Now what?');
    }
}

This way we are “booking” the constructor and at the same time, preventing customers from using it unless we decide to make it global in a future version. This would work as long as we keep the restriction of not being able to instantiate de class with the default constructor from the outside. Otherwise, after releasing the first version with the private constructor, an update that changes it into a global one would obviously need an update in existing customer code to start leveraging it. We could think of an alternative solution that would work without them needing to update nothing but our package.

 

Implementing an explicit prohibition

Let’s create an explicit, global no-param constructor that throws a special exception:

global class InstantiateMeIfYouCan {
    global InstantiateMeIfYouCan() {
        throw new InvalidInstantiationException();
    }

    global InstantiateMeIfYouCan(Boolean coolModeActivated) {
        ...
    }

    global void foo() {
        System.assert(false, 'OK, so you instantiated me. Now what?');
    }
}

By including the global constructor in the first version, we make sure it will be available in future releases if we need it. On the other hand, we are explicitly forbidding its usage by throwing an exception that we have created for this situation. And if we need to “unlock” the constructor and make it available in a new release, we can always change its implementation with no issues. Actually, the customer would need to update their code anyway in order to use the new constructor, unless they implemented something like this to anticipate this situation:

public class ISwearICanInstantiateYou {
    public static void foo() {
        VSTest.InstantiateMeIfYouCan instance;

        try {    // Firstly, try to instantiate with default values
            instance = new VSTest.InstantiateMeIfYouCan();
        }
        catch (VSTest.InvalidInstantiationException ex) {
            // Not available: use the constructor with an explicit argument
            instance = new VSTest.InstantiateMeIfYouCan(false);
        }

        instance.foo();
    }
} 

So now, there’s no need at all for the customer to create a new version of their code or product, because it will be prepared to work with and without the no-param constructor in advance.