Plugin Pipeline Simulation in FakeXrmEasy v2/3

Pipeline simulation has been a feature of FakeXrmEasy for several years. There have been some significant improvements in recent updates to versions 2 & 3 which make the feature even more useful. In this article I'll go through the new features and discuss the use of Pipeline Simulation in general.

As well as some minor improvements, Versions 2 & 3 of FakeXrmEasy include three very useful new features.

Automatic Plugin Step Registration
The first new feature is the addition of Automatic Plugin Step Registration. Previously, when using Pipeline Simulation, you had to explicitly register the Plugin Steps as part of the setup of your tests, like this:

Not only was this time-consuming, but it was also error-prone. Tests might end up failing becaue of invalid plugin registration, rather than because of genuine errors. With Automatic Plugin Step Registration, if you use a declarative method of registering your plugins steps with a tool like spkl, your plugin steps will automatically be registered in your tests. This is a great time-saver and will help you avoid making mistakes in registering plugin steps in your tests.
There's a sample solution showing how to get this working with spkl here and more documentation here.

Plugin Step Audit
The second new feature is called Plugin Step Audit. With this enabled (by using the UsePluginStepAudit pipeline option in the AddPipelineSimulation directive), FakeXrmEasy will keep a record of the Plugin steps which fire during your tests, and this can be used to verify that you have set your plugin registration up correctly, and that the pipeline is working as expected. This could be used, for example, to test that you have configured your filtering attributes correctly in your step registration. The advantage of using this feature is that you can test that your steps registration is configured correctly without having to set up test data in order to test it - if you only need to know that a plugin fires under certain circumstances, without needing to know what the result of the plugin firing is, then you can test this with plugin step auditing. I'll talk later about the principle that Unit Tests should only test one thing and this feature allows you to do exactly that.

Plugin Registration Validation
Additionally, the Automated Plugin Registration feature validates your plugin registration and prevents you from defining registrations which would not be valid when registering message processing steps against a real instance of CRM.
For example, if your plugin registration config contains two steps with the same GUID, you will get an error, or if you try and register a plugin using a configuration which you would not be able to do against a real Dynamics instance, such as registering a pre-image on a Create step.

Having explored the new features available in V2/3, I'd like to discuss the use of Pipeline Simulation in general, what its pros and cons are, and when I think you should use it.

The advantages of using Pipeline Simulation
I think there are three main advantages to using Pipeline Simulation when testing your Dynamics Code.

Allows for end-to-end testing of code
Plugin pipeline simulation allows you to perform full end-to-end testing of your transactions. So, if you have a plugin which creates or updates an entity, and this entity also has a plugin registered against it, you will be able to test the full end-to-end process. This is particularly useful when you have some code which is not called from the context of a plugin transaction, but, for example, an Azure Function. If your function creates a record and a plugin populates some attributes on that record which your Azure Function relies on, then pipeline simulation will allow you to test this. With pipeline simulation, the faked context behaves just as the real CRM instance would behave.

Troubleshoot the whole pipeline
Related to the above feature is the fact that with pipeline simulation its possible to troubleshoot and debug the entire transaction pipeline. Imagine for example that an Azure function which creates a record in CRM and then does something with that record, and that Azure function is not working as expected. With pipeline simulation, you can trouble shoot the entire function from start to finish and step into the plugin code even if it is called from the Azure Function. There is a good video demonstrating this here.

Testing plugin registration
This is another real bonus - if you use a declarative method such as spkl to register your plugins, you can test that your have registered your message processing steps correctly before you even deploy them to the database. You can check filtering attributes and images are defined correctly and, as mentioned above, the framework will even test whether your registration does anything 'illegal' like registering two steps with the same GUID.

Ease of use
Introducing newcomers to the practice of Unit Testing in general, and specifically when talking about the creation of fake plugin execution contexts, can be confusing if the developers are new to the concepts of Unit Testing and Fakes. Being able to demonstrate a Unit test which just consists of setting up some test data and then issuing a 'Create' message and testing the outcome is simpler than having to demonstrate the use of a fake Plugin Execution Context, registering input parameters, etc. Using Pipeline Simulation can be a good first step in the learning curve which comes with writing dataverse Unit Tests.

Why use anything else?
If using Pipeline Simulation in our Unit Tests is so good, should we always use it? Should we stop writing tests which invoke the plugin directly and simply call the CRUD or other messages which will cause the pipeline to fire? Why go through the complexity of writing Unit Tests which explicitly test our plugins when we could 'just' test our plugins by executing a service request and letting the plugin pipeline simulation execute the plugin(s) for us?

Well, the truth is, that when we use pipeline simulation, we are no longer truly Unit Testing our code - testing the entire execution pipeline is really a kind of integration testing, albeit with tests that use a fake execution context and with many significant advantages (speed, repeatability, etc) over integration testing with a real instance of CRM. Strictly speaking a Unit Test should only test one thing, and if our tests invoke a whole plugin pipeline, we can no longer say that we are just testing a single function.

Does this matter? Does the fact that we can no longer call these tests Unit Tests make any difference? Well, yes, in many cases it does. By simulating the entire plugin execution pipeline we are introducting fragility and dependencies in our tests. If a test fails, we can't be sure whether this is because the code under test is failing or because some other part of the pipeline is failing because it depends on certain data or certain conditions being true. Testing a pipeline plugin may well require more setup before each test and as the plugin pipeline grows in complexity so will the dependency on data which is not relevant to the test in question, but still required to allow the pipeline to run to completion without error. Tests which rely on the plugin pipeline execution are more likely to start failing some time after they have been written because changes to any of the individual plugins called within the plugin pipeline introduce further dependencies.

This is not to say that we shouldn't use pipeline simulation, just that we should think about the reasons why we are using it and should only use it when our tests require the entire pipeline to run, such as when testing calls from Azure Functions which require the plugin pipeline to execute. We can consider there to be three levels of plugin testing which can be carried out using FakeXrmEasy:

• Method
• Plugin
• Pipeline
  
  

Ideally, the code in your plugins should be broken into discrete methods for each function within the plugin. True Unit Testing will be done at this level - a method is called with certain parameters and a single outcome of that method call can be tested.

Sometimes we want to be able to test the execution of an entire plugin, not just the individual methods within it. We might want to test how the plugin reacts when called with different parameters, such as different messages, targets or images. Or what happens when a pluginn which contains multiple methods is called.

Finally, we may sometimes need to invoke the entire plugin pipeline. This should not so much be because we want to test the entire pipeline, although we may sometimes want to do this, but more because the code we are testing relies on the plugin pipeline executing because (for example) it expects a certain record to have been created or updated during the pipeline execution which is required later in the code being tested. It's also really useful to be able to test that your plugin step registration has been configured correctly.

As a developer, it's really up to you to make a judgement call as to which level of testing you will use and you will most likely find that you will use a mixture of all three methods to acheive a good level of code coverage and robustness in your tests.

I'd recommend watching the excellent videos explaining Pipeline Simulation on the FakeXrmEasy youtube channel.

In conclusion, I think that pipeline simulation, especially when used in conjunction with automatic plugin registration using a tool like spkl, is a really useful feature of FakeXrmEasy. I don't see it as a replacement for the other methods of testing plugin code, but an invaluable tool when you need to be able to test more than just the code in a specific plugin. It brings us ever closer to the ideal of being able to fully test our Dataverse code locally before we deploy it to a real Dynamics instance.