Hello Josh Hepworth (Unlicensed), James Denning (Unlicensed)

I’m not sure who should approve this decision, so just wanted to let you know.

Konstantine, Jayasri and myself had an extended conversation about writing automation tests for US related to the Identity Server API and CQRS pattern (basically US 1241 and 1242).

Initially we’ve planned to write tests separately for api and subscriber (for api check that event is placed in the event store, and for subscriber that is can read events from the event store and put data into local sql lite db).

We think that it’s better to write automation tests from the end user prospective, meaning that we need just to check that Identity Server API works correctly and don’t check the logic under the hood (it should be tested by BDD tests written by devs). Moreover, starting from the last refinement session, we are not creating a separate user story per each component, but one for the whole flow (see US1590 as an example).

So instead of having following test cases:

Send POST request to Identity Server API to create role
Check that event is stored in the Event Store
Check that Identity Subscriber can read event placed in the event store
Check that role is stored in the local sql lite data base

We would have

Send POST request to Identity Server API
Get the response and validate the result
Check that role can be retrieved from the Identity Server Read API (once we have it in place – US 1790)

Also it would simplify running automation tests on the VM environment a lot since we have one endpoint to check and don't need to figure out how to get access to internal container structure.

Here is a high level diagram that shows the identity entities flow, so the main idea is to test only endpoints (Identity API, Identity Read API, etc)

Please let us know what you think

Added by Josh

Approach A ‘Discrete System Interactions’	Approach B - ‘User Interactions’
Separated, discrete tests for each interation. Testing involves checking underlying components (event store, sql lite db, elastic search). Advantages: Complete verification of underlying systems Dis-Advantages: Time consuming to write Brittle, in that is tied to underlying implementations Doesn’t reflect complete user interaction Might miss important end-to-end integration verification. (E.g. test that api writes to event store, subscriber reads from it, but don't check that the respose is put with the correct corelation id)	Tests are written from the end-user perspective Advantages: Simplifies running of the automation tests (no need for access to underlying components) Tests don't take as long as 'discrete system interactions' approach. More robust to underlying changes in the system Dis-Advantages: Relies heavily on the BDD tests written by developers

Approach A ‘Discrete System Interactions’

Approach B - ‘User Interactions’

Separated, discrete tests for each interation. Testing involves checking underlying components (event store, sql lite db, elastic search).

Advantages:

Complete verification of underlying systems

Dis-Advantages:

Time consuming to write
Brittle, in that is tied to underlying implementations
Doesn’t reflect complete user interaction
Might miss important end-to-end integration verification. (E.g. test that api writes to event store, subscriber reads from it, but don't check that the respose is put with the correct corelation id)

Tests are written from the end-user perspective

Advantages:

Simplifies running of the automation tests (no need for access to underlying components)
Tests don't take as long as 'discrete system interactions' approach.
More robust to underlying changes in the system

Dis-Advantages:

Relies heavily on the BDD tests written by developers