Auditing

NB: Above diagram is a first draft. Please comment if it needs amending.

All necessary actions on the system must be audited. If there are events that users need to query or report on they will do that from the Audit Service. The audit action must also happen asynchronously.

Example Actions That Will be Audited

1. Administrator creates a role
2. Administrator updates a role
3. Administrator deletes a role
4. A call is created (Action is "Create Call")
5. Administrator Deletes a Call (Action is "Delete Call")
6. Administrator moves a call from Local Storage to Local Storage (Action is "Admin Move Call")
7. The System moves a call from Local Storage to Local Storage (Action is "Move Call")
8. The System moves a call from AWS S3 to Azure Hot Storage (Action is "Move Call to Hot")
9. The System moves a call from Azure Hot Storage to Azure Cold Storage (Action is "Move Call to Cold")
10. The System moves a call from Azure Cold Storage to Azure Hot Storage (Action is "Move Call from Cold")
11. System User Suppresses audio for their call (Action is "Suppress Audio")
12. System User performs a search on call metadata (Action is "Search Metadata")
13. System User performs a search on a call transcription (Action is "Search Transcription")

Example Queries that Will be Searched 

1. Show me all the actions that User Kirill performed.
2. Show me all the users that searched for (transcription.BeginsWith="Hello")
3. Show me all the users that were disabled between January and February 2018
4. Show me when a call was moved to cheaper storage.

DB Schema

Actions

As parts of the system have already been developed, once the architecture has been finalised, the BA team will create stories for all of the past actions to be audited. The search functionality may not be delivered as part of September MVP.

Example:

User Story

As a developer
I want to audit that a role has been created
So that it meets the criteria for actions that require auditing

Acceptance Criteria

1. Appropriate response is returned if action was audited
2. Appropriate response is returned if action was not audited
3. Audit data includes:
   1. Timestamp - The date and time when the action was carried out. Please note that as auditing is asynchronous this is not the time of the audit.
   2. TenantID - This is the Tenant ID of the Tenant that created the role.
   3. User - The GUID of the User who is creating the role
   4. Action - "Create Role" (Or do we use the Event Store event?)
   5. ObjectId - The GUID of the new role that is created
   6. CausationEventId - This is the identifier of the actual event in EventStore
   7. Detail - The name of the new role that is being created
   8. Status - The result of the Action. 1 = Success; 2= Failure
   9. Failure Detail 
      1. Insufficient Permissions
      2. Role already exists
      3. Invalid characters in Role name

Plan

Given the business requirements and suggested approach I think we overestimated this feature. In my view, estimation should be reduced to M/M+

1. Audit service. Service responsible for reading Audit stream and creating it's projection in Elastic Search
   1. AuditSubscriber - service itself
   2. Event Store projection that will copy events to the audit streams
   3. DevOps: start Event Store with custom projections
2. Audit API. Basic api to retrieve audit events
3. Extend commands event metadata and put there fields needed for the audit (initiator, entityID, etc)
4.  Generate audit events for auth operations (login/logout, permission validation failures on API Gateway, etc)

This plan doesn't include UI for the audit, I guess it can be moved to a separate feature and estimated to something like M itself since would require a lot of mappings and parsing depending on the event type

Event list to be tracked

• CreateDevice (Action is "")
• UpdateDevice (Action is "")
• DeleteDevice (Action is "")

• CreateAgent (Action is "")
• UpdateAgent (Action is "")
• DeleteAgent (Action is "")

Collector:

• CreateCollector
• UpdateCollector
• DeleteCollector

• Call created (Action is "Create Call")
• Administrator Deletes a call (Action is "")
• Administrator moves a call from Local Storage to Local Storage (Action is "Admin Move Call")
• The System moves a call from Local Storage to Local Storage (Action is "Move Call")
• The System moves a call from AWS S3 to Azure Hot Storage (Action is "Move Call to Hot")
• The System moves a call from Azure Hot Storage to Azure Cold Storage (Action is "Move Call to Cold")
• The System moves a call from Azure Cold Storage to Azure Hot Storage (Action is "Move Call from Cold")
• CallSuppression (Action is "")
• RecordOnDemand (Action is "")
• System User performs a search on call metadata (Action is "Search Metadata")
• System User performs a search on a call transcription (Action is "Search Transcription")
• Lock call (unable to delete call after that) (Action is "Lock Call")
• Unlock call (able to delete call after that) (Action is "Unlock Call")

• CreateRole (Action is "Create Role")
• UpdateRole (Action is "Update Role")
• DeleteRole (Action is "Delete Role")

• AddPermissionToRole (Action is "")
• RemovePermissionFromRole (Action is "")

• AddLicense (Action is "")
• UpdateLicense (Action is "")

Annotation Header

• Create annotation header
• Update annotation header
• Delete annotation header (disable annotation header)

Annotation 

• Create annotation 
• Update annotation 
• Delete annotation 

Questions

1. James Ratcliffe (Unlicensed), we need a list of all events we need to track in the system