Workspace documentation

Workspace documentation standards

The main purposes of workspace documentation are:

• handover of knowledge about the ETL jobs running and its' maintenance to a client,
• keeping track of important decisions made during the implementation,
• guarantee an easy maintenance by internal implementation team in the future.

Each workspace of any project should have documentation in the "Description" field. An exception can be made only for CPS or Monitoring Check workspaces.

Workspace documentation should cover:

1. The purpose of the workspace, data for which sources and events is processed and important business assumptions made for this ETL. 
2. Structure of the events' payloads which are processed in this workspace.
3. Pipeline details:
   • Name and short description of each configuration,
   • Important assumptions specific for this particular configuration (if any).
4. Schedule - daily/weekly/one time load and the reason if it is not obvious.
5. Workspace variables description, if the nature of the variables is not obvious (ex. database credentials).
6. Maintenance:
   
   • How to handle backloads,
   • Possible known reasons of failing and how to fix these,
   • etc.

Documentation review should be done by the Analyst responsible for a project review. An Analyst working on the project should prepare the workspace documentation after finalising the Workspace (or in the process of preparing it) before the workspace review.

Remember: Workspace documentation is a prerequisite for the workspace review. Responsibility of a reviewing Analyst is to check not only workspace logic and code, but its documentation as well.

Template

Workspace documentation template can be found here.

Example

A few examples of workspace documentation are below: