Tutorial: Setup for and use of standard format csv-file to OSCAL Component Definition json-file transformer¤

Here are step by step instructions for setup and transformation of trestle standard format csv-file into OSCAL Component Definition json-file using the compliance-trestle tool.

Objective¤

How to transform trestle standard format csv-file into a component-definition.json file.

There are 2 short steps shown below. The first is a one-time check/set-up of your environment. The second is a one-command transformation from .csv to component-definition.json.

Table: expected .csv content

The below table represents the expectations of trestle task csv-to-oscal-cd for the contents of the input csv-file for synthesis of the output OSCAL Component Definition json-file.

Column Name is the name of the expected column in the input csv-file. Any additional columns not identified here, for example foobar, are also extracted and placed into the output json-file as component.control-implementation.prop["foobar"].

Component Definition Locale is the path within the output json-file into witch the value is stashed.

Column Name	Value Type	Specification	Value Description	Component Definition Locale	Example Value
Rule_Id	String	required	A textual label that uniquely identifies a policy (desired state) that can be used to reference it elsewhere in this or other documents.	component.control-implementation.prop["Rule_Id"]	password_policy_min_length_characters
Rule_Description	String	required	A description of the policy (desired state) including information about its purpose and scope.	component.control-implementation.prop["Rule_Description"]	Ensure password policy requires minimum length of 12 characters
Profile_Reference_URL	String	required	A URL reference to the source catalog or profile for which this component is implementing controls for. A profile designates a selection and configuration of controls from one or more catalogs	component.control-implementation.source	https://github.com/usnistgov/oscal-content/blob/main/nist.gov/SP800-53/rev5/json/NIST_SP-800-53_rev5_HIGH-baseline_profile.json
Profile_Description	String	required	A description of the profile.	component.control-implementation.description	NIST Special Publication 800-53 Revision 5 HIGH IMPACT BASELINE
Component_Type	String	required	A category describing the purpose of the component.	component.type	Validation
Control_Mappings	String List (blank separated)	required	A list of textual labels that uniquely identify the controls or statements that the component implements.	component.control-implementation.implemented-requirement.statement.statement-id and component.control-implementation.implemented-requirement.control-id	ia-5.1_smt.a ia-5.1
Resource	String	required	A human readable name for the component.	component.title	Compliance Center
Parameter_Id	String	optional	A textual label that uniquely identifies the parameter associated with that policy (desired state) or controls implemented by the policy (desired state). A description of the parameter including the purpose and use of the parameter.	component.control-implementation.prop["Parameter_Id"] and component.control-implementation.set-parameter.param-id	minimum_password_length
Parameter_Description	String	optional	A description of the parameter including the purpose and use of the parameter.	component.control-implementation.prop["Parameter_Description"]	Minimum Password
Parameter_Default_Value	String	optional	A value recommended in this profile for the parameter of the control or policy (desired state).	component.control-implementation.set-parameter.values	12
Parameter_Value_Alternatives	String List (blank separated)	optional	ONLY for the policy (desired state) parameters: A value or set of values the parameter can take.	component.control-implementation.prop["Parameter_Value_Alternatives"]	12 8
Check_Id	String	optional	A textual label that uniquely identifies a check of the policy (desired state) that can be used to reference it elsewhere in this or other documents.	component.control-implementation.prop["Check_Id"]	check_password_policy_min_length_characters
Check_Description	String	optional	A description of the check of the policy (desired state) including the method (interview or examine or test) and procedure details.	component.control-implementation.prop["Check_Description"]	Check whether password policy requires minimum length of 12 characters
Fetcher	String	optional	A textual label that uniquely identifies a collector of the actual state (evidence) associated with the policy (desired state) that can be used to reference it elsewhere in this or other documents.	component.control-implementation.prop["Fetcher"]	fetch_password_policy_min_length_characters
Fetcher_Description	String	optional	A description of the collector of the actual state (evidence) associated with the policy (desired state) including the method (interview or examine or API) and questionaire	component.control-implementation.prop["Fetcher_Description"]	Fetch whether password policy requires minimum length of 12 characters
Resource_Instance_Type	String	optional	A textual label that uniquely identifies a resource (component) type from the resource instance id. This text is part of all instance ids of a particular resource at runtime. For example the text 'db2' is part of all instance ids of resource DB2.	component.control-implementation.prop["Resource_Instance_Type"]	DB2

Step 1: Install trestle in a Python virtual environment¤

Follow the instructions here to install trestle in a virtual environment.

Step 2: Transform profile data (CIS benchmarks)¤

Linux, Mac

Windows

Make these changes:

use backslashes `\` for file paths
use `md` instead of mkdir -p
put the url in double quotes for `curl`
use `more` instead of cat

Navigate to trestle workspace.

(venv.trestle)$ cd trestle.workspace

View configuration information.

(venv.trestle)$ trestle task csv-to-oscal-cd -i
trestle.core.commands.task:101 WARNING: Config file was not configured with the appropriate section for the task: "[task.csv-to-oscal-cd]"
Help information for csv-to-oscal-cd task.

Help information for csv-to-oscal-cd task.

Purpose: From csv produce OSCAL component_definition file.


Configuration flags sit under [task.csv-to-oscal-cd]:
  title                = (required) the component definition title.
  version              = (required) the component definition version.
  csv-file             = (required) the path of the csv file. [1st row are column headings; 2nd row are column descriptions; 3rd row and beyond is data]
  required columns:      $$Component_Title
                         $$Component_Description
                         $$Component_Type
                         $$Rule_Id
                         $$Rule_Description (see note 1)
                         $$Profile_Source (see note 1)
                         $$Profile_Description (see note 1)
                         $$Control_Id_List (see note 1)
                         $$Namespace
  optional columns:      $Check_Id (see note 2)
                         $Check_Description (see note 2)
                         $Target_Component (see note 3)
                         $Original_Risk_Rating (see note 1)
                         $Adjusted_Risk_Rating (see note 1)
                         $Risk_Adjustment (see note 1)
                         $Parameter_Id (see notes 1, 5)
                         $Parameter_Description (see notes 1, 5)
                         $Parameter_Value_Alternatives (see notes 1, 5)
  comment columns:       #Informational (see note 4)
  output-dir           = (required) the path of the output directory for synthesized OSCAL .json files.
  component-definition = (optional) the path of the existing component-definition OSCAL .json file.
  class.column-name    = (optional) the class to associate with the specified column name, e.g. class.Rule_Id = scc_class
  output-overwrite     = (optional) true [default] or false; replace existing output when true.
  validate-controls    = (optional) on, warn, or off [default]; validate controls exist in resolved profile.

Notes: [1] column is ignored for validation component type
       [2] column is required for validation component type
       [3] column is optional for validation component type, but may be needed to prevent Rule_Id collisions
       [4] column name starting with # causes column to be ignored
       [5] additional parameters are specified by adding a common suffix per set, for example: Parameter_Id_1, Parameter_Description_1, ...Parameter_Id_2...

Create data folder.

(venv.trestle)$ mkdir -p adjunct-data

Fetch sample csv-file.

(venv.trestle)$ curl 'https://raw.githubusercontent.com/oscal-compass/compliance-trestle/main/docs/tutorials/Transformers_and_Tasks/csv_to_oscal_cd/ocp4-sample-input.csv' > adjunct-data/ocp4-sample-input.csv

Fetch trestle task file.

(venv.trestle)$ curl 'https://raw.githubusercontent.com/oscal-compass/compliance-trestle/main/docs/tutorials/Transformers_and_Tasks/csv_to_oscal_cd/demo-csv-to-oscal-cd.config' > adjunct-data/task-files/demo-csv-to-oscal-cd.config

demo-csv-to-oscal-cd.config

[task.csv-to-oscal-cd]

csv-file = adjunct-data/ocp4-sample-input.csv
output-dir = component-definitions/ocp4-sample
title = ocp4-sample
version = 1.0

Perform and validate the transform.

(venv.trestle)$ trestle task csv-to-oscal-cd -c demo-csv-to-oscal-cd.config 
input: adjunct-data/ocp4-sample-input.csv
output: component-definitions/ocp4-sample/component-definition.json
Task: csv-to-oscal-cd executed successfully.

View the generated OSCAL.

(venv.trestle)$ component-definitions/ocp4-sample/component-definition.json

component-definition.json

{
  "component-definition": {
    "uuid": "83cc8984-b00a-4799-885c-60b689efebd0",
    "metadata": {
      "title": "ocp4-sample",
      "last-modified": "2022-11-18T17:06:49+00:00",
      "version": "1.0",
      "oscal-version": "1.0.2"
    },
    "components": [
      {
        "uuid": "c0080494-186a-421d-9afd-f51e0359cbd8",
        "type": "Service",
        "title": "OSCO",
        "description": "",
        "control-implementations": [
          {
            "uuid": "43a69f86-a3ad-40fa-ada6-2f988b951728",
            "source": "https://github.com/ComplianceAsCode/content/blob/master/products/ocp4/profiles/cis.profile",
            "description": "ocp4",
            "props": [
              {
                "name": "Rule_Id",
                "value": "content_rule_api_server_anonymous_auth",
                "remarks": "rule_set_0"
              },
              {
                "name": "Rule_Description",
                "value": "Ensure that the --anonymous-auth argument is set to false",
                "remarks": "rule_set_0"
              },
              {
                "name": "Check_Id",
                "value": "xccdf_org.ssgproject.content_rule_api_server_anonymous_auth",
                "remarks": "rule_set_0"
              },
              {
                "name": "Check_Description",
                "value": "Ensure that the --anonymous-auth argument is set to false",
                "remarks": "rule_set_0"
              },
              {
                "name": "Rule_Id",
                "value": "content_rule_api_server_basic_auth",
                "remarks": "rule_set_1"
              },
              {
                "name": "Rule_Description",
                "value": "Ensure that the --basic-auth-file argument is not set",
                "remarks": "rule_set_1"
              },
              {
                "name": "Check_Id",
                "value": "xccdf_org.ssgproject.content_rule_api_server_basic_auth",
                "remarks": "rule_set_1"
              },
              {
                "name": "Check_Description",
                "value": "Ensure that the --basic-auth-file argument is not set",
                "remarks": "rule_set_1"
              },
              {
                "name": "Rule_Id",
                "value": "content_rule_api_server_token_auth",
                "remarks": "rule_set_2"
              },
              {
                "name": "Rule_Description",
                "value": "Ensure that the --token-auth-file parameter is not set",
                "remarks": "rule_set_2"
              },
              {
                "name": "Check_Id",
                "value": "xccdf_org.ssgproject.content_rule_api_server_token_auth",
                "remarks": "rule_set_2"
              },
              {
                "name": "Check_Description",
                "value": "Ensure that the --token-auth-file parameter is not set",
                "remarks": "rule_set_2"
              },
              {
                "name": "Rule_Id",
                "value": "content_rule_api_server_https_for_kubelet_conn",
                "remarks": "rule_set_3"
              },
              {
                "name": "Rule_Description",
                "value": "Ensure that the --kubelet-https argument is set to true",
                "remarks": "rule_set_3"
              },
              {
                "name": "Check_Id",
                "value": "xccdf_org.ssgproject.content_rule_api_server_https_for_kubelet_conn",
                "remarks": "rule_set_3"
              },
              {
                "name": "Check_Description",
                "value": "Ensure that the --kubelet-https argument is set to true",
                "remarks": "rule_set_3"
              }
            ],
            "implemented-requirements": [
              {
                "uuid": "c2893d38-1be4-4b0e-a090-96e846e15a3b",
                "control-id": "CIS-1.2.1",
                "description": "",
                "props": [
                  {
                    "name": "Rule_Id",
                    "value": "content_rule_api_server_anonymous_auth"
                  }
                ]
              },
              {
                "uuid": "3c2f7129-9724-47c0-aadb-3b3c9c44995c",
                "control-id": "CIS-1.2.2",
                "description": "",
                "props": [
                  {
                    "name": "Rule_Id",
                    "value": "content_rule_api_server_basic_auth"
                  }
                ]
              },
              {
                "uuid": "a4e2862f-7a1b-4182-b827-f5e797f589db",
                "control-id": "CIS-1.2.3",
                "description": "",
                "props": [
                  {
                    "name": "Rule_Id",
                    "value": "content_rule_api_server_token_auth"
                  }
                ]
              },
              {
                "uuid": "daec13ab-829e-4dd6-a9d6-9ad18391681e",
                "control-id": "CIS-1.2.4",
                "description": "",
                "props": [
                  {
                    "name": "Rule_Id",
                    "value": "content_rule_api_server_https_for_kubelet_conn"
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  }
}

Congratulations! You have completed this tutorial.

Examples: csv files suitable for csv-to-oscsl-cd transformation¤

The examples given here comprise csv files that can be transformed into OSCAL Component Definitions. The csv files comprise required headings and values expected by the trestle task transformer, and may present optional ones as well.

OCP4 sample input

This is a simple example csv that has just one rule per control and one check per rule: ocp4-sample-input.csv

Multiple occurrence sample input

This is a simple example csv that has multiple checks per rule and multiple target-components per rule: rule-name-overlap.csv