Load Dataset Step

This type of step loads a system dataset or a dataset that is in the Datasets catalog.

JSON Request

The following template shows the body of a JSON request that could be used in a Load Dataset Step PUT or PATCH request. It lists all of the step's required and optional body parameters but excludes the read-only options. The default values for each parameter are shown. Below the request (in Schema Details) is a table that describes the complete schema, including the read-only parameters. Clicking a link in the template takes you to the schema details for that parameter.

{ "skipEsLoadIfIndexExists" : true, "edition" : "string", "gmLinkedDataset" : "string", "watchFldsDirectory" : true, "maskedPredicate" : [ "string" ], "fldsComponent" : [ "string" ], "transformQuery" : "string", "title" : "string", "type" : "LoadDatasetStep", "incrementalData" : [ "string" ], "enabled" : true, "contextProvider" : [ "string" ], "description" : "string", "ontology" : [ "string" ], "ignoreLoadErrors" : true, "disableLoadCounts" : true, "preGenerateStatistics" : true, "tags" : [ { "description" : "string", "title" : "string" } ], "tagTitle" : [ "string" ] }

Schema Details

The table below describes the Load Dataset Step schema.

You can also see the Load Dataset Step schema by expanding Schemas at the bottom of the Graph Studio REST API document and viewing LoadDatasetStep.

Property	Format	Required?	Description
uri (read-only)	"uri"	Auto-generated	The URI of the step.
creator (read-only)	"uri"	Auto-generated	The creator of the step.
created (read-only)	"dateTime"	Auto-generated	The timestamp when the step was created.
modifier (read-only)	"uri"	Auto-generated	The user who modified the step.
alltypes (read-only)	Array of strings	Optional	A list of the types related to the step, such as LoadDatasetStep, Step, LayerChild, etc.
contextAttribute (read-only)	Array of strings	Optional	A list of any context attributes that are used.
skipEsLoadIfIndexExists	boolean	Optional	This option applies to graphmarts with Elasticsearch indexes. It controls whether or not Graph Studio first checks to see if an index with the alias for the dataset already exists in Elasticsearch. If this setting is true and the index does exist, Graph Studio will not reload the index snaphsot into Elasticsearch.
edition	"uri"	Optional	If you want to load a certain edition in the dataset, you can include this property to specify the URI of the edition.
gmLinkedDataset	"uri"	Optional	The URI of the Linked Dataset Catalog entry that represents the target FLDS for the load. To get the catalog entry, you can retrieve data about the dataset and use the catalogEntry value.
watchFldsDirectory	boolean	Optional	Controls whether the FLDS directory is monitored for changes. If true and files change, Graph Studio marks the step (and layer) as needing a refresh.
maskedPredicate	[ "uri", "..." ]	Optional	To exclude certain triples from the load, you can specify a list of predicate URIs to filter out.
fldsComponent	[ "uri", "..." ]	Optional	If you want to load specific components in the dataset, you can include this property to list the component URIs.
transformQuery	"string"	Optional	If you want to hand-pick the data to load, you can include this property to run a SPARQL query that inserts specific values or filters out certain values.
title	"string"	Required	The name of the step.
type	"string"	Required	The type of step: "LoadDatasetStep".
incrementalData	"string"	Optional	Incremental load data associated with the step.
enabled	boolean	Optional	Controls whether the step is enabled or disabled.
contextProvider	[ "uri", "..." ]	Optional	A list of any referenced context providers (the data source URI). You can retrieve data for the parent layer to get a list of providers for that layer.
description	"string"	Optional	A brief description of the step.
ontology	[ "uri", "..." ]	Optional	A list of any models to associate with this step.
source	[ "uri", "..." ]	Optional
For Load Dataset Steps, this value must be http://cambridgesemantics.com/ontologies/Graphmarts#Self.

| | ignoreLoadErrors | boolean | Optional | Controls whether to ignore errors and proceed with the load or fail the step if there is an error. | | disableLoadCounts | boolean | Optional | Controls whether Graph Studio periodically queries Graph Lakehouse to count the total number of statements that are processed. Disabling the load count decreases the number of queries that run during activation. | | preGenerateStatistics | boolean | Optional | Controls whether Graph Lakehouse generates statistics on the data before the step is run. | | tags | Array of objects | Optional | Any tags on the step. | | tagTitle | ["string"] | Optional | A virtual property that is available for all objects. It lists the tags associated with the step or can be used to add a tag to the step without including a description. |