Deploy and management

This section outlines the various ways to set up, configure and initialize a Toolkit Setup. There are two fundamental approaches to running Toolkit: A traditional project based setup and a ToolkitManager API that allows for more flexible manipulation of configurations and installations.

Traditional project setup

The traditional Toolkit ‘pipeline approach’ means that you pick an existing Shotgun project and run a Toolkit project setup for this project. This is typically done in Shotgun Desktop or via the tank setup_project command. A Toolkit configuration is installed in a shared location on disk and a pipeline configuration in Shotgun is created to create an association between the installation on disk and the project in Shotgun.

Once the installation has completed, you can access functionality via the tank command for that project or run sgtk.sgtk_from_entity() or sgtk.sgtk_from_path() in order to create an API session.

Bootstrapping Toolkit

An alternative to the traditional project based setup was introduced in Core v0.18 - the ToolkitManager class allows for more flexible manipulation of toolkit setups and removes the traditional step of a project setup. Instead, you can launch an engine straight directly based on a Toolkit configuration. The manager encapsulates the deploy and configuration management process and makes it easy to create a running instance of Toolkit. It allows for several advanced use cases:

  • Bootstrapping via the Toolkit manager does not require anything to be set up or configured in Shotgun. No extensive project setup step is required.
  • A setup can be pre-bundled with for example an application plugin, allowing Toolkit to act as a distribution platform.
  • The application bundles that are required can be stored anywhere on the local machine or the network via the use of the SHOTGUN_BUNDLE_CACHE_FALLBACK_PATHS environment variable.
  • The Toolkit manager makes it easy to track remote resources (via the sgtk.descriptor framework).

The following example code can for example run inside maya in order to launch Toolkit’s default config for a given Shotgun Asset:

import sgtk

# Start up a Toolkit Manager
mgr = sgtk.bootstrap.ToolkitManager()

# Set the base configuration to the default config
# note that the version token is not specified
# The bootstrap will always try to use the latest version
mgr.base_configuration = "sgtk:descriptor:app_store?name=tk-config-default"

# now start up the maya engine for a given Shotgun object
e = mgr.bootstrap_engine("tk-maya", entity={"type": "Asset", "id": 1234})

Note that the example is primitive and for example purposes only as it will take time to execute and blocks execution during this period.

In this example, there is no need to construct any sgtk.Sgtk instance or run a tank command - the ToolkitManager instead becomes the entry point into the system. It will handle the setup and initialization of the configuration behind the scenes and start up a Toolkit session once all the required pieces have been initialized and set up.

ToolkitManager

class sgtk.bootstrap.ToolkitManager(sg_user=None)[source]

This class allows for flexible and non-obtrusive management of toolkit configurations and installations.

Parameters:sg_user (ShotgunUser) – Authenticated Shotgun User object. If you pass in None, the manager will provide a standard authentication for you via the shotgun authentication module and prompting the user if necessary. If you have special requirements around authentication, simply construct an explicit user object and pass it in.
extract_settings()[source]

Serializes settings that impact resolution of a pipeline configuration into an object and returns it to the user.

This can be useful when a process is used to enumerate pipeline configurations and another process will be bootstrapping an engine. Calling this method ensures the manager is configured the same across processes.

Those settings can be restored with ToolkitManager.restore_settings().

Note

Note that the extracted settings should be treated as opaque data and not something that should be manipulated. Their content can be changed at any time.

Returns:User defined values.
Return type:object
restore_settings(data)[source]

Restores user defined with :methd:`ToolkitManager.extract_settings`.

Note

Always use :methd:`ToolkitManager.extract_settings` to extract settings when you plan on calling this method. The content of the settings should be treated as opaque data.

Parameters:data (object) – Settings obtained from :methd:`ToolkitManager.extract_settings`
allow_config_overrides

Whether pipeline configuration resolution can be overridden via the environment. Defaults to True on manager instantiation.

pre_engine_start_callback

This callback will be invoked after the Toolkit instance has been created but before the engine is started.

This function should have the following signature:

def pre_engine_start_callback(ctx):
    '''
    Called before the engine is started.

    :param :class:"~sgtk.Context" ctx: Context into
        which the engine will be launched. This can also be used
        to access the Toolkit instance.
    '''
pipeline_configuration

The pipeline configuration that should be operated on.

By default, this value is set to None, indicating to the Manager that it should attempt to automatically find the most suitable pipeline configuration entry in Shotgun given the project and plugin id. In this case, it will look at all pipeline configurations stored in Shotgun associated with the project who are associated with the current user. If no user-tagged pipeline configuration exists, it will look for the primary configuration, and in case this is not found, it will fall back on the base_configuration(). If you don’t want this check to be carried out in Shotgun, please set do_shotgun_config_lookup() to False.

Alternatively, you can set this to a specific pipeline configuration. In that case, the Manager will look for a pipeline configuration that matches that name or id and the associated project and plugin id. If such a config cannot be found in Shotgun, it falls back on the base_configuration().

do_shotgun_config_lookup

Flag to indicate if the bootstrap process should connect to Shotgun and attempt to resolve a config. Defaults to True.

If True, the bootstrap process will connect to Shotgun as part of the startup, look for a pipeline configuration and attempt to resolve a toolkit environment to bootstrap into via the Pipeline configuration data. Failing this, it will fall back on the base_configuration().

If False, no Shotgun lookup will happen. Instead, whatever config is defined via base_configuration() will always be used.

plugin_id

The Plugin Id is a string that defines the scope of the bootstrap operation.

If you are bootstrapping into an entire Toolkit pipeline, e.g a traditional Toolkit setup, this should be left at its default None value.

If you are writing a plugin that is intended to run side by side with other plugins in your target environment, the entry point will be used to define a scope and sandbox in which your plugin will execute.

When constructing a plugin id for an integration the following should be considered:

  • Plugin Ids should uniquely identify the plugin.
  • The name should be short and descriptive.

We recommend a Plugin Id naming convention of service.dcc, for example:

  • A review plugin running inside RV: review.rv.
  • A basic set of pipeline tools running inside of Nuke: basic.nuke
  • A plugin containg a suite of motion capture tools for maya: mocap.maya

Please make sure that your Plugin Id is unique, explicit and short.

base_configuration

The descriptor (string or dict) for the configuration that should be used as a base fallback to be used whenever runtime and shotgun configuration resolution doesn’t resolve an override configuration to use.

bundle_cache_fallback_paths

Specifies a list of fallback paths where toolkit will go look for cached bundles in case a bundle isn’t found in the primary app cache.

This is useful if you want to distribute a pre-baked package, containing all the app version that a user needs. This avoids downloading anything from the app store or other sources.

Any missing bundles will be downloaded and cached into the primary bundle cache.

caching_policy

Specifies the config caching policy to use when bootstrapping.

ToolkitManager.CACHE_SPARSE will make the manager download and cache the sole config dependencies needed to run the engine being started. This is the default caching policy.

ToolkitManager.CACHE_FULL will make the manager download and cache all the config dependencies.

progress_callback

Callback function property to call whenever progress of the bootstrap should be reported back.

This function should have the following signature:

def progress_callback(progress_value, message):
    '''
    Called whenever toolkit reports progress.

    :param progress_value: The current progress value as float number.
                           values will be reported in incremental order
                           and always in the range 0.0 to 1.0
    :param message:        Progress message string
    '''
bootstrap_engine(engine_name, entity=None)[source]

Create an Sgtk instance for the given engine and entity, then launch into the given engine.

The whole engine bootstrap logic will be executed synchronously in the main application thread.

If entity is None, the method will bootstrap into the site config. This method will attempt to resolve the config according to business logic set in the associated resolver class and based on this launch a configuration. This may involve downloading new apps from the toolkit app store and installing files on disk.

Please note that the API version of the tk instance that hosts the engine may not be the same as the API version that was executed during the bootstrap.

Parameters:
  • engine_name – Name of engine to launch (e.g. tk-nuke).
  • entity (Dictionary with keys type and id, or None for the site.) – Shotgun entity to launch engine for.
Returns:

Engine instance.

bootstrap_engine_async(engine_name, entity=None, completed_callback=None, failed_callback=None)[source]

Create an Sgtk instance for the given engine and entity, then launch into the given engine.

This method launches the bootstrap process and returns immediately. The Sgtk instance will be bootstrapped asynchronously in a background thread, followed by launching the engine synchronously in the main application thread. This will allow the main application to continue its execution and remain responsive when bootstrapping the toolkit involves downloading files and installing apps from the toolkit app store.

If entity is None, the method will bootstrap into the site config. This method will attempt to resolve the config according to business logic set in the associated resolver class and based on this launch a configuration. This may involve downloading new apps from the toolkit app store and installing files on disk.

Two callback functions can be provided.

A callback function that handles cleanup after successful completion of the bootstrap with the following signature:

def completed_callback(engine):
    '''
    Called by the asynchronous bootstrap upon completion.

    :param engine: Engine instance representing the engine
                   that was launched.
    '''

A callback function that handles cleanup after failed completion of the bootstrap with the following signature:

def failed_callback(phase, exception):
    '''
    Called by the asynchronous bootstrap if an exception is raised.

    :param phase: Indicates in which phase of the bootstrap the exception
                  was raised. An integer constant which is either
                  ToolkitManager.TOOLKIT_BOOTSTRAP_PHASE or
                  ToolkitManager.ENGINE_STARTUP_PHASE. The former if the
                  failure happened while the system was still bootstrapping
                  and the latter if the system had switched over into the
                  Toolkit startup phase. At this point, the running core API
                  instance may have been swapped over to another version than
                  the one that was originally loaded and may need to be reset
                  in an implementation of this callback.

    :param exception: The python exception that was raised.
    '''
Parameters:
  • engine_name – Name of engine to launch (e.g. tk-nuke).
  • entity (Dictionary with keys type and id, or None for the site.) – Shotgun entity to launch engine for.
  • completed_callback – Callback function that handles cleanup after successful completion of the bootstrap.
  • failed_callback – Callback function that handles cleanup after failed completion of the bootstrap.
prepare_engine(engine_name, entity)[source]

Updates and caches a configuration on disk for a given project. The resolution of the pipeline configuration will follow the same rules as the method ToolkitManager.bootstrap_engine(), but it simply caches all the bundles for later use instead of bootstrapping directly into it.

Parameters:
  • engine_name (str) – Name of the engine instance to cache if using sparse caching. If None, all engine instances will be cached.
  • entity – An entity link. If the entity is not a project, the project for that entity will be resolved.
Returns:

Path to the fully realized pipeline configuration on disk and to the descriptor that spawned it.

Return type:

(str, sgtk.descriptor.ConfigDescriptor)

get_pipeline_configurations(project)[source]

Retrieves the pipeline configurations available for a given project.

In order for a pipeline configuration to be considered as available, the following conditions must be met:

  • There can only be one primary
  • If there is one site level and one project level primary, the site level primary is not available.
  • If there are multiple site level or multiple project level primaries, only the one with the lowest id is available, unless one or more of them is a Toolkit Classic Primary, in which case the Toolkit Classic Primary with the lowest id will be returned.
  • A Descriptor object must be able to be created from the pipeline configuration.
  • All sandboxes are available.

In practice, this means that if there are 3 primaries, two of them using plugin ids and one of them not using them, the one not using a plugin id will always be used.

This filtering also takes into account the current user and optional pipeline configuration name or id. If the pipeline_configuration() property has been set to a string, it will look for pipeline configurations with that specific name. If it has been set to None, any pipeline that can be applied for the current user and project will be retrieved. Note that this method does not support pipeline_configuration() being an integer.

Parameters:project (Dictionary with keys type and id.) – Project entity link to enumerate pipeline configurations for. If None, this will enumerate the pipeline configurations for the site configuration.
Returns:List of pipeline configurations.
Return type:List of dictionaries with keys type, id, name, project, and descriptor. The pipeline configurations will always be sorted such as the primary pipeline configuration, if available, will be first. Then the remaining pipeline configurations will be sorted by name field (case insensitive), then the project field and finally then id field.
get_entity_from_environment()[source]

Helper method that looks for the standard environment variables SHOTGUN_SITE, SHOTGUN_ENTITY_TYPE and SHOTGUN_ENTITY_ID and attempts to extract and validate them. This is typically used in conjunction with bootstrap_engine(). The standard environment variables read by this method can be generated by get_standard_plugin_environment().

Returns:Standard Shotgun entity dictionary with type and id or None if not defined.
resolve_descriptor(project)[source]

Resolves a pipeline configuration and returns its associated descriptor object.

Parameters:project (dict) – The project entity, or None.
static get_core_python_path()[source]

Computes the path to the current Toolkit core.

The current Toolkit core is defined as the core that gets imported when you type import sgtk and the python path is derived from that module.

For example, if the sgtk module was found at /path/to/config/install/core/python/sgtk, the return path would be /path/to/config/install/core/python

This can be useful if you want to hand down to a subprocess the location of the current process’s core, since sys.path and the PYTHONPATH are not updated after bootstrapping.

Returns:Path to the current core.
Return type:str

Exception Classes

class sgtk.bootstrap.TankBootstrapError[source]

Bases: tank.errors.TankError

Base class for all bootstrap related errors

Installing the sgtk module using pip

When running Toolkit using the bootstrap API above, you need access to the ToolkitManager class in order to kickstart the bootstrap process. Once you have started the bootstrap, toolkit will download all the necessary components for the given configuration you are bootstrapping into, potentially even including a different version of the core API than you are using to bootstrap with.

In order to fully automate this process programatically, you need an sgtk instance to begin with. One way to accomplish this is to use pip (see https://pip.pypa.io/). Use the following syntax:

pip install git+https://github.com/shotgunsoftware/tk-core@v0.18.35

If you want to add an sgtk core to a requirements.txt file, use the following syntax:

git+https://github.com/shotgunsoftware/tk-core@v0.18.35

Warning

In order to use pip, you currently need to have the git executable installed on the system that you are deploying to.

Warning

We strongly recommend always providing a version number. Not providing a version number will currently download the latest commit from the master branch and associate it with the highest available version number tag. Such downloads are likely to contain changes which have not yet been full tested.