Utilities

Logging

Toolkit uses the standard python logging for its log management. The LogManager class below acts as an interface that helps make it easy to access and manage Toolkit logging.

Logging hierarchy

All Toolkit logging is written into a sgtk.* logging namespace. This has been “sealed” so that log messages from Toolkit do not propagate up to the root logger. This is to ensure that Toolkit doesn’t interfere with other logging that has been already configured.

The following sub-hierarchies exist:

  • Each app, engine and bundle provides access to logging and these log streams are collected and organized under the sgtk.env logging namespace. Below this level, messages are broken down further by environment, engine, etc.

  • Logging from external tools and scripts is written to sgtk.ext.

  • All core logging is written to the sgtk.core logger.

Below is a simple log hierarchy to illustrate what this might look like in practice.

sgtk                                              Root point for all Toolkit logging
 |
 |- core                                          Root point for the Core API
 |   |
 |   |- descriptor                                Logging from core Modules
 |   |- path_cache
 |   |- hook
 |       |- create_folders                        Logging from a core hook
 |
 |- env                                           Logging from apps and engines
 |   |
 |   |- project                                   Toolkit Environment
 |       |
 |       |- tk-maya                               Toolkit Engine
 |             |
 |             |- startup                         Toolkit Engine Software Launcher
 |             |
 |             |- tk-multi-workfiles2             Toolkit App (or framework)
 |                  |
 |                  |- tkimp63c3b2d57f85          Toolkit Command Session
 |                  |   |
 |                  |   |- tk_multi_workfiles     Python hierarchy inside app's python folder
 |                  |       |
 |                  |       |- entity_tree
 |                  |
 |                  |
 |                  |
 |                  |- hook
 |                      |- scene_operations       Logging from a hook
 |
 |
 |- ext                                           Logging from associated external scripts
     |
     |- tank_cmd

Generating log messages in Toolkit

Generating log messages are done differently depending on your context. Below are a series of examples and best practice recipes explaining how to best apply logging to different scenarios.

Logging from within your App, Engine or Framework

Inside your app, a logger is available via logger(). Alternatively, you can also use the legacy methods log_debug|error|info|...(). This provides a basic level of general logging.

For code inside the python folder, which has been imported via Toolkit’s import_module() method, we recommend that you access a logger using the following method:

# at the top of the file, include the following
import sgtk
logger = sgtk.platform.get_logger(__name__)

def my_method():
    logger.debug("inside my code, i can log like this")

This logger will be grouped per invocation instance, meaning that you can see for example which dialog UI a particular collection of log messages comes from. An invocation is typically associated with someone launching the app from the Shotgun menu.

Note

Because log messages are grouped per invocation, this makes it easy to for example generate log files for export or import sessions running as part of an app. It also makes it possible to create a log window which displays the logging associated with a particular app UI dialog.

Logging from scripts and other external locations

If you want to add standard Toolkit logging to a script, simply use the following recipe:

# at the top of the file, include the following
import sgtk
logger = sgtk.LogManager.get_logger(__name__)

def my_method():
    logger.debug("inside my code, i can log like this")

All this logging will appear below the sgtk.ext logger.

Logging from inside the Core API

To emit log messages from inside the Toolkit Core API, use the following pattern:

# at the top of the file, include the following
import sgtk
logger = sgtk.LogManager.get_logger(__name__)

def my_method():
    logger.debug("inside my code, i can log like this")

Consuming log messages in Toolkit

Toolkit provides several ways to access the log information generated by the various methods and recipes shown above.

The general approach is to attach one or several log handlers to the root logging point of the hierarchy (sgtk). Each handler controls its own logging resolution, e.g. how much log information to display. The toolkit logging hierarchy itself is set to DEBUG resolution.

The Toolkit LogManager provides a default set of logging methods to help access log information.

Global debug

Toolkit has a concept of a global debug flag. This flag can be enabled by setting the TK_DEBUG environment variable or alternatively setting the LogManager.global_debug() property.

All log handlers that have been created using the LogManager will be affected by the flag.

Backend file logging

Log information is automatically written to disk by the LogManager. The location to which log files are written can be accessed via the LogManager.log_folder() property. Backend file logging is normally automatically enabled and end users do not need to worry about this. If you want debug logging to be written to these files, enable the global debug flag.

Note

If you are writing a toolkit plugin, we recommend that you initialize logging early on in your code by calling LogManager.initialize_base_file_handler(). This will ensure that all your logs are written to disk. If you omit this call, logging will automatically be started up as the engine is launched.

DCC Logging

Each toolkit engine integrates logging into the DCC. DCCs such as Maya, Nuke or houdini traditionally have a console of some sort where logging information typically should be dispatched.

Engine log output has traditionally been implemented by subclassing the log_info, log_error methods. In Core v0.18, a new and improved logging platform is introduced and we recommend that engines do not implement the log_xxx methods at all but instead implement a single _emit_log_message() method.

Standard Logging

If you want some sort of log output in addition to the logging an engine provides, you can add standard toolkit handlers. These handlers are created via the LogManager.initialize_custom_handler() method.

All log handlers created or registered via this method will respond to the global debug flag.

Note

If you want a raw log output that is not affected by any changes to the global debug flag, we recommend that you manually create your log handler and attach it to the sgtk root logger.

Python provides a large number of log handlers as part of its standard library. For more information, see https://docs.python.org/2/library/logging.handlers.html#module-logging.handlers

LogManager

class sgtk.log.LogManager(*args, **kwargs)[source]

Main interface for logging in Toolkit.

This class contains a series of methods to help standardize log output and access. Since Toolkit uses the standard python logging interface, you can manually configure and associate your logging if you like.

Note

This is a singleton class, so every time you instantiate it, the same object is returned.

static get_logger(log_name)[source]

Generates standard logger objects for Toolkit.

If you want to add standard toolkit logging to your code, the easiest way is to include the following at the top of your python file:

import sgtk
logger = sgtk.LogManager.get_logger(__name__)

This will pick up the module hierarchy of your code and parent it under the standard Toolkit logger.

Note

This method is useful if you are writing scripts, tools or wrappers. If you are developing a Toolkit app, framework or engine, you typically want to use sgtk.platform.get_logger() for your logging.

Note

To output logging to screen or to a console, we recommend using the initialize_custom_handler() convenience method.

Parameters:

log_name – Name of logger to create. This name will be parented under the sgtk namespace. If the name begins with tank., it will be automatically replaced with sgtk..

Returns:

Standard python logger.

static log_timing(func)[source]

Decorator that times and logs the execution of a method.

Sometimes it is useful to log runtime statistics about how long time a certain method takes to execute. In the case of Toolkit, it is particularly helpful when debugging issues to do with I/O or cloud connectivity.

If you have a method that for example connects to Shotgun to retrieve data, you can decorate it:

@sgtk.LogManager.log_timing
def my_shotgun_publish_method():
    '''
    Publishes lots of files to Shotgun
    '''
    # shotgun code here

In the debug logs, timings will be written to the sgtk.stopwatch logger:

[DEBUG sgtk.stopwatch.module] my_shotgun_publish_method: 0.633s
property global_debug

Controls the global debug flag in toolkit. Toggling this flag will affect all log handlers that have been created via initialize_custom_handler().

Note

Debug logging is off by default. If you want to permanently enable debug logging, set the environment variable TK_DEBUG.

property log_file

Full path to the current log file or None if logging is not active.

property log_folder

The folder where log files generated by initialize_base_file_handler() are stored.

property root_logger

Returns the root logger for Toolkit.

Note

If you want to add a custom logging handler to the root logger, we recommend using the initialize_custom_handler() method.

Warning

The root logger logs down to a debug resolution by default. Do not change the output level of logger as this will have a global effect. If you are connecting a logging handler and want to limit the stream of messages that are being emitted, instead adjust the logging level of the handler.

Returns:

log object

property base_file_handler

The base file handler that is used to write log files to disk in a default location, or None if not defined.

initialize_custom_handler(handler=None)[source]

Convenience method that initializes a log handler and attaches it to the toolkit logging root.

Note

If you want to display log messages inside a DCC, implement _emit_log_message().

Note

If global_debug() is set to True, the handler created will be set to debug level, otherwise it will be set to info level. Furthermore, the log handler will automatically adjust its log level whenever the global debug flag changes its state.

Calling this without parameters will generate a standard stream based logging handler that logs to stderr:

# start logging to stderr
import sgtk.LogManager
LogManager().initialize_custom_handler()

If you want to log to a file instead, create a log handler and pass that to the method:

handler = logging.FileHandler("/tmp/toolkit.log)
LogManager().initialize_custom_handler(handler)

The log handler will be configured to output its messages in a standard fashion.

Parameters:

handler – Logging handler to connect with the toolkit logger. If not passed, a standard stream handler will be created.

Returns:

The configured log handler.

uninitialize_base_file_handler()[source]

Uninitialize base file handler created with initialize_base_file_handler().

Returns:

The path to the previous log file that is being switched away from, None if no base logger was previously active.

initialize_base_file_handler(log_name)[source]

Create a file handler and attach it to the stgk base logger. This will write a rotating log file to disk in a standard location and will capture all log messages passed through the log hierarchy.

Note

Files will be written into the location on disk defined by log_folder().

When you start an engine via the sgtk.platform.start_engine() method, a file handler will automatically be created if one doesn’t already exist.

If you are manually launching toolkit, we recommend that you call this method to initialize logging to file early on in your setup. Calling it multiple times will not result in the information being written to multiple different files - only one file logger can exist per session.

Parameters:

log_name – Name of logger to create. This will form the filename of the log file. The .log will be suffixed.

Returns:

The path to the previous log file that is being switched away from, None if no base logger was previously active.

initialize_base_file_handler_from_path(log_file)[source]

Create a file handler and attach it to the sgtk base logger.

This method is there for legacy Toolkit applications and shouldn’t be used. Use initialize_base_file_handler instead.

Parameters:

log_file – Path of the file to write the logs to.

Returns:

The path to the previous log file that is being switched away from, None if no base logger was previously active.

Centralizing your settings

Instead of customizing your proxy settings on each of your project, it is possible to configure them once in a file and have your projects inherit these values, unless the project overrides itself the setting inside shotgun.yml.

Here’s an example:

# Login related settings
#
[Login]

# If specified, the username text input on the login screen will be populated
# with this value when logging into Toolkit for the very first time.
# Defaults to the user's OS login. Environment variables are actually resolved for
# all values in this file, which allows greater flexibility when sharing this configuration
# file with multiple users.
#
default_login=$USERNAME

# If specified, the site text input on the login screen will be populated with
# this value when logging into Toolkit the very first time.
# Defaults to https://mystudio.shotgunstudio.com.
#
default_site=https://your-site-here.shotgunstudio.com

# If specified, the Toolkit will use these proxy settings to connect to
# the Flow Production Tracking site and the Toolkit App Store. The proxy string should be of the
# forms 123.123.123.123, 123.123.123.123:8888 or
# username:pass@123.123.123.123:8888.
# Empty by default.
#
http_proxy=123.234.345.456:8888

# If specified, the Flow Production Tracking API will use these proxy settings to connect
# to the Toolkit App Store. The proxy string format is the same as http_proxy.
# If the setting is present in the file but not set, then no proxy will be used
# to connect to the Toolkit App Store, regardless of the value of the http_proxy
# setting.
# Empty by default.
#
app_store_http_proxy=123.234.345.456:8888

This file can be configured through multiple means and Toolkit will try to resolve the file in the following order:

  1. The SGTK_PREFERENCES_LOCATION environment variable,

  2. The SGTK_DESKTOP_CONFIG_LOCATION environment variable, for compatibility with the Flow Production Tracking. (deprecated)

  3. Inside the Flow Production Tracking preferences file

  4. Inside the Flow Production Tracking preferences file, for compatibility with the Flow Production Tracking. (deprecated)

Note

The Flow Production Tracking preferences file is located at:

  • Windows: %APPDATA%\Shotgun\Preferences\toolkit.ini

  • macOS: ~/Library/Preferences/Shotgun/toolkit.ini

  • Linux: ~/.shotgun/preferences/toolkit.ini

The Flow Production Tracking preferences file is located at:

  • Windows: %APPDATA%\Shotgun\desktop\config\config.ini

  • macOS: ~/Library/Caches/Shotgun/desktop/config/config.ini

  • Linux: ~/.shotgun/desktop/config/config.ini

Note that the SHOTGUN_HOME environment variable can impact the location of the Flow Production Tracking preferences file.

Note

When the http proxy is not specified in this file, the Flow Production Tracking will try to retrieve the operating system http proxy.

First, the environment will be scanned for variables named http_proxy, in case insensitive way. If both lowercase and uppercase environment variables exist (and disagree), lowercase will be preferred.

When such environment variables cannot be found:

  • for Mac OS X, proxy information will be looked for from Mac OS X System Configuration,

  • for Windows, proxy information will be looked for from Windows Systems Registry.

There is a restriction in these latter cases: the use of proxies which require authentication (username and password) is not supported.

Internally, the Flow Production Tracking uses Python function urllib.getproxies() to retrieve the operating system http proxy. More information about this function can be found here:

You can access those values programmatically.

class sgtk.util.UserSettings(*args, **kwargs)[source]

Handles finding and loading the user settings for Toolkit. The settings are cached in memory so the user settings object can be instantiated multiple times without any issue.

All the settings are returned as strings. If a setting is missing from the file, None will be returned. If the setting is present but has no value, an empty string will be returned.

As of this writing, settings can only be updated by editing the ini file manually.

Create the singleton instance if it hasn’t been created already. Once instantiated, the object will be cached and never be instantiated again for performance reasons.

property shotgun_proxy

Retrieves the value from the http_proxy setting.

property app_store_proxy

Retrieves the value from the app_store_http_proxy setting.

property default_site

Retrieves the value from the default_site setting.

property default_login

Retrieves the value from the default_login setting.

get_section_settings(section)[source]

Retrieves the name of the settings in a given section.

Parameters:

section (str) – Name of the section of the settings to retrieve.

Returns:

A list of setting’s name. If the section is missing, returns None.

get_setting(section, name)[source]

Provides access to any setting, including ones in user defined sections.

Parameters:
  • section (str) – Name of the section to retrieve the setting from. Do not include the brackets.

  • name (str) – Name of the setting under the provided section.

Returns:

The setting’s value if found, None if the setting is missing from the file or an empty string if the setting is present but has no value associated.

Return type:

str

get_boolean_setting(section, name)[source]

Provides access to any setting, including ones in user defined sections, and casts it into a boolean.

Values 1, yes, true and on are converted to True while 0, no, false``and ``off are converted to false. Case is insensitive.

Parameters:
  • section (str) – Name of the section to retrieve the setting from. Do not include the brackets.

  • name (str) – Name of the setting under the provided section.

Returns:

Boolean if the value is valid, None if not set.

Return type:

bool

Raises:

TankError – Raised if the value is not one of the accepted values.

get_integer_setting(section, name)[source]

Provides access to any setting, including ones in user defined sections, and casts it into an integer.

Parameters:
  • section (str) – Name of the section to retrieve the setting from. Do not include the brackets.

  • name (str) – Name of the setting under the provided section.

Returns:

Boolean if the value is valid, None if not set.

Return type:

bool

Raises:

TankError – Raised if the value is not one of the accepted values.

File System Utilities

Below is a collection of file system related convenience methods that make it easy to manage and create files and folders on disk in a standard fashion.

Note

These methods are not configurable or overridable by hooks. If you are developing apps or engines, we recommend using ensure_folder_exists() as this method calls out to a customizable hook implementation.

sgtk.util.filesystem

sgtk.util.filesystem.with_cleared_umask(func)[source]

Decorator which clears the umask for a method.

The umask is a permissions mask that gets applied whenever new files or folders are created. For I/O methods that have a permissions parameter, it is important that the umask is cleared prior to execution, otherwise the default umask may alter the resulting permissions, for example:

def create_folders(path, permissions=0777):
    log.debug("Creating folder %s..." % path)
    os.makedirs(path, permissions)

The 0777 permissions indicate that we want folders to be completely open for all users (a+rwx). However, the umask overrides this, so if the umask for example is set to 0777, meaning that I/O operations are not allowed to create files that are readable, executable or writable for users, groups or others, the resulting permissions on folders created by create folders will be 0, despite passing in 0777 permissions.

By adding this decorator to the method, we temporarily reset the umask to 0, thereby giving full control to any permissions operation to take place without any restriction by the umask:

@with_cleared_umask
def create_folders(path, permissions=0777):
    # Creates folders with the given permissions,
    # regardless of umask setting.
    log.debug("Creating folder %s..." % path)
    os.makedirs(path, permissions)
sgtk.util.filesystem.compute_folder_size(path)[source]

Computes and returns the size of the given folder.

Parameters:

path – folder to compute size for

Returns:

size in bytes

sgtk.util.filesystem.touch_file(path, permissions=0666)[source]

Touch a file and optionally set its permissions.

Parameters:
  • path – path to touch

  • permissions – Optional permissions to set on the file. Default value is 0666, creating a file that is readable and writable for all users.

Raises:

OSError - if there was a problem reading/writing the file

sgtk.util.filesystem.ensure_folder_exists(path, permissions=0775, create_placeholder_file=False)[source]

Helper method - creates a folder and parent folders if such do not already exist.

Parameters:
  • path – path to create

  • permissions – Permissions to use when folder is created

  • create_placeholder_file – If true, a placeholder file will be generated.

Raises:

OSError - if there was a problem creating the folder

sgtk.util.filesystem.copy_file(src, dst, permissions=0666)[source]

Copy file and sets its permissions.

Parameters:
  • src – Source file

  • dst – Target destination

  • permissions – Permissions to use for target file. Default permissions will be readable and writable for all users.

sgtk.util.filesystem.safe_delete_file(path)[source]

Deletes the given file if it exists.

Ignores any errors raised in the process and logs them as warnings. If the user does not have sufficient permissions to remove the file, nothing will happen, it will simply be skipped over.

Parameters:

path – Full path to file to remove

sgtk.util.filesystem.safe_delete_folder(path)[source]

Deletes a folder and all of its contents recursively, even if it has read-only items.

Note

Problems deleting any items will be reported as warnings in the log output but otherwise ignored and skipped; meaning the function will continue deleting as much as it can.

Parameters:

path – File system path to location to the folder to be deleted

sgtk.util.filesystem.copy_folder(src, dst, folder_permissions=0775, skip_list=None)[source]

Alternative implementation to shutil.copytree

Copies recursively and creates folders if they don’t already exist. Always skips system files such as "__MACOSX", ".DS_Store", etc. Files will the extension .sh, .bat or .exe will be given executable permissions.

Returns a list of files that were copied.

Parameters:
  • src – Source path to copy from

  • dst – Destination to copy to

  • folder_permissions – permissions to use for new folders

  • skip_list – List of file names to skip. If this parameter is omitted or set to None, common files such as .git, .gitignore etc will be ignored.

Returns:

List of files copied

sgtk.util.filesystem.move_folder(src, dst, folder_permissions=0775)[source]

Moves a directory.

First copies all content into target. Then deletes all content from sources. Skips files that won’t delete.

Note

The source folder itself is not deleted, it is just emptied, if possible.

Parameters:
  • src – Source path to copy from

  • dst – Destination to copy to

  • folder_permissions – permissions to use for new folders

sgtk.util.filesystem.backup_folder(src, dst=None)[source]

Moves the given directory into a backup location.

By default, the folder will be renamed by simply giving it a timestamp based suffix. Optionally, it can be moved into a different location.

  • backup_folder("/foo/bar") will move /foo/bar to /foo/bar.20160912_200426

  • backup_folder("/foo/bar", "/tmp") will move /foo/bar to /tmp/bar.20160912_200426

Parameters:
  • src – Folder to move

  • dst – Optional backup folder

sgtk.util.filesystem.create_valid_filename(value)[source]

Create a sanitized file name given a string. Replaces spaces and other characters with underscores

‘my lovely name ‘ -> ‘my_lovely_name’

Parameters:

value – String value to sanitize

Returns:

sanitized string

sgtk.util.filesystem.get_unused_path(base_path)[source]

Return an unused file path from the given base path by appending if needed a number at the end of the basename of the path, right before the first “.”, if any.

For example, /tmp/foo_1.bar.blah would be returned for /tmp/foo.bar.blah if it already exists.

If the given path does not exist, the original path is returned.

Note

The returned path is not _reserved_, so it is possible that other processes could create the returned path before it is used by the caller.

Parameters:

base_path (str) – Target path.

Returns:

A string.

sgtk.util.json

sgtk.util.json.load(fp, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, **kw)[source]

Deserialize fp (a .read()-supporting file-like object containing a JSON document) to a Python object.

This method is a simple thin wrapper around json.load() that ensures unserialized strings are utf-8 encoded str objects.

See the documentation for json.load() to learn more about this method.

sgtk.util.json.loads(s, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, **kw)[source]

Deserialize s (a str or unicode instance containing a JSON document) to a Python object.

This method is a simple thin wrapper around json.loads() that ensures unserialized strings are utf-8 encoded str objects.

See the documentation for json.loads() to learn more about this method.

sgtk.util.pickle

Toolkit’s pickle module isn’t a drop-in replacement for Python’s, but a wrapper around Python’s pickle module so collections and scalar types can be exchanged freely between Python 2 and Python 3 processes without having to worry about the subtle pickle serialization differences between the two.

If you wish to serialize your own custom classes to be exchanged between Python 2 and Python 3, you will need to sanitize data unpickled with Python 2 that was pickled with Python 3 and vice-versa. Otheriwse, your data will contain unexpected bytes or unicode objects instead of utf-8 encoded str instances.

Note

In the load/loads methods, Python 3’s bytes is always decoded back to a utf-8 encoded str. If you need to store arbitrary binary data, consider saving it as a base64 string instead.

sgtk.util.pickle.dump(data, fh)[source]

Write the pickled representation of data to a file object.

This methods wraps the functionality from the pickle.dump() method so pickles can be shared between Python 2 and Python 3.

Parameters:
  • data – The object to pickle and store.

  • fh – A file object

sgtk.util.pickle.dumps(data)[source]

Return the pickled representation of data as a str.

This methods wraps the functionality from the pickle.dumps() method so pickles can be shared between Python 2 and Python 3.

As opposed to the Python 3 implementation, it will return a str object and not bytes object.

Parameters:

data – The object to pickle and store.

Returns:

A pickled str of the input object.

Return type:

str

sgtk.util.pickle.load(fh)[source]

Read the pickled representation of an object from the open file object and return the reconstituted object hierarchy specified therein.

This method wraps the functionality from the pickle.load() method so unicode strings are always returned as utf8-encoded str instead of unicode objects in Python 2.

Parameters:

fh – A file object

Returns:

The unpickled object.

Return type:

object

sgtk.util.pickle.loads(data)[source]

Read the pickled representation of an object from a string and return the reconstituted object hierarchy specified therein.

This method wraps the functionality from the pickle.loads() method so unicode strings are always returned as utf8-encoded str instead of unicode objects in Python 2.

Parameters:

data (object) – A pickled representation of an object.

Returns:

The unpickled object.

Return type:

object

ShotgunPath

class sgtk.util.ShotgunPath(windows_path=None, linux_path=None, macosx_path=None)[source]

Helper class that handles a path on multiple operating systems.

Contains methods to easily cast multi-os path between shotgun and os representations and mappings. The ShotgunPath object automatically sanitizes any path that it is given. When working with local storages in Shotgun, roots are keyed by the tokens windows_path, linux_path and mac_path. When using sys.platform in python, you get back win32, darwin and linux2 depending on platform. This class makes it easy to perform operations and cast between representations and platforms.

Usage example:

>>> ShotgunPath.SHOTGUN_PATH_FIELDS
["windows_path", "linux_path", "mac_path"]

# construction
>>> p = ShotgunPath("C:\temp", "/tmp", "/tmp")
>>> p = ShotgunPath.from_shotgun_dict({ "windows_path": "C:\temp", "mac_path": None, "linux_path": "/tmp"})
>>> p = ShotgunPath.from_system_dict({ "win32": "C:\temp", "darwin": None, "linux2": "/tmp"})
>>> p = ShotgunPath.from_current_os_path("/tmp")

# access
>>> p.macosx
None
>>> p.windows
"C:\temp"
>>> p.linux
'/tmp
>>> p.current_os
'/tmp'

# boolean operations
>>> if p: print "a path value defined for windows, linux or mac"

# equality
>>> if p1 == p2: print "paths are same"

# multi-platform access
>>> p.as_shotgun_dict()
{ "windows_path": "C:\temp", "mac_path": None, "linux_path": "/tmp"}
>>> p.as_system_dict()
{ "win32": "C:\temp", "darwin": None, "linux2": "/tmp"}

# descriptor uri conversion
>>> p.as_descriptor_uri()
'sgtk:descriptor:path?linux_path=/tmp/foo'

# path manipulation
>>> p2 = p.join('foo')
>>> p2
<Path win:'c:\temp\foo', linux:'/tmp/foo', macosx:'/tmp/foo'>
Parameters:
  • windows_path – Path on windows to associate with this path object

  • linux_path – Path on linux to associate with this path object

  • macosx_path – Path on macosx to associate with this path object

SHOTGUN_PATH_FIELDS = ['windows_path', 'linux_path', 'mac_path']

A list of the standard path fields used by Shotgun.

static get_file_name_from_template(template, platform='linux')[source]

Returns the complete file name for the current platform based on file name template passed in.

Parameters:

template (str) – Template for a file name with a %s to indicate where the platform name should be inserted.

Returns:

Path with the OS name substituted in.

static get_shotgun_storage_key(platform='linux')[source]

Given a sys.platform constant, resolve a Shotgun storage key

Shotgun local storages handle operating systems using the three keys ‘windows_path, ‘mac_path’ and ‘linux_path’, also defined as ShotgunPath.SHOTGUN_PATH_FIELDS

This method resolves the right key given a std. python sys.platform:

>>> p.get_shotgun_storage_key('win32')
'windows_path'

# if running on a mac
>>> p.get_shotgun_storage_key()
'mac_path'
Parameters:

platform – sys.platform style string, e.g ‘linux2’, ‘win32’ or ‘darwin’.

Returns:

Shotgun storage path as string.

classmethod from_shotgun_dict(sg_dict)[source]

Creates a path from data contained in a std shotgun data dict, containing the paths windows_path, mac_path and linux_path

Parameters:

sg_dict – Shotgun query resultset with possible keys windows_path, mac_path and linux_path.

Returns:

ShotgunPath instance

classmethod from_system_dict(system_dict)[source]

Creates a path from data contained in a dictionary keyed by sys.platform constants.

Parameters:

system_dict – Dictionary with possible keys win32, darwin and linux2.

Returns:

ShotgunPath instance

classmethod from_current_os_path(path)[source]

Creates a path object for a path on the current platform only.

Parameters:

path – Path on the current os platform.

Returns:

ShotgunPath instance

classmethod normalize(path)[source]

Convenience method that normalizes the given path by running it through the ShotgunPath normalization logic. ShotgunPath.normalize(path) is equivalent to executing ShotgunPath.from_current_os_path(path).current_os.

Normalization include checking that separators are matching the current operating system, removal of trailing separators and removal of double separators. This is done automatically for all ShotgunPath, but sometimes it is useful to just perform the normalization quickly on a local path.

Parameters:

path (str) – Local operating system path to normalize

Returns:

Normalized path string.

property macosx

The macosx representation of the path

property windows

The Windows representation of the path

property linux

The Linux representation of the path

property current_os

The path on the current os

as_shotgun_dict(include_empty=True)[source]

The path as a shotgun dictionary. With include_empty set to True:

{ "windows_path": "C:\temp", "mac_path": None, "linux_path": "/tmp"}

With include_empty set to False:

{ "windows_path": "C:\temp", "linux_path": "/tmp"}
Parameters:

include_empty – Controls whether keys should be included for empty path values

Returns:

dictionary of paths keyed by standard shotgun keys.

as_system_dict(include_empty=True)[source]

The path as a dictionary keyed by sys.platform.

With include_empty set to True:

{ "win32": "C:\temp", "darwin": None, "linux2": "/tmp"}

With include_empty set to False:

{ "win32": "C:\temp", "linux2": "/tmp"}
Parameters:

include_empty – Controls whether keys should be included for empty path values

Returns:

dictionary of paths keyed by sys.platform.

as_descriptor_uri(for_development=False)[source]

Translates the path to a descriptor uri. For more information about descriptors, see the reference documentation.

This method will either return a dev or a path descriptor uri path string, suitable for use with for example pipeline configurations in Shotgun.

Parameters:

for_development (bool) – Set to true for a dev descriptor

Returns:

Dev or Path descriptor uri string representing the path

Raises:

ValueError if the path object has no paths defined

join(folder)[source]

Appends a single folder to the path.

Parameters:

folder – folder name as sting

Returns:

ShotgunPath object containing the new path

LocalFileStorageManager

class sgtk.util.LocalFileStorageManager[source]

Class that encapsulates logic for resolving local storage paths.

Toolkit needs to store cache data, logs and other items at runtime. Some of this data is global, other is per site or per configuration.

This class provides a consistent and centralized interface for resolving such paths and also handles compatibility across generations of path standards if and when these change between releases.

Note

All paths returned by this class are local to the currently running user and typically private or with limited access settings for other users.

If the current user’s home directory is not an appropriate location to store your user files, you can use the SHOTGUN_HOME environment variable to override the root location of the files. In that case, the location for the user files on each platform will be:

  • Logging: $SHOTGUN_HOME/logs

  • Cache: $SHOTGUN_HOME

  • Persistent: $SHOTGUN_HOME/data

  • Preferences: $SHOTGUN_HOME/preferences

Constant CORE_V17:

Indicates compatibility with Core 0.17 or earlier

Constant CORE_V18:

Indicates compatibility with Core 0.18 or later

Constant LOGGING:

Indicates a path suitable for storing logs, useful for debugging

Constant CACHE:

Indicates a path suitable for storing cache data that can be deleted without any loss of functionality or state.

Constant PERSISTENT:

Indicates a path suitable for storing data that needs to be retained between sessions.

Constant PREFERENCES:

Indicates a path that suitable for storing settings files and preferences.

classmethod get_global_root(path_type, generation=1)[source]

Returns a generic Shotgun storage root.

The following paths will be used:

  • On the mac, paths will point into ~/Library/PATH_TYPE/Shotgun, where PATH_TYPE is controlled by the path_type property.

  • On Windows, paths will created below a %APPDATA%/Shotgun root point.

  • On Linux, paths will be created below a ~/.shotgun root point.

Note

This method does not ensure that the folder exists.

Parameters:
  • path_type – Type of path to return. One of LocalFileStorageManager.LOGGING, LocalFileStorageManager.CACHE, LocalFileStorageManager.PERSISTENT, where logging is a path where log- and debug related data should be stored, cache is a location intended for cache data, e.g. data that can be deleted without affecting the state of execution, and persistent is a location intended for data that is meant to be persist. This includes things like settings and preferences.

  • generation – Path standard generation to use. Defaults to LocalFileStorageManager.CORE_V18, which is the current generation of paths.

Returns:

Path as string

classmethod get_site_root(hostname, path_type, generation=1)[source]

Returns a cache root where items can be stored on a per site basis.

For more details, see LocalFileStorageManager.get_global_root().

Note

This method does not ensure that the folder exists.

Parameters:
  • hostname – Shotgun hostname as string, e.g. ‘https://foo.shotgunstudio.com

  • path_type – Type of path to return. One of LocalFileStorageManager.LOGGING, LocalFileStorageManager.CACHE, LocalFileStorageManager.PERSISTENT, where logging is a path where log- and debug related data should be stored, cache is a location intended for cache data, e.g. data that can be deleted without affecting the state of execution, and persistent is a location intended for data that is meant to be persist. This includes things like settings and preferences.

  • generation – Path standard generation to use. Defaults to LocalFileStorageManager.CORE_V18, which is the current generation of paths.

Returns:

Path as string

classmethod get_configuration_root(hostname, project_id, plugin_id, pipeline_config_id, path_type, generation=1)[source]

Returns the storage root for any data that is project and config specific.

  • A well defined project id should always be passed. Passing None as the project id indicates that the site configuration, a special toolkit configuration that represents the non-project state in Shotgun.

  • Configurations that have a pipeline configuration in Shotgun should pass in a pipeline configuration id. When a pipeline configuration is not registered in Shotgun, this value should be None.

  • If the configuration has been bootstrapped or has a known plugin id, this should be specified via the plugin id parameter.

For more details, see LocalFileStorageManager.get_global_root().

Examples of paths that will be generated:

  • Site config: ROOT/shotgunsite/p0

  • Project 123, config 33: ROOT/shotgunsite/p123c33

  • project 123, no config, plugin id review.rv: ROOT/shotgunsite/p123.review.rv

Note

This method does not ensure that the folder exists.

Parameters:
  • hostname – Shotgun hostname as string, e.g. ‘https://foo.shotgunstudio.com

  • project_id – Shotgun project id as integer. For the site config, this should be None.

  • plugin_id – Plugin id string to identify the scope for a particular plugin or integration. For more information, see plugin_id(). For non-plugin based toolkit projects, this value is None.

  • pipeline_config_id – Shotgun pipeline config id. None for bootstraped configs.

  • path_type – Type of path to return. One of LocalFileStorageManager.LOGGING, LocalFileStorageManager.CACHE, LocalFileStorageManager.PERSISTENT, where logging is a path where log- and debug related data should be stored, cache is a location intended for cache data, e.g. data that can be deleted without affecting the state of execution, and persistent is a location intended for data that is meant to be persist. This includes things like settings and preferences.

  • generation – Path standard generation to use. Defaults to LocalFileStorageManager.CORE_V18, which is the current generation of paths.

Returns:

Path as string

OS detection

Below are a collection of convenience methods to detect which operating system is in use:

sgtk.util.is_linux(platform=None)[source]

Determine if the current platform is Linux.

Parameters:

platform – sys.platform style string, e.g ‘linux2’, ‘win32’ or ‘darwin’. If not provided, sys.platform will be used.

Returns:

True if the current platform is Linux, otherwise False.

Return type:

bool

sgtk.util.is_windows(platform=None)[source]

Determine if the current platform is Windows.

Parameters:

platform – sys.platform style string, e.g ‘linux2’, ‘win32’ or ‘darwin’. If not provided, sys.platform will be used.

Returns:

True if the current platform is Windows, otherwise False.

Return type:

bool

sgtk.util.is_macos(platform=None)[source]

Determine if the current platform is MacOS.

Parameters:

platform – sys.platform style string, e.g ‘linux2’, ‘win32’ or ‘darwin’. If not provided, sys.platform will be used.

Returns:

True if the current platform is MacOS, otherwise False.

Return type:

bool

Miscellaneous

sgtk.util.append_path_to_env_var(env_var_name, path)[source]

Append the path to the given environment variable. Creates the env var if it doesn’t exist already. will concatenate paths using : on linux and ; on windows

sgtk.util.prepend_path_to_env_var(env_var_name, path)[source]

Prepend the path to the given environment variable. Creates the env var if it doesn’t exist already. will concatenate paths using : on linux and ; on windows

sgtk.util.get_current_user(tk)[source]

Retrieves the current user as a dictionary of metadata values. Note: This method connects to shotgun the first time around. The result is then cached to reduce latency.

If a user has been authenticated via a login prompt, this method will return the credentials associated with that user. If Toolkit has been configured to use a script user to connect to Shotgun, a core hook will be executed to established which user is associated with the current session. This is usually based on the currently logged in user.

Returns:

None if the user is not found in Shotgun. Otherwise, it returns a dictionary with the following fields: id, type, email, login, name, image, firstname, lastname

Exceptions

class sgtk.util.EnvironmentVariableFileLookupError(var_name, path)[source]

Bases: TankError

Raised when an environment variable specifying a location points to configuration file that doesn’t exist.

Parameters:
  • var_name (str) – Name of the environment variable used.

  • path (str) – Path to the configuration file that doesn’t exist.

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class sgtk.util.ShotgunPublishError(error_message, entity=None)[source]

Bases: TankError

Raised when Toolkit is not able to register a published file in Shotgun.

The original message for the reported error is available in the ‘error_message’ property.

If a published file entity was created before the error happened, it will be available in the ‘entity’ property.

Parameters:
  • error_message (str) – An error message, typically coming from a caught exception.

  • entity (dict) – The Shotgun entity which was created, if any.

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class sgtk.util.PublishResolveError[source]

Bases: TankError

Base class for all errors relating to resolution of paths from publishes.

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class sgtk.util.PublishPathNotDefinedError[source]

Bases: PublishResolveError

Exception raised when a publish does not have a path defined for the current operating system platform. It may or may not have publish paths defined on other platforms.

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class sgtk.util.PublishPathNotSupported[source]

Bases: PublishResolveError

Exception raised when a publish has a path defined but it is using a path definition that cannot be resolved into a local path. This includes for example unsupported url schemes.

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.