Utilities
Logging
Toolkit uses the standard python logging for its
log management. The LogManager
class below
acts as an interface that helps make it easy to access
and manage Toolkit logging.
Logging hierarchy
All Toolkit logging is written into a sgtk.*
logging
namespace. This has been “sealed” so that log messages
from Toolkit do not propagate up to the root logger. This
is to ensure that Toolkit doesn’t interfere with other logging
that has been already configured.
The following sub-hierarchies exist:
Each app, engine and bundle provides access to logging and these log streams are collected and organized under the
sgtk.env
logging namespace. Below this level, messages are broken down further by environment, engine, etc.Logging from external tools and scripts is written to
sgtk.ext
.All core logging is written to the
sgtk.core
logger.
Below is a simple log hierarchy to illustrate what this might look like in practice.
sgtk Root point for all Toolkit logging
|
|- core Root point for the Core API
| |
| |- descriptor Logging from core Modules
| |- path_cache
| |- hook
| |- create_folders Logging from a core hook
|
|- env Logging from apps and engines
| |
| |- project Toolkit Environment
| |
| |- tk-maya Toolkit Engine
| |
| |- startup Toolkit Engine Software Launcher
| |
| |- tk-multi-workfiles2 Toolkit App (or framework)
| |
| |- tkimp63c3b2d57f85 Toolkit Command Session
| | |
| | |- tk_multi_workfiles Python hierarchy inside app's python folder
| | |
| | |- entity_tree
| |
| |
| |
| |- hook
| |- scene_operations Logging from a hook
|
|
|- ext Logging from associated external scripts
|
|- tank_cmd
Generating log messages in Toolkit
Generating log messages are done differently depending on your context. Below are a series of examples and best practice recipes explaining how to best apply logging to different scenarios.
Logging from within your App, Engine or Framework
Inside your app, a logger is available via logger()
.
Alternatively, you can also
use the legacy methods log_debug|error|info|...()
. This provides
a basic level of general logging.
For code inside the python
folder, which has been imported via
Toolkit’s import_module()
method,
we recommend that you access a logger using the following method:
# at the top of the file, include the following
import sgtk
logger = sgtk.platform.get_logger(__name__)
def my_method():
logger.debug("inside my code, i can log like this")
This logger will be grouped per invocation instance, meaning that you can see for example which dialog UI a particular collection of log messages comes from. An invocation is typically associated with someone launching the app from the Shotgun menu.
Note
Because log messages are grouped per invocation, this makes it easy to for example generate log files for export or import sessions running as part of an app. It also makes it possible to create a log window which displays the logging associated with a particular app UI dialog.
Logging from scripts and other external locations
If you want to add standard Toolkit logging to a script, simply use the following recipe:
# at the top of the file, include the following
import sgtk
logger = sgtk.LogManager.get_logger(__name__)
def my_method():
logger.debug("inside my code, i can log like this")
All this logging will appear below the sgtk.ext
logger.
Logging from inside the Core API
To emit log messages from inside the Toolkit Core API, use the following pattern:
# at the top of the file, include the following
import sgtk
logger = sgtk.LogManager.get_logger(__name__)
def my_method():
logger.debug("inside my code, i can log like this")
Consuming log messages in Toolkit
Toolkit provides several ways to access the log information generated by the various methods and recipes shown above.
The general approach is to attach one or several log handlers to the root
logging point of the hierarchy (sgtk
). Each handler controls its own
logging resolution, e.g. how much log information to display. The toolkit
logging hierarchy itself is set to DEBUG resolution.
The Toolkit LogManager
provides a default set of logging methods
to help access log information.
Global debug
Toolkit has a concept of a global debug flag. This flag can be
enabled by setting the TK_DEBUG
environment variable or
alternatively setting the LogManager.global_debug()
property.
All log handlers that have been created using the LogManager
will be affected by the flag.
Backend file logging
Log information is automatically written to disk by the LogManager
.
The location to which log files are written can be accessed via the
LogManager.log_folder()
property. Backend file logging is normally
automatically enabled and end users do not need to worry about this.
If you want debug logging to be written to these files, enable the
global debug flag.
Note
If you are writing a toolkit plugin, we recommend that you initialize logging early on in your code by calling
LogManager.initialize_base_file_handler()
. This will ensure that all your logs are written to disk. If you omit this call, logging will automatically be started up as the engine is launched.
DCC Logging
Each toolkit engine integrates logging into the DCC. DCCs such as Maya, Nuke or houdini traditionally have a console of some sort where logging information typically should be dispatched.
Engine log output has traditionally been implemented by subclassing
the log_info
, log_error
methods. In Core v0.18, a new and
improved logging platform is introduced and we recommend that engines
do not implement the log_xxx
methods at all but instead implement
a single _emit_log_message()
method.
Standard Logging
If you want some sort of log output in addition to the logging an
engine provides, you can add standard toolkit handlers. These handlers
are created via the LogManager.initialize_custom_handler()
method.
All log handlers created or registered via this method will respond to the global debug flag.
Note
If you want a raw log output that is not affected by any changes to the global debug flag, we recommend that you manually create your log handler and attach it to the
sgtk
root logger.
Python provides a large number of log handlers as part of its standard library. For more information, see https://docs.python.org/2/library/logging.handlers.html#module-logging.handlers
LogManager
- class sgtk.log.LogManager(*args, **kwargs)[source]
Main interface for logging in Toolkit.
This class contains a series of methods to help standardize log output and access. Since Toolkit uses the standard python logging interface, you can manually configure and associate your logging if you like.
Note
This is a singleton class, so every time you instantiate it, the same object is returned.
- static get_logger(log_name)[source]
Generates standard logger objects for Toolkit.
If you want to add standard toolkit logging to your code, the easiest way is to include the following at the top of your python file:
import sgtk logger = sgtk.LogManager.get_logger(__name__)
This will pick up the module hierarchy of your code and parent it under the standard Toolkit logger.
Note
This method is useful if you are writing scripts, tools or wrappers. If you are developing a Toolkit app, framework or engine, you typically want to use
sgtk.platform.get_logger()
for your logging.Note
To output logging to screen or to a console, we recommend using the
initialize_custom_handler()
convenience method.- Parameters:
log_name – Name of logger to create. This name will be parented under the sgtk namespace. If the name begins with
tank.
, it will be automatically replaced withsgtk.
.- Returns:
Standard python logger.
- static log_timing(func)[source]
Decorator that times and logs the execution of a method.
Sometimes it is useful to log runtime statistics about how long time a certain method takes to execute. In the case of Toolkit, it is particularly helpful when debugging issues to do with I/O or cloud connectivity.
If you have a method that for example connects to Shotgun to retrieve data, you can decorate it:
@sgtk.LogManager.log_timing def my_shotgun_publish_method(): ''' Publishes lots of files to Shotgun ''' # shotgun code here
In the debug logs, timings will be written to the
sgtk.stopwatch
logger:[DEBUG sgtk.stopwatch.module] my_shotgun_publish_method: 0.633s
- property global_debug
Controls the global debug flag in toolkit. Toggling this flag will affect all log handlers that have been created via
initialize_custom_handler()
.Note
Debug logging is off by default. If you want to permanently enable debug logging, set the environment variable
TK_DEBUG
.
- property log_file
Full path to the current log file or None if logging is not active.
- property log_folder
The folder where log files generated by
initialize_base_file_handler()
are stored.
- property root_logger
Returns the root logger for Toolkit.
Note
If you want to add a custom logging handler to the root logger, we recommend using the
initialize_custom_handler()
method.Warning
The root logger logs down to a debug resolution by default. Do not change the output level of logger as this will have a global effect. If you are connecting a logging handler and want to limit the stream of messages that are being emitted, instead adjust the logging level of the handler.
- Returns:
log object
- property base_file_handler
The base file handler that is used to write log files to disk in a default location, or None if not defined.
- initialize_custom_handler(handler=None)[source]
Convenience method that initializes a log handler and attaches it to the toolkit logging root.
Note
If you want to display log messages inside a DCC, implement
_emit_log_message()
.Note
If
global_debug()
is set to True, the handler created will be set to debug level, otherwise it will be set to info level. Furthermore, the log handler will automatically adjust its log level whenever the global debug flag changes its state.Calling this without parameters will generate a standard stream based logging handler that logs to stderr:
# start logging to stderr import sgtk.LogManager LogManager().initialize_custom_handler()
If you want to log to a file instead, create a log handler and pass that to the method:
handler = logging.FileHandler("/tmp/toolkit.log) LogManager().initialize_custom_handler(handler)
The log handler will be configured to output its messages in a standard fashion.
- Parameters:
handler – Logging handler to connect with the toolkit logger. If not passed, a standard stream handler will be created.
- Returns:
The configured log handler.
- uninitialize_base_file_handler()[source]
Uninitialize base file handler created with
initialize_base_file_handler()
.- Returns:
The path to the previous log file that is being switched away from, None if no base logger was previously active.
- initialize_base_file_handler(log_name)[source]
Create a file handler and attach it to the stgk base logger. This will write a rotating log file to disk in a standard location and will capture all log messages passed through the log hierarchy.
Note
Files will be written into the location on disk defined by
log_folder()
.When you start an engine via the
sgtk.platform.start_engine()
method, a file handler will automatically be created if one doesn’t already exist.If you are manually launching toolkit, we recommend that you call this method to initialize logging to file early on in your setup. Calling it multiple times will not result in the information being written to multiple different files - only one file logger can exist per session.
- Parameters:
log_name – Name of logger to create. This will form the filename of the log file. The
.log
will be suffixed.- Returns:
The path to the previous log file that is being switched away from, None if no base logger was previously active.
- initialize_base_file_handler_from_path(log_file)[source]
Create a file handler and attach it to the sgtk base logger.
This method is there for legacy Toolkit applications and shouldn’t be used. Use
initialize_base_file_handler
instead.- Parameters:
log_file – Path of the file to write the logs to.
- Returns:
The path to the previous log file that is being switched away from, None if no base logger was previously active.
Centralizing your settings
Instead of customizing your proxy settings on each of your project, it is possible to configure them once in a file
and have your projects inherit these values, unless the project overrides itself the setting
inside shotgun.yml
.
Here’s an example:
# Login related settings # [Login] # If specified, the username text input on the login screen will be populated # with this value when logging into Toolkit for the very first time. # Defaults to the user's OS login. Environment variables are actually resolved for # all values in this file, which allows greater flexibility when sharing this configuration # file with multiple users. # default_login=$USERNAME # If specified, the site text input on the login screen will be populated with # this value when logging into Toolkit the very first time. # Defaults to https://mystudio.shotgunstudio.com. # default_site=https://your-site-here.shotgunstudio.com # If specified, the Toolkit will use these proxy settings to connect to # the Flow Production Tracking site and the Toolkit App Store. The proxy string should be of the # forms 123.123.123.123, 123.123.123.123:8888 or # username:pass@123.123.123.123:8888. # Empty by default. # http_proxy=123.234.345.456:8888 # If specified, the Flow Production Tracking API will use these proxy settings to connect # to the Toolkit App Store. The proxy string format is the same as http_proxy. # If the setting is present in the file but not set, then no proxy will be used # to connect to the Toolkit App Store, regardless of the value of the http_proxy # setting. # Empty by default. # app_store_http_proxy=123.234.345.456:8888
This file can be configured through multiple means and Toolkit will try to resolve the file in the following order:
The
SGTK_PREFERENCES_LOCATION
environment variable,The
SGTK_DESKTOP_CONFIG_LOCATION
environment variable, for compatibility with the Flow Production Tracking. (deprecated)Inside the Flow Production Tracking preferences file
Inside the Flow Production Tracking preferences file, for compatibility with the Flow Production Tracking. (deprecated)
Note
The Flow Production Tracking preferences file is located at:
Windows:
%APPDATA%\Shotgun\Preferences\toolkit.ini
macOS:
~/Library/Preferences/Shotgun/toolkit.ini
Linux:
~/.shotgun/preferences/toolkit.ini
The Flow Production Tracking preferences file is located at:
Windows:
%APPDATA%\Shotgun\desktop\config\config.ini
macOS:
~/Library/Caches/Shotgun/desktop/config/config.ini
Linux:
~/.shotgun/desktop/config/config.ini
Note that the SHOTGUN_HOME
environment variable can impact the location
of the Flow Production Tracking preferences file.
Note
When the http proxy is not specified in this file, the Flow Production Tracking will try to retrieve the operating system http proxy.
First, the environment will be scanned for variables named http_proxy
, in case insensitive way.
If both lowercase and uppercase environment variables exist (and disagree), lowercase will be preferred.
When such environment variables cannot be found:
for Mac OS X, proxy information will be looked for from Mac OS X System Configuration,
for Windows, proxy information will be looked for from Windows Systems Registry.
There is a restriction in these latter cases: the use of proxies which require authentication (username and password) is not supported.
Internally, the Flow Production Tracking uses Python function urllib.getproxies()
to retrieve
the operating system http proxy. More information about this function can be found here:
You can access those values programmatically.
- class sgtk.util.UserSettings(*args, **kwargs)[source]
Handles finding and loading the user settings for Toolkit. The settings are cached in memory so the user settings object can be instantiated multiple times without any issue.
All the settings are returned as strings. If a setting is missing from the file,
None
will be returned. If the setting is present but has no value, an empty string will be returned.As of this writing, settings can only be updated by editing the
ini
file manually.Create the singleton instance if it hasn’t been created already. Once instantiated, the object will be cached and never be instantiated again for performance reasons.
- property shotgun_proxy
Retrieves the value from the
http_proxy
setting.
- property app_store_proxy
Retrieves the value from the
app_store_http_proxy
setting.
- property default_site
Retrieves the value from the
default_site
setting.
- property default_login
Retrieves the value from the
default_login
setting.
- get_section_settings(section)[source]
Retrieves the name of the settings in a given section.
- Parameters:
section (str) – Name of the section of the settings to retrieve.
- Returns:
A list of setting’s name. If the section is missing, returns
None
.
- get_setting(section, name)[source]
Provides access to any setting, including ones in user defined sections.
- Parameters:
- Returns:
The setting’s value if found,
None
if the setting is missing from the file or an empty string if the setting is present but has no value associated.- Return type:
- get_boolean_setting(section, name)[source]
Provides access to any setting, including ones in user defined sections, and casts it into a boolean.
Values
1
,yes
,true
andon
are converted toTrue
while0
,no
,false``and ``off
are converted to false. Case is insensitive.- Parameters:
- Returns:
Boolean if the value is valid, None if not set.
- Return type:
- Raises:
TankError – Raised if the value is not one of the accepted values.
- get_integer_setting(section, name)[source]
Provides access to any setting, including ones in user defined sections, and casts it into an integer.
- Parameters:
- Returns:
Boolean if the value is valid, None if not set.
- Return type:
- Raises:
TankError – Raised if the value is not one of the accepted values.
File System Utilities
Below is a collection of file system related convenience methods that make it easy to manage and create files and folders on disk in a standard fashion.
Note
These methods are not configurable or overridable
by hooks. If you are developing apps or engines, we
recommend using ensure_folder_exists()
as this method calls out to a customizable hook implementation.
sgtk.util.filesystem
- sgtk.util.filesystem.with_cleared_umask(func)[source]
Decorator which clears the umask for a method.
The umask is a permissions mask that gets applied whenever new files or folders are created. For I/O methods that have a permissions parameter, it is important that the umask is cleared prior to execution, otherwise the default umask may alter the resulting permissions, for example:
def create_folders(path, permissions=0777): log.debug("Creating folder %s..." % path) os.makedirs(path, permissions)
The 0777 permissions indicate that we want folders to be completely open for all users (a+rwx). However, the umask overrides this, so if the umask for example is set to 0777, meaning that I/O operations are not allowed to create files that are readable, executable or writable for users, groups or others, the resulting permissions on folders created by create folders will be 0, despite passing in 0777 permissions.
By adding this decorator to the method, we temporarily reset the umask to 0, thereby giving full control to any permissions operation to take place without any restriction by the umask:
@with_cleared_umask def create_folders(path, permissions=0777): # Creates folders with the given permissions, # regardless of umask setting. log.debug("Creating folder %s..." % path) os.makedirs(path, permissions)
- sgtk.util.filesystem.compute_folder_size(path)[source]
Computes and returns the size of the given folder.
- Parameters:
path – folder to compute size for
- Returns:
size in bytes
- sgtk.util.filesystem.touch_file(path, permissions=0666)[source]
Touch a file and optionally set its permissions.
- Parameters:
path – path to touch
permissions – Optional permissions to set on the file. Default value is 0666, creating a file that is readable and writable for all users.
- Raises:
OSError - if there was a problem reading/writing the file
- sgtk.util.filesystem.ensure_folder_exists(path, permissions=0775, create_placeholder_file=False)[source]
Helper method - creates a folder and parent folders if such do not already exist.
- Parameters:
path – path to create
permissions – Permissions to use when folder is created
create_placeholder_file – If true, a placeholder file will be generated.
- Raises:
OSError - if there was a problem creating the folder
- sgtk.util.filesystem.copy_file(src, dst, permissions=0666)[source]
Copy file and sets its permissions.
- Parameters:
src – Source file
dst – Target destination
permissions – Permissions to use for target file. Default permissions will be readable and writable for all users.
- sgtk.util.filesystem.safe_delete_file(path)[source]
Deletes the given file if it exists.
Ignores any errors raised in the process and logs them as warnings. If the user does not have sufficient permissions to remove the file, nothing will happen, it will simply be skipped over.
- Parameters:
path – Full path to file to remove
- sgtk.util.filesystem.safe_delete_folder(path)[source]
Deletes a folder and all of its contents recursively, even if it has read-only items.
Note
Problems deleting any items will be reported as warnings in the log output but otherwise ignored and skipped; meaning the function will continue deleting as much as it can.
- Parameters:
path – File system path to location to the folder to be deleted
- sgtk.util.filesystem.copy_folder(src, dst, folder_permissions=0775, skip_list=None)[source]
Alternative implementation to
shutil.copytree
Copies recursively and creates folders if they don’t already exist. Always skips system files such as
"__MACOSX"
,".DS_Store"
, etc. Files will the extension.sh
,.bat
or.exe
will be given executable permissions.Returns a list of files that were copied.
- Parameters:
src – Source path to copy from
dst – Destination to copy to
folder_permissions – permissions to use for new folders
skip_list – List of file names to skip. If this parameter is omitted or set to None, common files such as
.git
,.gitignore
etc will be ignored.
- Returns:
List of files copied
- sgtk.util.filesystem.move_folder(src, dst, folder_permissions=0775)[source]
Moves a directory.
First copies all content into target. Then deletes all content from sources. Skips files that won’t delete.
Note
The source folder itself is not deleted, it is just emptied, if possible.
- Parameters:
src – Source path to copy from
dst – Destination to copy to
folder_permissions – permissions to use for new folders
- sgtk.util.filesystem.backup_folder(src, dst=None)[source]
Moves the given directory into a backup location.
By default, the folder will be renamed by simply giving it a timestamp based suffix. Optionally, it can be moved into a different location.
backup_folder("/foo/bar")
will move/foo/bar
to/foo/bar.20160912_200426
backup_folder("/foo/bar", "/tmp")
will move/foo/bar
to/tmp/bar.20160912_200426
- Parameters:
src – Folder to move
dst – Optional backup folder
- sgtk.util.filesystem.create_valid_filename(value)[source]
Create a sanitized file name given a string. Replaces spaces and other characters with underscores
‘my lovely name ‘ -> ‘my_lovely_name’
- Parameters:
value – String value to sanitize
- Returns:
sanitized string
- sgtk.util.filesystem.get_unused_path(base_path)[source]
Return an unused file path from the given base path by appending if needed a number at the end of the basename of the path, right before the first “.”, if any.
For example,
/tmp/foo_1.bar.blah
would be returned for/tmp/foo.bar.blah
if it already exists.If the given path does not exist, the original path is returned.
Note
The returned path is not _reserved_, so it is possible that other processes could create the returned path before it is used by the caller.
- Parameters:
base_path (str) – Target path.
- Returns:
A string.
sgtk.util.json
- sgtk.util.json.load(fp, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, **kw)[source]
Deserialize
fp
(a.read()
-supporting file-like object containing a JSON document) to a Python object.This method is a simple thin wrapper around
json.load()
that ensures unserialized strings are utf-8 encodedstr
objects.See the documentation for
json.load()
to learn more about this method.
- sgtk.util.json.loads(s, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, **kw)[source]
Deserialize
s
(astr
orunicode
instance containing a JSON document) to a Python object.This method is a simple thin wrapper around
json.loads()
that ensures unserialized strings are utf-8 encodedstr
objects.See the documentation for
json.loads()
to learn more about this method.
sgtk.util.pickle
Toolkit’s pickle
module isn’t a drop-in replacement for Python’s, but a wrapper around Python’s pickle
module so collections and scalar types can be exchanged freely between Python 2 and Python 3 processes without having to worry about the subtle pickle serialization differences between the two.
If you wish to serialize your own custom classes to be exchanged between Python 2 and Python 3, you will need to sanitize data unpickled with Python 2 that was pickled with Python 3 and vice-versa. Otheriwse, your data will contain unexpected bytes
or unicode
objects instead of utf-8 encoded str
instances.
Note
In the load
/loads
methods, Python 3’s bytes
is always decoded back to a utf-8 encoded str
. If you need to store arbitrary binary data, consider saving it as a base64 string instead.
- sgtk.util.pickle.dump(data, fh)[source]
Write the pickled representation of
data
to a file object.This methods wraps the functionality from the
pickle.dump()
method so pickles can be shared between Python 2 and Python 3.- Parameters:
data – The object to pickle and store.
fh – A file object
- sgtk.util.pickle.dumps(data)[source]
Return the pickled representation of
data
as astr
.This methods wraps the functionality from the
pickle.dumps()
method so pickles can be shared between Python 2 and Python 3.As opposed to the Python 3 implementation, it will return a
str
object and notbytes
object.- Parameters:
data – The object to pickle and store.
- Returns:
A pickled str of the input object.
- Return type:
- sgtk.util.pickle.load(fh)[source]
Read the pickled representation of an object from the open file object and return the reconstituted object hierarchy specified therein.
This method wraps the functionality from the
pickle.load()
method so unicode strings are always returned as utf8-encodedstr
instead ofunicode
objects in Python 2.- Parameters:
fh – A file object
- Returns:
The unpickled object.
- Return type:
- sgtk.util.pickle.loads(data)[source]
Read the pickled representation of an object from a string and return the reconstituted object hierarchy specified therein.
This method wraps the functionality from the
pickle.loads()
method so unicode strings are always returned as utf8-encodedstr
instead ofunicode
objects in Python 2.
ShotgunPath
- class sgtk.util.ShotgunPath(windows_path=None, linux_path=None, macosx_path=None)[source]
Helper class that handles a path on multiple operating systems.
Contains methods to easily cast multi-os path between shotgun and os representations and mappings. The ShotgunPath object automatically sanitizes any path that it is given. When working with local storages in Shotgun, roots are keyed by the tokens
windows_path
,linux_path
andmac_path
. When usingsys.platform
in python, you get backwin32
,darwin
andlinux2
depending on platform. This class makes it easy to perform operations and cast between representations and platforms.Usage example:
>>> ShotgunPath.SHOTGUN_PATH_FIELDS ["windows_path", "linux_path", "mac_path"] # construction >>> p = ShotgunPath("C:\temp", "/tmp", "/tmp") >>> p = ShotgunPath.from_shotgun_dict({ "windows_path": "C:\temp", "mac_path": None, "linux_path": "/tmp"}) >>> p = ShotgunPath.from_system_dict({ "win32": "C:\temp", "darwin": None, "linux2": "/tmp"}) >>> p = ShotgunPath.from_current_os_path("/tmp") # access >>> p.macosx None >>> p.windows "C:\temp" >>> p.linux '/tmp >>> p.current_os '/tmp' # boolean operations >>> if p: print "a path value defined for windows, linux or mac" # equality >>> if p1 == p2: print "paths are same" # multi-platform access >>> p.as_shotgun_dict() { "windows_path": "C:\temp", "mac_path": None, "linux_path": "/tmp"} >>> p.as_system_dict() { "win32": "C:\temp", "darwin": None, "linux2": "/tmp"} # descriptor uri conversion >>> p.as_descriptor_uri() 'sgtk:descriptor:path?linux_path=/tmp/foo' # path manipulation >>> p2 = p.join('foo') >>> p2 <Path win:'c:\temp\foo', linux:'/tmp/foo', macosx:'/tmp/foo'>
- Parameters:
windows_path – Path on windows to associate with this path object
linux_path – Path on linux to associate with this path object
macosx_path – Path on macosx to associate with this path object
- SHOTGUN_PATH_FIELDS = ['windows_path', 'linux_path', 'mac_path']
A list of the standard path fields used by Shotgun.
- static get_file_name_from_template(template, platform='linux')[source]
Returns the complete file name for the current platform based on file name template passed in.
- Parameters:
template (str) – Template for a file name with a
%s
to indicate where the platform name should be inserted.- Returns:
Path with the OS name substituted in.
- static get_shotgun_storage_key(platform='linux')[source]
Given a
sys.platform
constant, resolve a Shotgun storage keyShotgun local storages handle operating systems using the three keys ‘windows_path, ‘mac_path’ and ‘linux_path’, also defined as
ShotgunPath.SHOTGUN_PATH_FIELDS
This method resolves the right key given a std. python sys.platform:
>>> p.get_shotgun_storage_key('win32') 'windows_path' # if running on a mac >>> p.get_shotgun_storage_key() 'mac_path'
- Parameters:
platform – sys.platform style string, e.g ‘linux2’, ‘win32’ or ‘darwin’.
- Returns:
Shotgun storage path as string.
- classmethod from_shotgun_dict(sg_dict)[source]
Creates a path from data contained in a std shotgun data dict, containing the paths windows_path, mac_path and linux_path
- Parameters:
sg_dict – Shotgun query resultset with possible keys windows_path, mac_path and linux_path.
- Returns:
ShotgunPath
instance
- classmethod from_system_dict(system_dict)[source]
Creates a path from data contained in a dictionary keyed by sys.platform constants.
- Parameters:
system_dict – Dictionary with possible keys win32, darwin and linux2.
- Returns:
ShotgunPath
instance
- classmethod from_current_os_path(path)[source]
Creates a path object for a path on the current platform only.
- Parameters:
path – Path on the current os platform.
- Returns:
ShotgunPath
instance
- classmethod normalize(path)[source]
Convenience method that normalizes the given path by running it through the
ShotgunPath
normalization logic.ShotgunPath.normalize(path)
is equivalent to executingShotgunPath.from_current_os_path(path).current_os
.Normalization include checking that separators are matching the current operating system, removal of trailing separators and removal of double separators. This is done automatically for all
ShotgunPath
, but sometimes it is useful to just perform the normalization quickly on a local path.- Parameters:
path (str) – Local operating system path to normalize
- Returns:
Normalized path string.
- property macosx
The macosx representation of the path
- property windows
The Windows representation of the path
- property linux
The Linux representation of the path
- property current_os
The path on the current os
- as_shotgun_dict(include_empty=True)[source]
The path as a shotgun dictionary. With
include_empty
set to True:{ "windows_path": "C:\temp", "mac_path": None, "linux_path": "/tmp"}
With
include_empty
set to False:{ "windows_path": "C:\temp", "linux_path": "/tmp"}
- Parameters:
include_empty – Controls whether keys should be included for empty path values
- Returns:
dictionary of paths keyed by standard shotgun keys.
- as_system_dict(include_empty=True)[source]
The path as a dictionary keyed by sys.platform.
With
include_empty
set to True:{ "win32": "C:\temp", "darwin": None, "linux2": "/tmp"}
With
include_empty
set to False:{ "win32": "C:\temp", "linux2": "/tmp"}
- Parameters:
include_empty – Controls whether keys should be included for empty path values
- Returns:
dictionary of paths keyed by sys.platform.
- as_descriptor_uri(for_development=False)[source]
Translates the path to a descriptor uri. For more information about descriptors, see the reference documentation.
This method will either return a dev or a path descriptor uri path string, suitable for use with for example pipeline configurations in Shotgun.
- Parameters:
for_development (bool) – Set to true for a dev descriptor
- Returns:
Dev or Path descriptor uri string representing the path
- Raises:
ValueError if the path object has no paths defined
- join(folder)[source]
Appends a single folder to the path.
- Parameters:
folder – folder name as sting
- Returns:
ShotgunPath
object containing the new path
LocalFileStorageManager
- class sgtk.util.LocalFileStorageManager[source]
Class that encapsulates logic for resolving local storage paths.
Toolkit needs to store cache data, logs and other items at runtime. Some of this data is global, other is per site or per configuration.
This class provides a consistent and centralized interface for resolving such paths and also handles compatibility across generations of path standards if and when these change between releases.
Note
All paths returned by this class are local to the currently running user and typically private or with limited access settings for other users.
If the current user’s home directory is not an appropriate location to store your user files, you can use the
SHOTGUN_HOME
environment variable to override the root location of the files. In that case, the location for the user files on each platform will be:Logging:
$SHOTGUN_HOME/logs
Cache:
$SHOTGUN_HOME
Persistent:
$SHOTGUN_HOME/data
Preferences:
$SHOTGUN_HOME/preferences
- Constant CORE_V17:
Indicates compatibility with Core 0.17 or earlier
- Constant CORE_V18:
Indicates compatibility with Core 0.18 or later
- Constant LOGGING:
Indicates a path suitable for storing logs, useful for debugging
- Constant CACHE:
Indicates a path suitable for storing cache data that can be deleted without any loss of functionality or state.
- Constant PERSISTENT:
Indicates a path suitable for storing data that needs to be retained between sessions.
- Constant PREFERENCES:
Indicates a path that suitable for storing settings files and preferences.
- classmethod get_global_root(path_type, generation=1)[source]
Returns a generic Shotgun storage root.
The following paths will be used:
On the mac, paths will point into
~/Library/PATH_TYPE/Shotgun
, where PATH_TYPE is controlled by the path_type property.On Windows, paths will created below a
%APPDATA%/Shotgun
root point.On Linux, paths will be created below a
~/.shotgun
root point.
Note
This method does not ensure that the folder exists.
- Parameters:
path_type – Type of path to return. One of
LocalFileStorageManager.LOGGING
,LocalFileStorageManager.CACHE
,LocalFileStorageManager.PERSISTENT
, where logging is a path where log- and debug related data should be stored, cache is a location intended for cache data, e.g. data that can be deleted without affecting the state of execution, and persistent is a location intended for data that is meant to be persist. This includes things like settings and preferences.generation – Path standard generation to use. Defaults to
LocalFileStorageManager.CORE_V18
, which is the current generation of paths.
- Returns:
Path as string
- classmethod get_site_root(hostname, path_type, generation=1)[source]
Returns a cache root where items can be stored on a per site basis.
For more details, see
LocalFileStorageManager.get_global_root()
.Note
This method does not ensure that the folder exists.
- Parameters:
hostname – Shotgun hostname as string, e.g. ‘https://foo.shotgunstudio.com’
path_type – Type of path to return. One of
LocalFileStorageManager.LOGGING
,LocalFileStorageManager.CACHE
,LocalFileStorageManager.PERSISTENT
, where logging is a path where log- and debug related data should be stored, cache is a location intended for cache data, e.g. data that can be deleted without affecting the state of execution, and persistent is a location intended for data that is meant to be persist. This includes things like settings and preferences.generation – Path standard generation to use. Defaults to
LocalFileStorageManager.CORE_V18
, which is the current generation of paths.
- Returns:
Path as string
- classmethod get_configuration_root(hostname, project_id, plugin_id, pipeline_config_id, path_type, generation=1)[source]
Returns the storage root for any data that is project and config specific.
A well defined project id should always be passed. Passing None as the project id indicates that the site configuration, a special toolkit configuration that represents the non-project state in Shotgun.
Configurations that have a pipeline configuration in Shotgun should pass in a pipeline configuration id. When a pipeline configuration is not registered in Shotgun, this value should be None.
If the configuration has been bootstrapped or has a known plugin id, this should be specified via the plugin id parameter.
For more details, see
LocalFileStorageManager.get_global_root()
.Examples of paths that will be generated:
Site config:
ROOT/shotgunsite/p0
Project 123, config 33:
ROOT/shotgunsite/p123c33
project 123, no config, plugin id review.rv:
ROOT/shotgunsite/p123.review.rv
Note
This method does not ensure that the folder exists.
- Parameters:
hostname – Shotgun hostname as string, e.g. ‘https://foo.shotgunstudio.com’
project_id – Shotgun project id as integer. For the site config, this should be None.
plugin_id – Plugin id string to identify the scope for a particular plugin or integration. For more information, see
plugin_id()
. For non-plugin based toolkit projects, this value is None.pipeline_config_id – Shotgun pipeline config id. None for bootstraped configs.
path_type – Type of path to return. One of
LocalFileStorageManager.LOGGING
,LocalFileStorageManager.CACHE
,LocalFileStorageManager.PERSISTENT
, where logging is a path where log- and debug related data should be stored, cache is a location intended for cache data, e.g. data that can be deleted without affecting the state of execution, and persistent is a location intended for data that is meant to be persist. This includes things like settings and preferences.generation – Path standard generation to use. Defaults to
LocalFileStorageManager.CORE_V18
, which is the current generation of paths.
- Returns:
Path as string
OS detection
Below are a collection of convenience methods to detect which operating system is in use:
- sgtk.util.is_linux(platform=None)[source]
Determine if the current platform is Linux.
- Parameters:
platform – sys.platform style string, e.g ‘linux2’, ‘win32’ or ‘darwin’. If not provided, sys.platform will be used.
- Returns:
True if the current platform is Linux, otherwise False.
- Return type:
Miscellaneous
- sgtk.util.append_path_to_env_var(env_var_name, path)[source]
Append the path to the given environment variable. Creates the env var if it doesn’t exist already. will concatenate paths using : on linux and ; on windows
- sgtk.util.prepend_path_to_env_var(env_var_name, path)[source]
Prepend the path to the given environment variable. Creates the env var if it doesn’t exist already. will concatenate paths using : on linux and ; on windows
- sgtk.util.get_current_user(tk)[source]
Retrieves the current user as a dictionary of metadata values. Note: This method connects to shotgun the first time around. The result is then cached to reduce latency.
If a user has been authenticated via a login prompt, this method will return the credentials associated with that user. If Toolkit has been configured to use a script user to connect to Shotgun, a core hook will be executed to established which user is associated with the current session. This is usually based on the currently logged in user.
- Returns:
None if the user is not found in Shotgun. Otherwise, it returns a dictionary with the following fields: id, type, email, login, name, image, firstname, lastname
Exceptions
- class sgtk.util.EnvironmentVariableFileLookupError(var_name, path)[source]
Bases:
TankError
Raised when an environment variable specifying a location points to configuration file that doesn’t exist.
- Parameters:
- with_traceback()
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
- class sgtk.util.ShotgunPublishError(error_message, entity=None)[source]
Bases:
TankError
Raised when Toolkit is not able to register a published file in Shotgun.
The original message for the reported error is available in the ‘error_message’ property.
If a published file entity was created before the error happened, it will be available in the ‘entity’ property.
- Parameters:
- with_traceback()
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
- class sgtk.util.PublishResolveError[source]
Bases:
TankError
Base class for all errors relating to resolution of paths from publishes.
- with_traceback()
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
- class sgtk.util.PublishPathNotDefinedError[source]
Bases:
PublishResolveError
Exception raised when a publish does not have a path defined for the current operating system platform. It may or may not have publish paths defined on other platforms.
- with_traceback()
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
- class sgtk.util.PublishPathNotSupported[source]
Bases:
PublishResolveError
Exception raised when a publish has a path defined but it is using a path definition that cannot be resolved into a local path. This includes for example unsupported url schemes.
- with_traceback()
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.